The teams faced a shootout after Colombia's Yerry Mina scored one of his signature headed goals in stoppage time to cancel out England captain Harry Kane's 57th minute penalty and the score was still level at 1-1 after extra time.
However, Los Cafeteros hit back through Yerry Mina's last-gasp equaliser to force extra time.
England goalkeeper Jordan Pickford said "we knew we were capable of winning" on penalties as his shootout save boosted his side over Colombia and into the World Cup quarterfinals, ending their torturous history from the spot.
The win sees England move on to the quarters where it will face Sweden, while Colombia goes home. Colombia, meanwhile, topped Group H with a win over Senegal on the final day of group stage play but are playing with a less-than-100-percent James Rodriguez.
Fast forward 12 years and, although this wasn't quite as straightforward, England have now put an end to an unwanted record.
But more than ever when it then fell to Eric Dier to finish the job.
It's too early to suggest this historical event will lead to World Cup success, as the team showed many serious issues we'll get to in a bit.
Ronaldo can't beat Uruguay on his own, warns Portugal coach
They're fortunate to have advanced and need to find a way to capitalize on scoring opportunities to get by a tough Uruguay team. He required assistance to make it off the field, provided by Ronaldo, who aided his opponent all the way to the touchline.
All that preparation, and all that meticulous planning, not just when it came to how England would play but how they would cope inside the pressure cooker of a World Cup knock-out game.
On paper, England's tilt with Colombia at Moscow's intimate Spartak Stadium is more or less a wash - England is 12th in the Federation Internationale de Football Association rankings and Colombia is 16th, not that the rankings have meant much in an upset-heavy World Cup that's already seen Germany, Portugal, Argentina and Spain eliminated.
Will Colombia beat England for the first time? The last time they failed to find the net was in a 2-0 loss to England in the 1998 edition. We've not got a good record against them, we've always underestimated them, they have good players and a clear way of playing, and it's bloody hard to play against.
Colombia's first sniff at goal came to Johan Mojica, who blazed high and wide from distance on the left six minutes later, while Juan Quintero dragged an effort wide of the right post from 22 yards out on 33 minutes after being teed up by Radamel Falcao.
Fox (live in most markets) and Telemundo are both included in the "Fubo Premier" bundle, which has a channel package that is largely tailored towards worldwide soccer fans and also has World Cup live streams available in 4K. England's Jordan Pickford made a massive save on Carlos Bacca and Mateus Uribe was denied by the crossbar in the shootout.
If they make it past Sweden and into the semifinals, then assume Croatia will be England's most likely opposition.