The Liberal Democrats have won the Eastleigh by-election, with the UK Independence Party pushing the Conservatives into third place.
Leader Nick Clegg said the party's candidate Mike Thornton had pulled off a "stunning victory" which had been secured "against the odds".
UKIP leader Nigel Farage said its best-ever performance in a Westminster poll showed it had "connected with voters".
David Cameron said the Tories would recover from a "disappointing" result.
The by-election was called after former Lib Dem cabinet minister Chris Huhne resigned as an MP following an admission he had perverted the course of justice over driving licence points.
But the party, which has held Eastleigh since another by-election in 1994, won despite a fall in its share of the vote of more than 14 percentage points since the 2010 general election.
UKIP candidate Diane James got 11,571 votes, on a 19.3% swing from the Lib Dems.
Conservative Maria Hutchings won 10,559 votes, representing a 14 percentage points fall in her share of the vote since the general election, when she came second to Mr Huhne.
Labour's John O'Farrell was fourth with 4,088 votes.
Lib Dem Mr Thornton, who has been a parish and borough councillor since 2007, said: "The people of Eastleigh recognise that the Liberal Democrats have always had a superb record of delivery, we've always listened to what people want, and we always make sure that we do a good job."
At a victory celebration in the constituency, Mr Clegg said the party had won in the "most exceptionally difficult circumstances" - given the manner of Huhne's departure and allegations surrounding the party's former chief executive Lord Rennard.
"We held our nerve. We stood our ground... We overcame the odds with a stunning victory," he said, adding that the result proved the Lib Dems "can be a party of government and still win".
UKIP's Nigel Farage said the surge in support for his party was not a "freak result" but a continuation of a trend which had seen it rise in the national polls.
"We have really connected with voters in this constituency," he told the BBC. "And that is because we are talking about issues that the other parties would like to brush under the carpet."
Prime Minister and Conservative leader David Cameron said his party would not "change tack" on the economy, immigration or welfare in response to the poor performance.
"This is a by-election. It's mid-term. It's a protest. That's what happens in by-elections," he said.
"It's disappointing for the Conservative Party but we must remain true to our principles, true to our course, and that way we can win people back."
Labour leader Ed Miliband said he would have preferred to have done better but it was "tough" territory for his party as it had never come close to winning Eastleigh before, even in its 1997 landslide election year.
The result, he added, showed Labour needed to "redouble its efforts" to increase its appeal to voters, in the south of England and elsewhere, who were not traditional supporters.
Turnout was 52.7%, down from 69.3% at the 2010 general election.
Results in full:
Mike Thornton (Liberal Democrat) 13,342 (32.06%, -14.48%)
Diane James (UKIP) 11,571 (27.80%, +24.20%)
Maria Hutchings (Conservative) 10,559 (25.37%, -13.96%)
John O'Farrell (Labour) 4,088 (9.82%, +0.22%)
Danny Stupple (Independent) 768 (1.85%, +1.56%)
Dr Iain Maclennan (National Health Action Party) 392 (0.94%)
Ray Hall (Beer, Baccy and Crumpet Party) 235 (0.56%)
Kevin Milburn (Christian Party) 163 (0.39%)
Howling Laud Hope (Monster Raving Loony Party) 136 (0.33%)
Jim Duggan (Peace Party) 128 (0.31%)
David Bishop (Elvis Loves Pets) 72 (0.17%)
Michael Walters (English Democrats) 70 (0.17%, -0.30%)
Daz Procter (Trade Unionists and Socialists Against Cuts) 62 (0.15%)
Colin Bex (Wessex Regionalist) 30 (0.07%)