Nigel Farage has said UKIP can become a major force in Parliament at next year's election after its victory in the Rochester and Strood by-election.
UKIP's leader said Mark Reckless's win made him think the party could take twice as many seats as it had thought.
Mr Reckless gained 16,867 votes, 2,920 more than Conservative Kelly Tolhurst's 13,947, with Labour's Naushabah Khan on 6,713 - ahead of the Green Party.
The Lib Dems came fifth with their lowest vote in a by-election.
Mr Reckless, whose defection from the Tories to UKIP triggered the contest in Kent, travelled to London soon after his election to take his seat in Parliament, saying his constituents expected him "to get back to work".
As he was sworn in at the House of Commons, he was accompanied by UKIP's other MP Douglas Carswell, another former Conservative who won a by-election last month.
In other developments:
- Shadow attorney general Emily Thornberry resigned from the Labour front bench just hours before the result after being accused of "sneering" by tweeting a photo of a Rochester house with flags and a white van outside
- David Cameron said he was "absolutely determined" to win the seat back for the Conservatives in May's general election
- Ed Miliband said the result was a "devastating" setback for David Cameron and he was determined to meet the challenge posed by UKIP
- UKIP won its first seat on Medway Council after Conservative defector Chris Irvine was elected from the Peninsula ward.
In his acceptance speech early on Friday morning, Mr Reckless said Rochester and Strood had been UKIP's 271st most winnable seat and "if UKIP can win here, we can win across the country".
Speaking to the BBC's political editor Nick Robinson, Mr Reckless urged other Conservative MPs to join UKIP but said it was difficult to predict how many, if any, would do so because it was a "very individual decision".
The prime minister vowed to win the seat back in less than six months' time.
"This result was closer than the forecasters predicted," he said. "I am absolutely determined to win this seat back because absolutely anything other than a Conservative government will put our recovery at risk and mean Ed Miliband in Downing Street.
"I am more determined than ever that we deliver security for Britain."
UKIP leader Nigel Farage denied the result was simply a protest vote against the government, or that voters would return to the larger parties in the general election.
He said the message of Rochester and Strood was: "If you vote UKIP, you get UKIP" and "people out there who vote UKIP intend to stay UKIP".
He said he believed the number of seats that UKIP could win in May had "probably doubled".
By Norman Smith, BBC assistant political editor
UKIP's victory was in many ways even more impressive than their triumph in Clacton. The ease with which they demolished a 9,000 Tory majority was striking and this after the Conservatives had strained every sinew to halt the UKIP bandwagon.
UKIP now insists no Tory seat is safe and has suggested other Conservative MPs are more likely to defect.
For the Tories the result was not perhaps the meltdown they had feared and certainly there is no indication so far of panic or calls for Mr Cameron to go.
Senior Tories also believe they are poised to win back this seat at the general election.
For Labour, not only was their share of the vote almost halved - but there was also despair at the damaging tweet by their former shadow attorney general Emily Thornberry, seemingly mocking "white van man".
As for the Lib Dems, not only were they again overhauled by the Greens, but they secured a derisory 349 votes - their lowest total ever.
After it won May's European elections, UKIP said it would seriously target up to 20 seats in May but this could now be increased after the Rochester triumph.
After the Conservatives lost last month's Clacton by-election to UKIP, Mr Cameron promised to "throw the kitchen sink" at the Rochester campaign.
UKIP's winning margin was smaller than it was in Clacton, but Mr Reckless still won 42.1% of the votes cast, on a turnout of 50,67%.
Green Party candidate Clive Gregory came fourth with 1,692 votes, and the Liberal Democrats came fifth with 349.
Lib Dem Party President Tim Farron blamed the party's poor showing, its worst ever in a Westminster by-election, on tactical voting.
Elections expert Prof John Curtice, from Strathclyde University, said UKIP still faced a challenge to convert its by-election success into seats at the general election.
He said: "It's certainly a very good night for UKIP, and makes life much more difficult for the major parties, but the challenge facing UKIP is clear."
Nick Robinson said it was "extremely unlikely" any further Tory MPs would defect in the short term, wary of having to fight a by-election so close to a general election.
Asked whether he could guarantee this would not happen, Chief Whip Michael Gove told the BBC's PM programme he was "convinced that nobody will".
Describing Mr Cameron as "the standout politician of our age", he said he believed he would lead the Conservatives to victory at the next election.
Labour was dealt a blow when shadow attorney general Emily Thornberry resigned from its front bench after being accused of "snobbery" by tweeting a photograph of a house in Strood displaying England flags and with a white van parked outside.
The Islington South and Finsbury MP, who stood down after talks with Ed Miliband, said it had not been her intention to "upset or insult" anybody.
Shadow foreign secretary Douglas Alexander said Mr Miliband "did not hold back in making clear how angry he was" about the tweet.
He told the BBC that politicians could not take "any voter, community or class for granted" and must treat the public with a "deep and fundamental respect".
But Mr Farage suggested Labour was "increasingly anti-English".
"This Labour party or new Labour believes that any sense of English identity is disreputable and wrong," he said.
Rochester and Strood by-election full results
Mark Reckless (UKIP) 16,867 (42.10%)
Kelly Tolhurst (Conservative) 13,947 (34.81%)
Naushabah Khan (Labour) 6,713 (16.76%)
Clive Gregory (Green) 1,692 (4.22%)
Geoff Juby (Lib Dem) 349 (0.87%)
Hairy Knorm Davidson (Official Monster Raving Loony Party) 151 (0.38%)
Stephen Goldsbrough (Ind) 69 (0.17%)
Nick Long (People Before Profit) 69 (0.17%)
Jayda Fransen (Britain First) 56 (0.14%)
Mike Barker (Ind) 54 (0.13%)
Charlotte Rose (Ind) 43 (0.11%)
Dave Osborn (Patriotic Socialist Party) 33 (0.08%)
Christopher Challis (Ind) 22 (0.05%)