David Cameron has dismissed the defection of Conservative MP Mark Reckless to UKIP as "senseless and counter-productive".
Mr Reckless's move was followed by the resignation of a minister over a newspaper allegation, in a double blow for the PM at the start of the Tory conference.
Mr Cameron told the BBC Mr Reckless's defection was "frustrating".
And only a Tory government could deliver an EU referendum.
In an echo of the statement he made when another of his backbenchers, Douglas Carswell, defected to UKIP earlier this month, Mr Cameron said the choice facing voters at the next election was between a Conservative and Labour government.
"So to act in a way that makes the Conservative government less likely is senseless and counter-productive."
The Conservative leader said he had not been aware of Mr Reckless's plans to quit but suggested the backbencher - who has triggered a by-election in his Rochester and Strood constituency - would not be much of a loss to the party because he "rarely voted with the government".
UKIP leader Nigel Farage told Andrew Marr it "wouldn't surprise" him to see more defections, and that "Labour backbenchers are thinking about this as well".
Meanwhile, former civil society minister Brooks Newmark, who has quit over reports that he sent an explicit image of himself to an undercover reporter, told the BBC he had been a "complete fool"
Benefit cap plan
Mr Cameron wants to use his week in Birmingham to unveil a series of eye-catching policies aimed at winning the party an overall majority in May's election.
Chancellor George Osborne has said a Conservative government would cut the maximum level of benefits a household can claim from £26,000 a year to £23,000.
At the same time, unemployed young people aged 18 to 21 would be given six months to find work or training - after which their jobseeker's allowance payments would be withdrawn unless they agreed to take part in "community projects" such as cleaning up local parks.
Most unemployed 18 to 21-year-olds would also be prevented from claiming housing benefit in order to leave home under the Tory proposals.
Mr Cameron also promised help for young people to get on the housing ladder, saying a Tory government would build 100,000 homes exclusively available to under-40s in England at 20% below market rate.
But the Conservative leader has begun his final conference before the general election by defending his position on Europe amid ongoing unease among right wingers.
In an interview on the BBC Andrew Marr show, he promised to focus on tightening immigration rules when he attempted to renegotiate Britain's relationship with Brussels before the referendum promised by the end of 2017.
He stopped short of saying he would recommend Britain leave the EU - and vote no in the referendum - if he failed to get the concessions he wants from the other EU nations - something his Eurosceptic MPs have been demanding.
"If I don't achieve that it will be for the British public to decide whether to stay in or get out," he said.
But he added: "I have said this all my political life: if I thought that it wasn't in Britain's interest to be in the European Union, I wouldn't argue for us to be in it."
On Saturday, Culture Secretary Sajid Javid told the Daily Mail the UK could still prosper if it chose to exit the EU.
"I think it would open up opportunities. I am not afraid of that at all," he added.
Mr Reckless told rapturous delegates at UKIP's conference in Doncaster on Saturday: "People feel ignored, taken for granted, over-taxed, over-regulated, ripped off and lied to."
Meanwhile, Conservative peer Lord Ashcroft called on the prime minister to do more to win over undecided voters.
"Starting today in Birmingham, Cameron must help them resolve their quandary in his party's favour," he wrote in the Sunday Times.
"Cameron must show that his purpose is not merely a matter of dry economics but the creation of a country where everyone can share in the prosperity that the Tories are accused of wanting to preserve for the few."
The prime minister is due to deliver his set-piece speech on Wednesday as he closes the conference.