Maldives President Mohamed Nasheed has resigned after weeks of demonstrations and a mutiny by some police officers.
Vice-President Waheed Hassan was sworn in vowing to uphold the "rule of law".
Mr Nasheed is being held against his will by security forces at the presidential palace in the capital, his brother told the BBC.
Tensions escalated after the Maldives army arrested a senior judge last month, prompting bitter street protests in the Indian Ocean island chain.
"Security forces say they can't release him, he wants to come home," Mr Nasheed's brother, Ibrahim Nasheed, told the BBC, adding the security forces said there were concerns for his safety if he was released.
Earlier, announcing his resignation on state television, Mr Nasheed said it would be "better for the country in the current situation" if he stood down.
A source close to the the country's first democratically elected president described Tuesday's developments as a "coup by the former regime".
The army and the vice-president deny the claims.
"It was not a coup at all, it was the wish of the people,'' Ahmed Thoufeeg, the vice president's secretary, told AP news agency. Mr Hassan's office has denied widespread reports that the military put pressure on Mr Nasheed to resign.
The British Foreign Office has advised against "all but essential travel" to the capital Male, where a third of the population live, but said it had received no reports of the unrest affecting the airport or tourist areas.
'No iron fist'
Mr Nasheed announced his resignation during a televised news conference.
"It will be better for the country in the current situation if I resign. I don't want to run the country with an iron fist. I am resigning," Mr Nasheed said.
Earlier, a group of mutinying police officers took control of the state broadcaster in the capital, Male, and began playing out messages in support of former President Maumoon Abdul Gayoom.
Several journalists were said to be detained in the facility.
Sources in the office of Mr Nasheed told the BBC Tuesday's protest took place in front of military headquarters, a high-security zone.
Soldiers used tear gas to break up a demonstration by supporters of Mr Gayoom.
On Monday, around 50 policemen stood down in favour of the protesters and refused to obey orders. The president's office denied reports that the army fired rubber bullets at the protesting police officers.
Sarah Harvey, a Briton who works in Male, told the BBC: "Last night was really noisy - there were protesters on foot and on scooters storming down the streets and chanting until the early hours of the morning.
"[Today] there's a curfew in the streets, but a few people are out and about," she added.
"I've heard that [ruling party] MDP activists were encouraging people to come out on the streets tonight to protest against the attempted coup."
Tensions rose significantly last month when the army arrested a senior criminal court judge, Abdulla Mohamed.
The government alleged that the judge's rulings - such as the release of an opposition activist detained without a warrant - were politically motivated.
It claimed the dispute with the judge was not an isolated incident, but indicative of a more deep-rooted problem with the Maldives judicial system and the checks and balances it has to ensure it stays independent.
Human rights groups added their voices to calls for the judge to be released - and, as matters grew increasingly heated, there were demands for the United Nations to be brought in to resolve the dispute.
Mr Nasheed was elected on a wave of optimism in 2008, in the islands' first multi-party election.
Mr Nasheed, a former human rights campaigner, beat long-time ruler Mr Gayoom, who had been in power for 30 years and was widely seen as autocratic.
Since then, correspondents say, the country has been gripped by constitutional gridlock - parties opposed to the president have dominated parliament since general elections the following year.
A one-time political prisoner, Mr Nasheed became a vocal figure in office on issues relating to the environment and climate change.
But he has faced constant opposition - from those loyal to former President Gayoom and from religious conservatives who accuse him of being anti-Islamic, says the BBC's South Asia analyst Jill McGivering.
That pressure has intensified with the prospect of fresh presidential elections, scheduled for next year. Opposition parties are jockeying for power as they try to extend their influence.
The wider question is how this crisis will affect the forthcoming elections - and what it says about the transition in the Maldives to mature democracy, our correspondent says.