Libyan PM Ali Zeidan has been seized from his hotel in the capital, Tripoli by a former rebel militia loosely allied to the government.
The group said it arrested Mr Zeidan following a prosecutor's warrant, but the government has denied this.
An official said he was being held at an interior ministry anti-crime department and being "treated well".
There has been anger in Libya over a US commando raid on Saturday which seized senior al-Qaeda suspect Anas al-Liby.
Many saw the raid as a breach of Libyan sovereignty amid growing pressure on the government to explain if it was involved.
On Monday, Libya demanded an explanation from the US ambassador over the arrest of Mr Liby, who is wanted in the US over the 1998 bombings of US embassies in Kenya and Tanzania.
A number of militia groups operate in Libya - they are nominally attached to government ministries but often act independently and, correspondents say, often have the upper hand over police and army forces.
The government has been struggling to contain these militia, who control parts of the country, two years after the revolt which overthrew Muammar Gaddafi.
Mr Zeidan was taken by armed men from Tripoli's Corinthia Hotel - considered one of the more secure buildings in the city - in the early hours of Thursday morning.
The head of security at the hotel, Abd al-Razaq ben Shaban, told Reuters news agency "revolutionaries" had arrived carrying papers with an order from the prosecutor general for the arrest of the prime minister.
However, state-run National Libyan TV quoted Justice Minister Salah al-Marghani as saying that the prosecutor general had issued no warrant for Mr Zeidan's arrest.
The Revolutionaries Operations Room said it had seized Mr Zeidan, and was acting on the orders of the prosecutor general in accordance with Libya's criminal code.
A spokesman for the group told Reuters that they took Mr Zeidan following comments by US Secretary of State John Kerry that the Libyan government had been "aware of the operation".
The Libyan government said on its website earlier that Mr Zeidan had been taken from a hotel "to an unknown place for unknown reasons by a group thought to be from the Tripoli Revolutionaries Operations Room and the Committee for Fighting Crime".
Al-Arabiya TV station broadcast images on Thursday morning which showed Mr Zeidan looking dishevelled and being escorted by what the station said were armed men.
Libya's cabinet has been summoned for an immediate meeting under the leadership of the deputy prime minister.
UK Foreign Secretary William Hague condemned the capture and called for Mr Zeidan's immediate release.
"It is vital that the process of political transition in Libya is maintained. The government and people of Libya have our full support at this concerning time," he said.
US state department spokeswoman Jen Psaki said the US was looking into the reports and was "in close touch with senior US and Libyan officials on the ground".
'Act of sabotage'
Libyan politician Guma El Gamaty told the BBC that the US raid against Mr Liby on Saturday had sparked anger on the Libyan streets.
"Accusations have been pointed at the prime minister that there is some sort of a collusion - that the prime minister knew in advance that Abu Anas al-Liby was going to be kidnapped," he said.
In an interview with the BBC on Monday, Mr Zeidan had said Libya was being used as a base to export weapons throughout the region, and called on the West to help stop militancy in Libya.
Last month Mr Zeidan visited the UK and appealed for British help to remove weapons from the country amid fears of increased arms smuggling to Syria.
In April he urged Libyans to back their government in the face of "people who want to destabilise the country".
He also complained at that time of other attacks and "acts of sabotage" carried out by separate groups, against the interior ministry and national TV headquarters.