MP and former Attorney General Dominic Grieve has lost a vote of no-confidence by local Conservatives.
It followed a "robust discussion" about Brexit by party association members in Beaconsfield, Buckinghamshire.
Mr Grieve has been a prominent campaigner for another referendum on the UK leaving the European Union.
He said he had no intention of standing down but would now need to ask the local association to reselect him as a candidate in the next general election.
Conservative Party chairman Brandon Lewis said the vote - passed by 182 to 131 - had no formal standing under party rules.
He said: "Dominic is somebody who has contributed to Parliament, is a clear strong Conservative and he is an asset to the party."
Opposition to him is reported to have been organised by a former UKIP candidate, and more than 38,000 people have signed an online petition calling for his de-selection.
Beaconsfield Constituency Conservative Association chairman Jackson Ng said: "Our members had a robust discussion with Mr Grieve on Brexit before voting on a motion of confidence in him as our MP, which, I can confirm with a heavy heart that he failed to retain.
"He remains our Conservative MP but I will be speaking as soon as possible to my fellow officers and the executive council."
It came just hours after the Prime Minister's Brexit deal was voted down by MPs for the third time.
A number of Conservative MPs swiftly criticised the no confidence vote.
Former Tory MP Anna Soubry, who defected to join the Independent Group, branded the vote "disgraceful".
Labour MP Stella Creasy said: "We disagree on many things. I doubt Beaconsfield conservatives could find someone more experienced, more capable and more principled than him for their MP."