Justice Secretary David Gauke has survived a no-confidence vote by his constituency Conservative Association.
The South West Hertfordshire MP opposes a no-deal Brexit.
Political campaign group Leave.EU had tweeted it wanted to "claim our first cabinet scalp".
The no-confidence motion was defeated 123 to 61. Mr Guake told the BBC's Today programme the vote had been "hanging over him" but he had been "pretty confident" he would win.
"You never quite know who's going to turn up on the night so I'm very pleased to have won and won pretty comfortably," he said.
'Hell or high water'
The closed meeting was held at Kings Langley Community Centre and lasted for more than two and a half hours.
The justice secretary tweeted immediately after the result a "no deal would be immensely damaging to the UK" and told the Today programme that strictly enforcing a deadline for leaving the EU of 31 October would make a no-deal Brexit an "inevitable consequence".
He said: "I think the position that Boris Johnson is setting out is that the position of the government is that we leave the EU by 31 October with or without a deal, come hell or high water.
"I personally think we should try to avoid hell or high water, so I don't think I could serve [in the cabinet] in those circumstances."
Tonight, I argued that:— David Gauke (@DavidGauke) June 28, 2019
We should not allow the Party to be taken over by entryists.
We should be a broad church.
No deal would be immensely damaging to the UK.
I defeated a motion of no confidence 123 to 61. I am grateful to the members of my association for their support.
Mr Guake also called for his party to "be a broad church" and urged Conservative voters in support of a "pragmatic, moderate centre" to join the party.
"There are plenty of people who are Conservative voters who want to ensure that the party is that broad church and I would say, don't just roll your eyes at what happens in the party, sign up," he added.
Prior to the vote he received support from many of his cabinet colleagues, including Work and Pensions Secretary Amber Rudd who tweeted: "Tonight's motion of no confidence in David Gauke is ludicrous.
"Rounding on colleagues in this way is the type of behaviour you'd expect from the hard left."
Tonight’s motion of no confidence in @DavidGauke is ludicrous.— Amber Rudd MP (@AmberRuddHR) June 28, 2019
Rounding on colleagues in this way is the type of behaviour you’d expect from the hard left.
He has my full support, and I hope this is shared by @Jeremy_Hunt and @BorisJohnson.
Leave.EU replied to Ms Rudd saying: "Thanks for your feedback, Amber. You're next."
Thanks for your feedback, Amber. You're next. https://t.co/4F1w5c7uEm— Leave.EU (@LeaveEUOfficial) June 28, 2019
Energy Minister Claire Perry tweeted that the Leave.EU "hit list ignores the fact that MPs like David have voted repeatedly for a firm EU exit - if other MPs had joined us rather than indulging in their Brexit fantasies we would be out by now".
Pro-Remain Conservatives Phillip Lee and Dominic Grieve, the former attorney general, were subject to votes of no confidence from their local associations earlier this year.