Efforts to advance Brexit are "under attack", Lord Lamont has said, warning peers not to try and make more changes to a bill on triggering Article 50.
The ex-Tory chancellor said calls for a "meaningful vote" on the final deal were "cover" for not wanting to leave.
Peers will debate the issue on Tuesday, having already backed a guarantee of the rights of EU residents to remain.
The Lords must "see sense" and allow Theresa May to begin official talks as soon as possible, Lord Lamont said.
The prime minister wants to notify the EU of the UK's intention to leave by the end of March but needs the approval of Parliament to do so.
The House of Commons has approved legislation which would kick-start the two-year process, but the Lords has already amended the bill and Labour, Lib Dem and crossbench peers are seeking further changes when debate resumes.
Ministers have given verbal assurances that Parliament will have a vote on the terms of the UK's exit before the final package is considered by the European Parliament.
Were MPs or peers to reject a deal agreed with the other 27 states, the government has said the UK would still leave, albeit without any formal arrangements.
But critics are concerned what will happen if there is no deal to vote on, something the government has said is unlikely but possible, and they want Parliament to have the authority to send ministers back to the negotiating table to secure better terms.
Lord Lamont, a Brexit-supporting peer who served as chancellor between 1990 and 1993, said arguments for a "meaningful vote" seemed to be predicated on the view that the British public, which voted to leave the EU last year by a margin of 51.9% to 48.1%, would have a change of heart at some point.
"The result was clear, the question was simple and unambiguous and yet Brexit is under attack on several fronts," he said in a speech in London.
"Some say maybe in the future the British people will change their minds. By that, they mean that they would like to change their minds for them."
Adding to the Article 50 bill was not the same as scrutinising it, he argued.
"Amendments should not be used to as a cover by those who are seeking to oppose the results of the referendum. I hope my colleagues in the Lords will see sense."
If the Lords pass the "meaningful vote" amendment, the clause will be reconsidered by MPs - who rejected it by a margin of 326 to 293 last month and would be expected to do the same again.
But former Conservative minister Bob Neill, who voted against the government on the issue, has indicated that without further concessions he might rebel again.
"I'd certainly be inclined to vote in that way unless the government is able to come up with alternative assurances on the floor of the Commons," he told BBC Radio 4's The Westminster Hour.
"If there is no deal, that means that we would potentially leave the EU straight on to World Trade Organization terms and without any transitional arrangements.
"I believe that would be deeply damaging for this country and I think Parliament should have the right to consider that."
Cabinet minister David Lidington has said any sign Parliament wanted to limit Mrs May's room for manoeuvre or was prepared to rethink Brexit if the talks went badly would make "an ambitious, mutually beneficial deal much more difficult".
No 10's official spokesman told reporters on Monday that the UK should not do anything that might encourage the EU to "give us a bad deal".