David Davis is in "cloud cuckoo land" if he believes Brexit talks are heavily weighted in favour of the UK, a former Treasury civil servant told the BBC.
The Brexit Secretary and other minister should "rein back" "hard and unconstructive" talk, former Treasury civil servant Sir Brian Unwin said.
The UK has said it will begin formal talks about exiting the EU by the end of March 2017.
But Conservative MP John Redwood said the UK was in a "very strong position".
Mr Redwood, who advised Margaret Thatcher on European issues as head of her policy unit, said Sir Brian's comments were a "great pity" as "I'm sure he wishes our country well".
Mr Davis told the House of Commons on 10 October: "One of the things that I have discovered in the past few months is that in many areas - not just the City, and not just as regards cars - the balance of negotiating advantage is incredibly heavily stacked our way."
Sir Brian, who was a senior civil servant under Labour and Conservative governments - including Margaret Thatcher's, when he helped negotiated the UK's budget rebate - said he felt ministers and civil servants had a "horrific" job ahead.
He told BBC Radio 4's World at One: "I saw that Mr Davis the other day was reported to have said that the negotiating odds are unbelievably weighted on our side. Well I think that is utter rubbish, I mean, it really is cloud cuckoo land."
He said opinion was "hardening" towards the UK among EU countries: "They do not want the integrity and the future of the European Union to be put at risk by Brexit and in particular they do not want the United Kingdom to emerge with advantages which might encourage other member states to think of leaving."
He predicted Theresa May, who is in Brussels for her first EU summit as prime minister, might not get a "terribly cosy" reception from her fellow leaders adding: "I really do think the language which the present government is using needs to be tempered if they are to have a successful negotiation."
Sir Brian, who was president of the European Investment Bank, also said he felt it would be "suicidal" were Britain to leave the EIB - which is owned by the 28 EU member states, as its funding had been "enormously important" to UK infrastructure.
The prime minister has said Article 50 will be triggered by the end of March 2017, beginning the process of formal talks with the EU, which could last up to two years before Brexit happens.
EU leaders have insisted that the UK will need to accept freedom of movement of people if it wants to retain access to the EU's single market and European Council president Donald Tusk said last week that the only alternative to a "hard Brexit" - leaving the single market in return for more control over immigration - was "no Brexit".
The government is also facing fierce criticism from UK opposition parties - and a landmark legal challenge - over its refusal to offer MPs a vote on its Brexit strategy before triggering Article 50.
But Mr Redwood, a leading Brexit campaigner, told the BBC the UK was in a "very strong position" in comparison with the EU "because we pay the money to them, they don't pay money to us.
"They sell much more to us than we sell to them. Many of the things they sell to us would attract tariffs under WTO rules whereas many of the things we sell to them wouldn't attract tariffs under WTO rules.
"We must be united as a country in being confident, friendly and outgoing in this negotiation, and it doesn't help to have people in Britain constantly saying 'oh, we're in a very weak position and this is a major problem'.
"We should be very positive, because we all want the best for our country and the best for Brexit, and the sooner we get it over with, the better for us - because it gets rid of the uncertainties - but the better for them."