The government has a "moral responsibility" to protect the interests of Gibraltar during Brexit negotiations, a Lords committee says.
The EU committee said the single market and cross-border travel were vital to the territory's economy, and warned the UK government not to let Spain use trade talks to claim sovereignty.
Gibraltar voted overwhelmingly for Remain in June's referendum.
The government says it will ensure the territory's priorities are understood.
Gibraltar has been a British territory since 1713, but Spain continues to claim sovereignty over the enclave, and the government in Madrid called for joint sovereignty in the aftermath of the Brexit vote.
The vote of 95.9% in favour of staying in the EU made Gibraltar by far the strongest pro-Remain area taking part in the referendum.
Peers said 40% of its workforce crossed the border from Spain every day, and its economy was "underpinned" by the single market.
The committee said it "strongly endorses" the UK government's promise never to enter into sovereignty discussions with Spain against the will of the Gibraltarian people, and called on ministers to engage "positively and pragmatically with Spain, emphasising the mutual importance of the economic relationship between the UK and Spain".
Prime Minister Theresa May has confirmed the UK will not remain a member of the EU single market after Brexit, instead seeking a new free trade deal.
The peers said while it was unclear how withdrawal would affect the border, there were "serious potential economic implications for both Gibraltar and the surrounding area of Spain".
"Now set to leave the EU, the territory has placed its trust in the UK to secure a Brexit deal that meets Gibraltar's needs," they said.
"The committee stresses that the UK government has a moral responsibility to ensure Gibraltar's voice is heard, and its interests respected, throughout the negotiating process."
'Out the room'
The committee also said Gibraltar should be made a priority in talks over security co-operation so its border cannot be used by criminals.
And it suggested existing rules governing the passport-free Schengen area could allow continued movement between Gibraltar and Spain.
Formal Brexit talks are set to begin once Article 50 of the Lisbon Treaty has been invoked by the UK government before the end of March.
The committee added: "The UK government must remain alert to and resist any attempts by Spain to involve the sovereignty dispute in EU withdrawal and future trade negotiations, or to encroach upon Gibraltar's sovereignty through the medium of EU laws or policies when the UK is 'out of the room', after Brexit."
In the official document setting out its Brexit strategy, the UK government said a new joint ministerial council with the government of Gibraltar would help ensure its views were heard.
"We will continue to involve them fully in our work, respect their interests and engage with them as we enter negotiations, and strengthen the bonds between us as we forge a new relationship with the EU and look outward into the world," its White Paper adds.