Senior Conservatives have called for the party to pull together after it suffered its worst results in English local elections since 1995.
The Conservatives lost 1,334 councillors in Thursday's votes.
Home Secretary Sajid Javid admitted voters had "issues of trust" over Brexit, and said the European elections would "be even more challenging".
Health Secretary Matt Hancock said the party needed to listen to the results and be "in the mood for compromise".
Both PM Theresa May and Labour leader Jeremy Corbyn have insisted they will push ahead with seeking a cross-party agreement on Brexit, following the results.
Labour had been expected to make gains but lost 82 seats in the elections, while the Liberal Democrats - who have campaigned for a further vote on leaving the EU - were the main beneficiary of Tory losses, gaining 703 seats.
The Greens and independents also made gains, as UKIP lost seats.
Elections were held for 248 English councils, six mayors, and all 11 councils in Northern Ireland - where a second day of counting is continuing. No elections took place in Scotland or Wales.
Speaking to BBC Radio 4's Today programme, Mr Hancock said "the mood of the nation is, 'get on, deliver Brexit, and then move on'".
But he said the Tories might have to move towards Labour's proposal of a permanent customs union - in order to solve the current impasse in Westminster over Brexit.
The EU customs union means that once goods have cleared customs in one country and the commonly agreed tariffs (charges on imports) have been paid, they can be shipped to others in the union without further charges.
A country does not have to be a member of the EU to be part of the customs union, but members cannot negotiate their own independent trade deals with countries from the rest of the world.
Mrs May's government has previously ruled out remaining in a customs union after the UK leaves the EU, arguing it would prevent the UK from setting its own trade policy.
Labour has suggested the EU may show flexibility over the issue and allow the UK "a say" in future trade deals.
Mr Hancock suggested "coming up with something in-between", and called for "an open dialogue in which we can make an agreement".
Foreign Secretary Jeremy Hunt also said there was a "glimmer of hope" that a compromise between the Conservative and Labour "core-voters" could be reached.
"If we can find a solution that delivers the benefits of the customs union without signing up to the current arrangements, then I think there will be potential," he said.
'Persevere with talks'
Justice Secretary David Gauke told BBC News that the local election results should be seen as a "punishment" to both the Conservatives and the Labour Party "for failing to find a way through" the Brexit conundrum.
He added: "We have to persevere with the talks with the Labour Party. I think that is the best opportunity to find a way through here."
The MP for Hertfordshire South West also rejected calls to oust Mrs May, saying: "We should back the prime minister... so that we can bring the country together again - we can unite the Conservative Party and find a practical way through."
The UK was due to leave the EU on 29 March, but the deadline has been pushed back to 31 October.
Mr Javid said this was a big factor in the Conservative Party losing control of 45 councils on Thursday - in its worst performance since John Major's party lost 2,000 councillors in 1995.
In a rallying cry to Conservatives in Aberdeen, he said that "a divided party cannot unite a divided nation".
The home secretary said the party risked losing voters' trust after "not delivering on a promise at the heart of our last manifesto".
And, speaking about the European elections, due to take place on 23 May, he said: "We shouldn't be surprised if people tick the protest box on the ballot paper."
Either search using your postcode or council name or click around the map to show local results.
Lisa Nandy, Labour MP for Wigan, also said the results reflected the public's frustration with the two main parties' "perceived inability... to get our act together".
She told the Today programme there was "no single magic bullet" to solving Brexit, but "the fact that people are not clear on what our policy is, is harming us in both Remain and Leave areas alike".
Ms Nandy said failing to leave the European Union would be a "final breach of trust" and her party must respect the referendum result.
However, she said she believed the Brexit effect on the election results had been "enormously overstated" and many in towns like Wigan "just didn't feel like Labour spoke for them".
The BBC projects that, if the local election results it analysed were replicated across Britain, both the Conservatives and Labour would get 28% of the total vote.
Polling expert Prof Sir John Curtice said the days of the Conservatives and Labour dominating - as happened in the 2017 election when they won 80% of the vote between them - "may be over".
He said it was only the second time in history that the two main parties' projected national share of the vote had fallen below 30%.
The only other occasion was in 2013, when UKIP performed strongly in local elections.