The UK is starting to be seen as a "divided, hostile and narrow-minded" country since its vote to leave the EU, Andy Burnham has warned.
In his last conference speech as shadow home secretary, Mr Burnham accused the government of planning a "hard-line, right-wing" exit from the EU.
He said Labour must fight for a "fair Brexit" that addressed immigration concerns and protected the UK economy.
The government said it would make a success of leaving the EU.
Mr Burnham, who is running for the Greater Manchester mayoralty, told Labour conference he was stepping down from the shadow cabinet to concentrate on his campaign.
In his speech, the outgoing shadow home secretary urged the party to unite and take the fight to the Conservatives, warning of what he sees as the risks of continued Conservative government.
"While we've been turning inwards, they've been taking liberties. Bringing back selection in our schools. Abandoning promises to the north. Plotting a hard-line, right-wing Brexit that burns Britain's bridges," he said.
"We are the only hope for people who don't want that. Yes, Labour must listen to our voters who voted to Leave. But let's be the champions of a fair Brexit, not a hard Brexit."
He warned that a "hard Brexit" would hurt the UK economy, hitting the poorest hardest and "turn Britain into a place it has never been: divided, hostile, narrow-minded".
"That is how the world is starting to see us. They think we've changed."
'Selfish little world'
There had been a "frightening spike" in hate crimes since the referendum vote on 23 June, Mr Burnham said.
He repeated his call for the government to grant EU nationals living in the UK the right to stay in the country after Brexit - a commitment the government says it wants to make as long as it can secure a reciprocal deal for Britons living abroad.
And he criticised Mrs May over the UK's response to the migrant crisis affecting Europe: "At every stage of the biggest refugee crisis since the end of the Second World War, she has dragged her feet.
"As Europe has struggled with the enormity of it, Britain has looked wrapped up in its own selfish little world."
Mr Burnham praised the "courage" of his predecessor, Yvette Cooper and Labour peer Alf Dubs for their campaigning on refugees, saying it had been left to them to "stand up for basic British decency".
Mr Burnham also warned that Labour could not afford to ignore voters' concerns about immigration, saying it must "fully face up" to the fact that "millions" of its supporters backed Brexit because they wanted "change".
"We haven't yet even begun to show to them that we understand why," he said.
He stressed that Brexit supporters were not "narrow-minded" or "xenophobic" and warned the party that its supporters would think the party has "abandoned" them if it argued in favour of the status quo on immigration.
Commenting on Mr Burnham's speech, Brexit minister Robin Walker said "many" Labour MPs "still don't even accept that we should leave the EU.
"Labour cannot make a success of Brexit because they don't think Britain can thrive outside the European Union - setting their face against millions of their own lifelong supporters."
Leader Jeremy Corbyn said in his end of conference speech that Labour under his leadership would not offer "false promises" on immigration numbers.
He said a new migrant impact fund would address "the real issues of immigration" and pledged his party would not "fan the flames of fear".