David Laws was "on balance" right to quit the cabinet but has the talent to return, Iain Duncan Smith has said.
The chief secretary to the Treasury resigned after admitting claiming expenses to pay rent to his partner.
Mr Duncan Smith, Work and Pensions Secretary, told the BBC he was "deeply sorry" about Mr Laws quitting.
But he said he agreed with Mr Laws' view that he could not do the "toughest job" of overseeing spending cuts while beset by personal problems.
Mr Duncan Smith told the BBC's Andrew Marr Show his respect for Mr Laws had grown since they started working together in the coalition, and he was a "thoroughly decent person".
"I have no questions at all that he has the talent to be back... I've been in the spotlight myself, as you know, so I wish him the very best and hope the media leaves him alone a bit now."
Mr Duncan Smith's comments came in the wake of other tributes to Mr Laws from Conservative PM David Cameron and his Lib Dem deputy Nick Clegg.
The coalition's Conservative Schools Secretary Michael Gove also paid tribute to his Lib Dem coalition colleague, telling Sky News Mr Laws had never been a "tribal" politician.
Mr Gove added that Mr Laws had given up a lucrative career in the City for politics because he believed in public service.
Saturday's Daily Telegraph revealed that between 2004 and 2007, the Yeovil MP claimed between £700 and £950 a month - about £40,00 in total - to sub-let a room in a flat in Kennington, south London, from his secret long-term partner, lobbyist James Lundie.
Mr Laws has since admitted he had claimed allowances since 2001 - meaning the total is likely to exceed £40,000.
Since 2006, Parliamentary rules have banned MPs from "leasing accommodation from a partner".
The Yeovil MP said he had wanted to keep his relationship with James Lundie private.
He also queried whether Mr Lundie counted as a "partner" as defined by the rules.
As well as resigning from his cabinet role he said he would pay back the £40,000 he had claimed and has now referred himself to the parliamentary standards commissioner.
The prime minister wrote in response to Mr Laws' resignation: "The last 24 hours must have been extraordinarily difficult and painful for you.
"You are a good and honourable man. I am sure that, throughout, you have been motivated by wanting to protect your privacy rather than anything else.
"In your short time at the Treasury, you have made a real difference, setting the government on the right path to tackle the deficit which poses such a risk to our economy."
Mr Clegg said he had always admired his fellow Liberal Democrat's integrity.
The Lib Dem leader said: "I very much hope that when those questions are answered there will be an opportunity for him to rejoin the government because, as everyone has seen in recent weeks, he has so much to contribute to national life.
"When all is said and done, this has come about because of David's intense desire to keep his own private life private. His privacy has now been cruelly shattered."
Chancellor George Osborne said he was "very sorry" to lose Mr Laws from the Treasury.
"It was as if he had been put on earth to do the job that was asked of him," he said.
"I spoke to David several times over the last 24 hours and I have a huge admiration for the way he has conducted himself in the most difficult circumstances."
But Labour MP Stephen Pound said Mr Laws' explanation that he wanted to keep his personal life private made no sense: "You don't pay money to someone you're in a close relationship with and expect the state to dip into its purse to pay it for you."
However, former Liberal leader Lord Steel said: "His mistake did not cost the taxpayer a penny since he could have been paying to rent a room elsewhere.
"Therefore his resignation seems to be an over-the-top reaction to newspaper stories and has cost the new government an able minister."
However, former independent MP Martin Bell said Mr Laws had not been unfairly treated.
Mr Bell told the BBC: "I don't think it's a witch hunt, if you go into public life you expect public scrutiny.
"This has got nothing to do with his personal relations with whoever, it has to do with expenses, and this has been going on for a few years and you have to make sure that whatever you claim is squeaky clean and if you don't you'll be punished."
Ric Pallister, a Liberal Democrat member of South Somerset District council, said everyone in Mr Laws' constituency would have a view on whether he should remain as an MP.
He said: "All I do know is that David Laws has done an an enormous amount for so many people.
"Those that work at AgustaWestland and the people that come into his surgeries day in and day out, with their own personal tragedies, David has managed to sort out and fix.
"Those people in the end will make the decision as to whether David should or should not continue."