David Laws has said he never sought to exploit the expenses system and had paid a "high price" for wanting to keep his sexuality private.
Mr Laws, who quit as Chief Secretary to the Treasury after disclosures about his expenses, said he should have been "more open" about his circumstances.
"I accept I should have set a better example as a public figure," he said in a statement to local media.
He said he hoped to remain as Yeovil MP, if his constituents backed him.
The Liberal Democrat resigned as Chief Secretary to the Treasury, saying he did not feel he could continue with the "crucial work" of sorting out the nation's finances after it emerged that he claimed expenses to pay rent to his partner, James Lundie.
Under rules introduced in 2006, MPs are not allowed to lease accommodation from a partner.
Mr Laws had already apologised for a breach of the rules, referred himself to the Parliamentary Standards Commissioner and agreed to repay £40,000.
Senior politicians from the coalition government have praised Mr Laws and said they hope he can return to office some time in the future.
Mr Laws, who was in the cabinet for less than three weeks, told the Western Daily Press the past few days had been the "toughest of his life".
"It has been a very emotional experience to find so many people willing to stand by me at this difficult time," he said.
"My problems have been caused by my unwillingness to be open about my sexuality and not by any intention to exploit the MPs' expenses system."
He said his wish to keep his sexuality private was influenced by the fact he had grown up at a time when homosexuality was still regarded as "wrong or shameful" and said "the further time went on the more difficult it seemed to be to tell the truth".
When the rules changed in 2006 preventing MPs from claiming expenses on properties leased from relatives or partners, Mr Laws said he should "probably have changed our arrangements".
He also acknowledged that Mr Lundie was "much more" than a friend.
"I have paid a high price for trying to keep my sexuality a secret. Losing your privacy, your Cabinet job and your perceived integrity within 48 hours isn't very easy.
"But I accept that I should have been more open and should have set a better example as a public figure."
Mr Laws said he intended to "get back" to his work as MP for Yeovil as soon as possible, a job which he said he "loved".
But he added: "Over the weeks ahead, I will want to understand whether I still have the confidence of my constituents, without which it would be difficult to continue my work."
Mr Laws has been succeeded as Chief Secretary to the Treasury by Lib Dem colleague Danny Alexander.