More House of Fraser closures are "anticipated" and the department store chain's owner warned of continuing job losses on the High Street.
The chain, part of Mike Ashley's Frasers Group that owns Sport Direct, has closed about 10 of the 59 stores bought out of administration in 2018.
Frasers said it could not say how many would close as rent negotiations were continuing with landlords.
But the group's finance boss told the BBC the High Street is still suffering.
Frasers Group, whose other brands include Evans Cycles, Jack Wills and Flannels, reported annual pre-tax profits down 20% to £143.5m as lockdown store closures led to "the most challenging year" in its history.
However, group revenue rose 6.9% to £3.96bn helped by acquisitions. The company also said it was seeing "a semblance of normality" returning to trading, although there was a warning that this could be hit by any further lockdowns from a second Covid-19 wave.
More shop closures could not be ruled out, it said, as it invests £100m online operations following a shift in consumer buying habits during lockdown.
The group highlighted the continuing threat to House of Fraser outlets.
"There are anticipated to be further closures over the coming period, the number of which will depend on the outcome of lease negotiation," Frasers said.
Chris Wootton, finance director of Frasers Group, told BBC Breakfast: "The High Street as a whole is absolutely in a mess. Business rates certainly contribute to that."
Next April, the government moratorium on business rates ends, and they return to normal levels based on values assessed in 2015. Mr Wootton called that "insanity".
"Unless the government does something, thousands and thousands of jobs and businesses will go. House of Fraser was paying double, treble, quadruple the rates they should be, and that's not sustainable."
Frasers Group has now reopened the majority of its stores after lockdown, while online sales were described as being "strong".
The company also renewed its attack on the management of Debenhams during the High Street crisis. The chain is now in administration for the second time.
There were reports last week that Mr Ashley, who saw a near £150m investment in Debenhams wiped after the first administration, is interested in taking on 30 stores.
"We continued to follow the further demise of Debenhams during the year with much frustration and disappointment as it entered administration for a second time," Frasers said in a statement.
"We raised our concerns and gave numerous warnings about what we were seeing there, much of which has materialised. Our offers of help were repeatedly disregarded and it is scandalous that this business has now been in administration twice."
Talking to the BBC, Mr Wootton declined to be drawn on buying Debenhams. "We look at things with varying degrees of seriousness."
Publication of Frasers Group's results was delayed a week, which the company said was due to finalising the figures in accordance with new accounting rules. The results were also delayed last year.
On Monday, it was disclosed that chairman David Daly had bought a small parcel of shares "in error" during the closed period when directors are not allowed to trade.
Despite it being "another chaotic situation" for Frasers, said Global Data retail analyst Emily Salter, the company's reorganisation appeared to be paying off.
Analysts at Liberum described the profit numbers as "robust" and pointed to further growth over the next 12 months.
However, Sophie Lund-Yates, equity analyst at Hargreaves Lansdown, said that despite the "decent set of results", Mr Ashley still had to show how he planned to re-organise Frasers' array of brands for the future.
"Exactly how all the pieces of Ashley's High Street puzzle fit together remains to be seen," she said. "Getting the stores where they need to be, and securing long-term, sustainable growth is going to take a while, and a lot of money."
Shares in Frasers, down a third so far this year, had jumped 14% by mid-morning on Thursday.