Labour is calling for juries to be cut from 12 members to seven, to stem the "gravest crisis" in the justice system since World War Two.
Shadow justice secretary David Lammy said action was needed to clear the backlog of thousands of cases.
He argued that smaller juries and the use of more temporary courts would allow socially distanced trials.
The government has not ruled out such a move but insists measures it is taking to clear the backlog are working.
Last week four criminal justice watchdogs warned that courts in England and Wales were straining under pressure from the coronavirus pandemic.
Jury trials ground to a halt at the start of the first lockdown, when people were advised to stay at home except in limited circumstances.
When they resumed, there were severe delays and numerous cancellations due to social-distancing requirements.
Recent figures revealed that the number of unheard cases in crown courts had reached a record 54,000.
The backlog means some from last year may not go before a jury until 2022, and it could be years before the courts get back on track.
Labour wants the temporary return of so-called "wartime juries" of seven rather than 12 members to speed up the process.
"Victims of rape, murder, domestic abuse, robbery and assault are facing delays of up to four years because of the government's failure to act," Mr Lammy said.
He also urged the government to speed up the rollout of temporary "Nightingale courts" to hear civil, family and tribunals work, as well as non-custodial crime cases.
Leading lawyers are sceptical about Labour's proposal to reach back into wartime history.
The Criminal Bar Association - representing barristers who prosecute and defend trials - says a panel of seven may allow more courtrooms to be used, but it wouldn't solve what it says is chronic underfunding - and potentially undermines one of the most important safeguards in our society.
The Law Society, for solicitors, wants to see evidence that smaller panels would ease backlogs without risking injustices.
The Ministry of Justice's internal modelling calculated last year that reduced juries would lead to a 10% increase in cases - but that was before courtrooms received new Covid-proof screens that have allowed more trials to run.
Scotland's courts are using cinemas to host juries - and while that is not being actively discussed in England, it's not been ruled out either.
Even if juries were slimmed, courts would still need to tightly control the number of defendants who can use their cells and courtroom docks to meet Public Health England's guidelines.
In April last year, the head of judiciary in England and Wales, Lord Burnett, backed the idea of reducing the number of jurors if social distancing continued.
In June, Justice Secretary Robert Buckland told the BBC he was "very attracted" by the idea of smaller juries, as had happened in wartime, and judge-only trials in less serious cases.
The Ministry of Justice says it has now installed plastic screens in more than 450 courtrooms and jury deliberation rooms to reduce Covid risks.
It says the safety measures are designed for 12-person juries and that the impact of lowering the number of jurors would be negligible.
However, a spokesman said nothing was being ruled out and ministers were continuing to consider every option available to ensure courts recover quickly.
"This approach is already delivering results, with magistrates' backlogs falling significantly and the number of cases being dealt with in the crown courts reaching pre-Covid levels last month," he added.
The spokesman also said: "We know more must be done and are investing £110m into a range of measures to drive this recovery further, including opening more Nightingale courts."