The government's "digital champion" has called for a revolution in the way it deals with the public online.
Martha Lane Fox, who reviewed the Directgov website, said it must become easier to access government services and make payments over the internet.
Shifting half of its contacts with the public online could save the government more than £2bn a year, she said.
Cabinet Office Minister Francis Maude said it was "ridiculous" that many benefits were not available online.
BBC technology correspondent Rory Cellan-Jones said the words "fragmentation, duplication and bureaucracy" leapt out of Ms Lane Fox's review of the Directgov website.
She contrasts the process of applying for a student loan, which ends with the printing out and signing of a 30-page document, with the simplicity of booking a flight.
Her report recommends that Directgov becomes the place for all government transactions - from paying out benefits to collecting car tax - but with a brief to take a service culture to the public sector.
But Ms Lane Fox - who officially launched the report on Tuesday - says that as this revolution proceeds, some services will need to be turned off.
She said: "Government should take advantage of the more open, agile and cheaper digital technologies to deliver simpler and more effective digital services to users, particularly to disadvantaged groups who are some of the heaviest users of government services.
"But this is just the beginning. The government must look at more dramatic measures - such as syndicating and opening up information and services to other organisations - to be able to offer genuine improvements to consumers, taxpayers, business and citizens in the UK."
She is also calling for Business Link, the online advice service for companies, and Directgov to be "converged" into a single platform and for a senior figure to work within the Cabinet Office to drive forward the changes.
The internet entrepreneur was appointed to advise Labour on digital inclusion in 2009 and is now acting in a similar role for the coalition government.
As well as encouraging maximum internet availability and usage, the lastminute.com founder sits on the government's efficiency board, chaired by Mr Maude.
Her focus is to help reduce government costs by making services more accessible and usable.
Ms Lane Fox says "bridging the digital divide" is a key economic priority.
Commenting on the report, Mr Maude said: "There is no excuse for not making quality online services the default solution for people needing government services."
He said it was "inconvenient, expensive, wasteful and ridiculous" that some benefit applications forms were not available online and big changes were needed to "personalise" government services as well as to cut down on waste.
"This does not mean we will abandon groups that are less likely to access the internet," he added. "We recognise that we cannot leave anyone behind.
"Every single government service must be available to everyone - no matter if they are online or not."