The Patients Association is calling for an end to the "geographical strait jacket" of having to register with a GP near to where you live.
An official consultation has found 77% of those who replied back change - but 70% of health-care staff were against.
Doctors' leaders say opening up practice boundaries could be expensive and undermine local care.
The Department of Health says it intends to give patients more choice from next April.
Its consultation attracted 5,459 responses.
GPs throughout the UK can refuse to register patients who do not live near their surgery, but health ministers in England have said for some time they want to change the system.
Labour's Andy Burnham promised to "open up real choice in primary care" in a speech in September 2009.
And the current Health Secretary, Andrew Lansley, gave a commitment in a White Paper last summer that every patient would have a clear right to choose any GP surgery with an open list.
Vanessa Bourne, from the Patients Association, said: "Here we have something that is nothing to do with the patient, only to do with their address.
"It's a geographical strait jacket - and it's not the way modern health care should be operating. In some cases, patients are desperate to change GP.
"Giving patients a proper choice would help them take more interest in what they actually require from a GP."
Teresa de La Rosa, a busy estate agent, wants to sign up with a GP near her office in west London, because it has more flexible surgery hours than the surgery in her neighbourhood.
She said: "I can spend an hour in the car getting to and from the doctor where I'm registered at the moment.
"I'd like to be able to register with the surgery that's just a short walk away from my office. It's that simple.
"I think we're working to what the professionals want, rather than what patients want. The system feels very old-fashioned."
At Paddington Green Health Centre, where she wanted to register, the senior partner Dr Neville Purssell is sympathetic - but says practice boundaries exist to serve their communities.
He said: "What we are doing is providing a service for our local population.
"It would be harder for a patient living out of the area to link in with other areas of the team, such as district nurses.
"And the most significant issue arises in a loss of continuity of care when patients become unwell.
"If they need home visits, that's something you just won't get if you're not easily linked in with the locality."
'Complicated and expensive'
The British Medical Association (BMA) says there are cheaper and simpler ways of making the system easier for patients, than a wholesale abolition of practice boundaries.
A spokeswoman said: "We're not against it in principle - but we feel it will be incredibly complicated and expensive to sort out."
The BMA has suggested widening practice boundaries, particularly in busy urban areas, and increasing the use of remote consultations.
The Royal College of GPs has told Prime Minister David Cameron that removing boundaries would undermine GPs' relationships with local communities - and called on him to revise the idea.
A Department of Health spokesman said: "We know that some GPs have concerns and are working to address them.
"We will discuss our proposals with GP representatives, and aim to give patients far greater choice of GP practice from April 2012."