It is "time to get stop and search right", the home secretary has said, as she launched a public consultation on the police powers in England and Wales.
About a million stops take place each year - but only 9% lead to an arrest.
When the tactic was misused it wasted police time and undermined public confidence, Theresa May said, adding that no-one should be stopped "just on the basis of their skin colour".
Police gave backing to improvements that would not undermine public safety.
Announcing the six-week public consultation, Mrs May told MPs there were disparities in search-to-arrest ratios across England and Wales, with some being "far too low for comfort".
She said she wanted the powers to be used fairly to build confidence in the police.
"I want to make sure that stop and search is used fairly and in everybody's interest. I want to see stop and search used only when it's needed," she told the Commons.
"I want to see higher search-to-arrest ratios. I want to see better community engagement and I want to see more efficient recording practices across the country."
It takes on average 16 minutes to conduct a stop and search and process the details. That amounted to 312,000 hours each year and the equivalent of 145 full-time police officers, she said.
Mrs May said she wanted to make the process more efficient and cut bureaucracy to stop "pointless" stops and searches.
"At its best, stop and search is a vital power in the fight against crime. At its worst, it's a waste of police time and serves to undermine public confidence in the police," she said.
"It's time to get stop and search right."
She also said the government had to recognise concerns that people from ethnic minorities were being targeted disproportionately.
"The official statistics show that if you're from a black or minority ethnic background, you're up to seven times more likely to be stopped and searched by the police than if you're white," she said.
"We shouldn't rush to conclusions about those statistics, but everybody involved in policing has a duty to make sure that nobody is ever stopped just on the basis of their skin colour or ethnicity."
'No paper exercise'
Steve Williams, chairman of the Police Federation of England and Wales which represents tens of thousands of officers, said: "Any decision to invoke stop-and-search powers must be justified and officers must be accountable for the decisions they make as part of their commitment to policing by consent.
"In our view this tool is essential - however if there are ways in which it can be further improved without a reduction in public safety, then we are keen to take part in the debate."
Black police officer Nick Glynn, 46, who estimates he has been stopped and searched about 30 times in his life, warned there was a "cost" to the individual each time it happened.
"It can be embarrassing, it can be upsetting and it has a ripple effect through that person and their family and friends," he said.
Labour's Keith Vaz, chairman of the home affairs select committee, said it was important the consultation was not just a "paper exercise", and urged Mrs May to visit communities most affected by it.
Diane Abbott, Labour MP for the London boroughs of Hackney North and Stoke Newington, said proper training was vital to stop the "important weapon" being "detrimental to community relations".
Campaigners welcomed Mrs May's statement.
Nick Pickles, director of privacy and civil liberties campaign group Big Brother Watch, said: "Today's statement is an important step towards ensuring the public, particularly people from ethnic minorities, can have confidence that they can walk the streets without fearing they will be subject to further unjustified use of stop-and-search powers."
Last month, the government's equality watchdog said police use of stop-and-search powers had fallen, without compromising crime reduction.
The Equality and Human Rights Commission (EHRC) also found that four of the five forces it studied had reduced targeting of minority groups.
It said its report showed "clear evidence that where forces use an approach based on evidence rather than hunches or generalisations, they have not only seen reductions in crimes rates in line with overall trends, but have also increased public confidence in the police".
The five forces covered by the report were Dorset Police, Leicestershire Constabulary, Thames Valley Police, West Midlands Police and the Metropolitan Police.
A previous report by EHRC, published in 2010, had suggested black people were six times more likely to be stopped and searched than white people.