Surveillance laws that allow police officers to access people's phone records are not fit for purpose, the Home Affairs Select Committee has said.
The Regulation of Investigatory Powers Act (Ripa) has been used to access journalists' records in some cases.
The committee said journalists' sources should be "fully protected" and access to data under Ripa was "secretive".
The Home Office said there were measures in place to ensure police powers were not abused.
Police officers have also failed to routinely record the professions of individuals who have had their communications data accessed, MPs said.
Earlier this year, it emerged police had used their powers under Ripa to obtain information about phone calls involving newspaper reporters.
The Metropolitan Police used the Act to obtain telephone records of the Sun's newsdesk to try to identify who had leaked the "Plebgate" story involving former Conservative chief whip Andrew Mitchell.
Kent constabulary also used its powers under Ripa to obtain phone records of a journalist investigating the Chris Huhne speeding points scandal, as well as those of one of his sources - despite a judge agreeing the source could remain confidential.
Committee chairman Keith Vaz said: "Ripa is not fit for purpose. We were astonished that law enforcement agencies failed to routinely record the professions of individuals who have had their communications data accessed under the legislation.
"Using Ripa to access telephone records of journalists is wrong and this practice must cease. The inevitable consequence is that this deters whistleblowers from coming forward."
He told BBC Radio 4's Today programme that journalists' records should be kept privileged, "otherwise we get into a situation where legislation introduced for completely different purposes is being used in a mission creep to be able to control sections that were never intended to be controlled".
Half a million pieces of information are accessed every year under the legislation, Mr Vaz added.
He told the programme it had been used for "trawling", saying: "We have felt for some time that public officials are using this piece of legislation for what was not intended by it."
The committee called on the Home Office to hold a consultation on an amended Ripa code of practice, which would give special provisions to those dealing with privileged information.
Security minister James Brokenshire said communications data was "an absolutely critical tool" used "to investigate crime, safeguard national security and protect the public".
He insisted there were already safeguards against abuse of police powers to access data.
'Patchy at best'
The Home Office said new rules would ensure "extra consideration" was given in cases where police want to see the phone records of those in sensitive professions, such as journalists.
The revised code will be published before Christmas, according to the government.
Metropolitan Police Commissioner Sir Bernard Hogan-Howe said that officers did not want to interfere with the work of bona fide journalists and proportionality was needed.
He added: "In cases that I'm aware of, the police have been investigating a crime where a journalist is believed to have been involved."
Privacy lobby group Big Brother Watch said the current situation was "intolerable".
Emma Carr, who is the director of the group, said: "When a senior parliamentary committee says that the current legislation is not fit for purpose, then this simply cannot be ignored.
"It is now abundantly clear that the law is out of date, the oversight is weak and the recording of how the powers are used is patchy at best. The public is right to expect better. This is intolerable."
Human rights organisation Liberty said more safeguards were needed to stop data being accessed without judicial warrants.
Its director of policy Isabella Sankey said: "The secret use of Ripa to investigate journalists' sources will chill anyone who values free speech and a free press.
"But what's disturbing is that the abuses detailed in this damning report are the tip of the iceberg."