The American Civil Liberties Union plans to sue the US authorities over the "no-fly" list, saying it stops innocent people from travelling.
The legal action is on behalf of 10 US citizens or legal permanent US residents who say they have been prevented from flying.
Six of those are currently stuck outside the US.
The list is maintained by the US authorities as part of the measures aimed at preventing attacks.
But critics say there are problems with "false positives" where a person is stopped from flying because they have the same name as someone on the list.
And, the ACLU says, people are usually not told the reason for their inclusion on the list and have no effective way to dispute it.
Those wrongly on the list are being denied their rights to "due process" under the law, guaranteed by the Fifth Amendment of the US constitution, the ACLU says.
The legal action is against the FBI, the Department of Justice and the Terrorist Screening Centre.
"More and more Americans who have done absolutely nothing wrong have found themselves unable to fly and in many cases unable to return to the US without any explanation from the government", said Ben Wizner, a lawyer with the ACLU.
"The government provides no opportunity for people on the list to object."
US citizen Ayman Latif, 32, a former marine who was born in Miami, is one of those listed in the legal action. Having spent a year-and-a-half in Egypt learning Arabic, he has been unable to return to the US since being stopped at Cairo airport in April.
After contacting the US Embassy and the FBI, Mr Latif was subsequently interviewed by two agents at the beginning of June.
They would not tell him why he was on the no-fly list.
"They asked me all kinds of crazy questions like have I ever been to Yemen? I said no. Do I know any Nigerians? Have I been to any training camps?
"It made me feel uncomfortable. They tried to make me feel like a criminal. But I've done nothing wrong."
Since the interview, and despite numerous calls, the FBI has not contacted him to say he can fly.
The list was beefed-up in the wake of the alleged Christmas Day airliner plot after criticism over suspect Umar Farouk Abdulmutallab's absence from it.
Those on the list are not allowed to board flights to the US, or in the US.
"As a matter of long standing policy, we do not comment on whether a particular individual may be on a watch list and we will not do so here," an FBI spokesman said.
"Several recent high-profile attempted terror plots against US targets, including the attempted Christmas Day attack and the Times Square incident, remind us of the need to remain vigilant and thoroughly investigate every lead to fend off any potential threats to the US."