Thousands of "infiltrators" - including many Tories - may be planning to vote in Labour's leadership election, Andy Burnham's campaign has said.
Michael Dugher, Mr Burnham's campaign chairman, suggested not enough was being done to address the issue.
BBC News understands 3,000 applications have been rejected. There have been claims some are being unfairly blocked.
Acting leader Harriet Harman said the process was "rigorous" and non-genuine supporters would be "weeded out".
"If they have voted their votes will be cancelled," she said.
Mr Burnham, Jeremy Corbyn, Yvette Cooper and Liz Kendall are all standing.
Polls suggest Mr Corbyn is the frontrunner.
New regulations allow members of the public to sign up to vote as a "registered supporter" for £3.
But some have been told their vote will be discounted.
Mr Burnham's team have written to Labour's general secretary claiming "a Conservative MP, Conservative media commentators, and Conservative councillors" had all been rejected.
The letter added: "This suggests the 121,000 registered supporters could include several thousand Tory infiltrators, as well as supporters of other parties seeking to have a vote in the election."
Mr Dugher said the party was "allowing the issue to drift, and potentially leaving insufficient time for the party to act".
There is "potentially more that can be done by the party to weed out Tories and other rogue voters," he added.
Comedian Jeremy Hardy, meanwhile, suggested the party was seeking to "rig" the election by wrongly barring left-wing supporters of Mr Corbyn from taking part.
He accused Labour of trying to "change the rules of a game during an election".
"The Labour Party might be trying to invite a legal challenge so that they can say 'let's just scrap the election'," Mr Hardy added. "They are so desperate that I wouldn't be surprised."
Labour MP Kate Hoey also said a former Labour leader of Lambeth Council, Joan Twelves, had been "rejected".
And former Labour MP Andrew MacKinlay accused the party of misusing canvass returns - collected at election time to gauge the level of support for candidates - to find people who have previously voted for other parties.
He said the exclusion process had been "arbitrary" and "unfair".
Labour's verification process
Labour says all applications to join as a full member, affiliate, or a supporter are assessed by a verification team.
The party says it has 40 staff in Newcastle and 30 at its London HQ working to check applications and anyone who does not support Labour's "aims and values", anyone not registered as an elector at the address they have given or anyone previously excluded from the party would be rejected.
MPs, local constituency parties and regional offices are also carrying out checks and providing further information. Applicants' social media accounts are also being looked at.
The latest figures show the party had as many as 189,703 affiliates, 121,295 registered supporters and 299,275 members would could potentially vote.
But Ms Harman defending the system, saying: "We have a very rigorous process both to make sure that those people who are entitled to vote under this new system do get a vote, but also to make sure that those people who should not be eligible for a vote because they don't support the Labour party or because they support another party don't get a vote."
Ms Cooper said the party should make sure processes are "robust", but she wanted as many people as possible to take part.
A spokesperson for Mr Corbyn said he had confidence in the management of the election.
"This latest internally-faced intervention is an attempt to distract the leadership election onto 'process' rather than real political issues," the spokesperson added.
Also on Friday, Mr Corbyn said he would formally apologise on behalf of the Labour Party for taking the country to war with Iraq if elected leader.
Mr Corbyn told the Guardian the party would "never again flout the United Nations and international law".
Labour leadership contest
- Who are the candidates? Andy Burnham, Yvette Cooper, Jeremy Corbyn, Liz Kendall
- Dates: Ballot papers were sent out on 14 August; voting can take place by post or online. They must be returned by 10 September. The result is announced on 12 September
- Who can vote? All party members, registered supporters and affiliated supporters - including those joining via a union. More than 160,000 people signed up to vote as supporters, full members or union affiliates in the final days before the registration deadline, bringing the total size of the electorate to 610,000
- What is the voting system? The Alternative Vote system is being used, with voters asked to rank candidates in order of preference
- How does it work? If no candidate wins outright with more than 50% of first preferences, whoever is in fourth place drops out and the second preferences of their backers are reallocated to the other candidates. If there is still no winner the third placed candidate is then eliminated with their second preferences similarly reallocated. The candidate who has accumulated the most votes through the different rounds then wins.