Conservative parliamentary candidate for Dudley North Afzal Amin has resigned after being accused of plotting with the English Defence League over a march against a new "mega-mosque" in Dudley that he would then take credit for stopping.
But community leaders say claims about the mosque have been misrepresented.
Branded at times a "mega mosque" or "super mosque", the plans have attracted EDL supporters from across the country, although community leaders have in turn accused them of "scaremongering".
The proposals date back to 2005 and it has been the subject of a long-running battle between Dudley Council and the local Muslim association until last year, when plans were agreed.
Both the Muslim Association and the local authority are united on one issue - that community relations are broadly very good in the area.
Kurshid Ahmed, former chairman of the Dudley Muslim Association, accused Mr Amin of a "cheap form of political opportunism".
The Conservatives have targeted the Dudley North seat, with Labour defending a majority of just 649.
Peter Lowe, leader of the Labour-led Dudley Council, dismissed Mr Amin's suggestions of "serious community tensions", describing his alleged actions as "below contempt".
The land at the centre of the centre of the dispute, on Hall Street, was bought by Dudley Muslim Association 10 years ago, with the aim of replacing its current building on Castle Hill, which it said it had outgrown.
Proposals include a community and training centre, sports facilities and a two-storey car park, but it has been the mosque itself and its three prayer halls that have been the focus of opposition.
Proposed Dudley Central Mosque
- Expected to accommodate up to 750 worshippers at any one time, in three halls.
- Will include community meeting rooms, employment and training advice, as well as sports facilities.
- Revised plans approved in November include a smaller minaret, fewer outbuildings and more car parking space.
Mr Ahmed said far from being a "mega mosque", as critics have labelled it, it will be considerably smaller than many others even in the immediate area.
"The current mosque has a capacity for 500 worshippers and the new one will be able to accommodate 750, which we think will be enough for the size of the population over the next 20 to 30 years," he said.
"But there are mosques in the Black Country with capacities of more than 1,000, so it's really just medium sized.
"The facts have been completely distorted and misrepresented."
By comparison, Birmingham Central Mosque said it typically attracted some 4,000 worshippers for Friday prayers, with the main hall able to host 3,000 people at a time.
'Call to prayer'
Other opposition has centred around the planned minaret, which some people claimed would dominate the local skyline.
At 62ft high (19m) - scaled back from 109ft in original plans - Mr Ahmed said it would be much lower than local churches and several other buildings in the area.
Some critics have also claimed it would lead to the traditional Adhan or call to prayer sounding across the local neighbourhood.
Mr Ahmed described those suggestions as "scaremongering" and said, like the current mosque, any call to prayer would only be heard within the building itself.
Since the plans were first published in 2007 they have prompted widespread opposition, with thousands of letters of objection and the council consistently turning down plans.
Some residents called for social housing on the site, while others said it was out of keeping with the local area.
The Dudley Muslim Association and the local authority have both lost decisions in the High Court in recent years during a long-running planning battle.
It is the length of the dispute, coupled with "misinformation", that local Muslim leaders blame for the unwanted attention.
Revised plans were finally approved in November, but that has not stopped the demonstrations.
A protest last month, involving about 1,000 demonstrators, largely passed off without violence, but previous rallies have been less peaceful.
Two EDL protests in 2010 led to arrests for violent disorder and possession of an offensive weapon, while Labour MP Ian Austin said mosques, as well as Hindu and Sikh temples were all targeted.
He described Dudley as a community "in which people generally get on well" and accused Mr Amin of trying to "use the situation".
Council leader Mr Lowe echoed that view, saying community relations were "good".
"That's not to say there haven't been disagreements within the community, but that's to be expected and they've been dealt with by sitting down and talking together," he said.
"The recent EDL rally had very few Dudley residents, judging by the flags represented. The flags were mainly from the North West and North."