Boris Johnson's comments about women wearing the burka have given an impression that the Tories are "insensitive to Muslim communities", an independent review has said.
The prime minister ordered a report in 2019 into how the party handles discrimination allegations.
It found anti-Muslim views were seen at local association and individual level.
But claims of "institutional racism" were not borne out by evidence of the way complaints were handled.
However, there is "clear evidence" the Conservative party's complaints system is "in need of overhaul", according to Professor Swaran Singh's independent review.
The report warned it "should make for uncomfortable reading for the party".
In a statement Prof Singh said: "For those unable or unwilling to read all of its 44,000 words, here's a summary: we found evidence of discrimination, and while the problem is not systemic the party must now act to root it out."
Responding to the report, Conservative party co-chair Amanda Milling apologised "to anyone who has been hurt by discriminatory behaviour of others or failed by our system".
She said the party accepted all the recommendations and would publish a plan to implement them in six weeks' time.
Former Conservative Chancellor Sajid Javid said the report contained "distressing examples of anti-Muslim sentiment" but insisted that Mr Johnson was not Islamophobic and "respects anyone from any background, any community".
Another former cabinet minister - and long-standing critic of the party over Islamophobia - Baroness Sayeeda Warsi told Sky News the report showed that "the party was institutionally racist" and said the Equalities and Human Rights Commission (EHRC) should investigate.
She also described the prime minister's apology as "mealy-mouthed" but added there was "an acceptance that what he said was wrong".
Labour's shadow equalities secretary Marsha de Cordova said the report was "a damning indictment of the discrimination rife in the Conservative Party, and it goes all the way up to the prime minister".
She urged Mr Johnson to make a "full and proper apology" for his burka comments and to acknowledge "the pain and hurt he has caused in the Muslim community".
The Muslim Council of Britain gave a "guarded welcome" to the report but said "it does not address the structural nature of Islamophobia in the Party" or "how racism had impacted many elements of its culture".
What is the report?
The report was commissioned in December 2019 after accusations of Islamophobic behaviour by some Conservative party members and representatives.
The review said such cases "give an impression to some of a party and leadership insensitive to Muslim communities".
Speaking to the inquiry, Mr Johnson said: "I am obviously sorry for any offence taken" and added he would not "use some of the offending language from my past writings" now.
Concerning his 2016 London mayoral campaign, the report said Zac Goldsmith - now Lord Goldsmith "accepts poor judgement in the way his campaign was conducted but forcefully denies harbouring anti-Muslim sentiments or using such sentiments for political advantage".
The report also found:
- From 2015 to 2020 the party's central database recorded 1,418 complaints concerning 727 incidents of alleged discrimination
- Two-thirds of all incidents reported related to allegations of anti-Muslim discrimination
- Three-quarters of all incidents recorded in the Conservatives' complaints database involved social media activity
- Sanctions were issued in 231 cases with 50% resulting in a suspension and 29% an expulsion from the party
- "No evidence" that complaints related to Islam are treated differently from those related to other forms of discrimination
"Judging by the extent of complaints and findings of misconduct by the party itself that relate to anti-Muslim words and conduct, anti-Muslim sentiment remains a problem within the party," the report added.
"This is damaging to the party, and alienates a significant section of society."
The Singh report said the Conservatives should, within six weeks, publish a plan laying out how it intends to tackle the failings highlighted by the investigation.
It also recommended producing a code of conduct and said at least one member of every party association should receive training on the complaints process.
Responding to Prof Singh's report, the Equality and Human Rights Commission said it would assess the report and await the Conservative Party's response.
Why was the report commissioned?
The Conservative Party has been accused of failing to tackle Islamophobia within its ranks for a number of years.
In 2018, the then-chairman of the Conservative Muslim Forum Mohammed Amin, accused his party of putting "electoral concerns" ahead of "decisive action" on tackling Islamophobia.
The Conservatives' first female Muslim cabinet minister, Baroness Warsi, has also long decried her party's response to the issue, accusing it of "turning a blind eye" and becoming "institutionally Islamophobic".
During the 2019 race to lead the Conservative party, then-home secretary Sajid Javid challenged all of his fellow contenders - including eventual winner Mr Johnson - to hold an external investigation into the allegations.