The Manchester Arena bomber had been a "subject of interest" and opportunities to stop him were missed, a review says.
Its author, David Anderson QC, said it was conceivable Salman Abedi's attack, which killed 22 people, could have been avoided had "cards fallen differently".
But he said it was "unknowable" whether reopening investigations into Abedi would have thwarted his plans, adding: "MI5 assesses that it would not."
Greater Manchester Police said its officers would "never stop learning".
After the Manchester bombing and three terror attacks in London this year, counter-terror police and MI5 conducted internal reviews. Mr Anderson carried out an independent assessments of their findings.
The reviews, which remain largely secret, are summarised in Mr Anderson's report, and show:
- Abedi had been a "Subject of Interest" for MI5 - meaning someone they were investigating - between January and July 2014, and then again in October 2015
- On two occasions in the months before Abedi attacked, MI5 received intelligence, but its significance was not fully appreciated at the time and, in hindsight, was "highly relevant" to the planned attack
- Abedi was identified by MI5 as one of "a few dozen" people who needed further consideration. The meeting to do that was due to take place on 31 May, nine days after the attack
- There was no security service port alert against Abedi, so he was not questioned at the border when he returned to the UK from Libya four days before the attack
Butt had been identified by MI5 and the police as someone who wanted to attack the UK two years earlier.
He was still a "live subject of interest" who was under investigation at the time of the attack, though more for his intention to travel to Syria and for radicalising others.
He was also the main target of "Operation Hawthorn" - but this was suspended twice because of a lack of resources after the Bataclan attack in Paris and the Westminster Bridge attack.
Operation Hawthorn had resumed and was running on the day Butt attacked.
Mr Anderson, a former independent reviewer of terror legislation, said: "Despite elevated threat levels, the fundamentals are sound and the great majority of attacks continue to be thwarted.
"But the shock of these incidents has prompted intensive reflection and a commitment to significant change.
"In particular, MI5 and the police have identified the need to use data more effectively, to share knowledge more widely, to improve their own collaboration and to assess and investigate terrorist threats on a uniform basis, whatever the ideology that inspires them."
Could MI5 really have stopped the attacks?
By Dominic Casciani, BBC home affairs correspondent
It says not - but Mr Anderson believes there were opportunities. Given the scale of terrorism-related activity since 2013 - when the Syria crisis gave a boost to recruitment in the UK - there is no doubt that the security service has been juggling a huge numbers of cases.
The real question is whether the manpower is matched with the right data tools and relationships with other bodies to stop more of the threats before it is too late.
Data analysis will play an increasingly important role in trying to spot individuals who may pose a threat after years of being quiet.
Perhaps the most important change to come is that MI5 may be told to share some of what it knows with other agencies - such as local councils - in the hope that people on the ground can provide the missing piece of information they need to disrupt a threat.
This raises huge cultural challenges for an organisation that necessarily operates below the radar.
Some relatives of victims have given their views on the report.
Steve Goodman, whose step-granddaughter 15-year-old Olivia Campbell-Hardy was killed in the blast, said: "The police were doing their jobs as best they could.
"Unfortunately information is not always reliable."
Dan Hett lost his brother Martyn, 29, in the explosion.
In a series of tweets said he could not "fathom how complicated modern antiterrorism intelligence is".
He added that the positive aspects of the emergency services' response should also be highlighted.
Terror attacks this year
- A man in a hired car drove into pedestrians on Westminster Bridge then stabbed a police officer outside Parliament on 22 March
- A suicide bomber targeted young people at the end of a concert by US singer Ariana Grande at Manchester Arena on 22 May
- A van hit pedestrians on London Bridge before three men got out and stabbed people in nearby Borough Market on 3 June
- A group of Muslim worshippers were hit when a van mounted the pavement and drove into them in Finsbury Park, in north London, on 19 June
The home secretary said nine terror attacks had been prevented in the UK since the Westminster attack in March.
In a statement to the Commons, Ms Rudd said MI5 and the police had made 126 recommendations.
These included issues such as data sharing and analysis and how so-called "closed subjects" should be managed, as well as a new approach to managing domestic extremism, particularly of right-wing groups.
"We will shortly be announcing the budgets for policing for 2017-18, and I am clear that we must ensure counter terrorism policing has the resources needed to deal with the threats that we face," she told MPs.
Mr Anderson's predecessor Lord Carlile said the 126 recommendations should be "put into effect as soon as possible".
The Met Police said the number of dangerous, radicalised individuals was "a major issue".
Commissioner Cressida Dick said her force needed "to make rapid progress in implementing the recommendations, many of which require new technology, better infrastructures and resources".
Following the publication of the report, the prime minister's spokesman said the government would pay £9.8m in special funding to Greater Manchester Police, in relation to its response to the Manchester Arena bombing.
The Mayor of Greater Manchester Andy Burnham said the report would be difficult reading for the people of his city.
"It is clear that things could, and perhaps should, have been done differently and that wrong judgements have been made," he said.
But, he said, it should reassure the public to know MI5 were closing in on Abedi.
It would be much more worrying if nothing had been known about the attack, he added.
Chris Phillips, a former head of the National Counter Terrorism Security Office, said: "When you look back, within terrorism, you will always find some way that we could have stopped something.
"I always equate it to spinning plates. They've got hundreds and thousands of plates spinning at any given time.
"Someone has to make some risk-assessed decisions as to who is at the top of the pile to be watched."
The current threat level for terrorism in the UK is severe, meaning an attack is highly likely,