Istanbul, Tokyo and Madrid will compete to host the 2020 Olympics and Paralympics after Qatar's Doha and Azerbaijan's Baku were cut from the list on Wednesday.
The shortlist was announced at the International Olympic Committee's executive meeting in Quebec City.
Doha and Baku were rejected for a second time in a row after failing to make the final list for the 2016 Games.
The winning host city will be named on 7 September 2013 in Buenos Aires.
The 15-member executive board, headed by IOC president Jacques Rogge, chose the finalists after examining a technical evaluation report compiled by a panel of Olympic experts.
Japan's Tokyo hosted the Olympics in 1964, while neither Istanbul in Turkey nor Spain's Madrid have held the Games. Madrid is bidding for a third consecutive time, Tokyo a second time in a row and Istanbul a fifth time overall.
Doha, which was proposing to hold the 2020 Olympics in October rather than the usual July/August schedule to avoid the Gulf Arab state's searing summer heat, is already hosting the 2022 World Cup football tournament.
"This is a great disappointment for the Doha team," said Noora Al Mannai, chief executive of Doha 2020. "With so many sports venues already in place and budgeted for, we felt that we offered the IOC great certainty and a low cost Games plan as well as an exciting legacy vision, especially around developing women's sport in the Middle East.
"However for Doha, it will always be a question of when not if."
Rome pulled out of the running in February because of the country's efforts to head off a debt crisis.
London will host the 2012 Summer Games from 27 July - 12 August, while the Brazilian city of Rio de Janeiro will stage the 2016 Olympics and Paralympics.