Fifa president Gianni Infantino wants to address the "sensitive issue" of football at the Olympics.
Brazil won the men's tournament for the first time at Rio 2016, beating Germany at the Maracana Stadium.
However, while Infantino feels the women's game has developed due to its inclusion in the Olympics, he says problems remain to be solved.
"The men's game, which is primarily for under-23 players with a few added, is not a good solution," Infantino said.
"The tournament is neither fish nor bird in this form. Also, the football calendar is too tight."
Apart from 1896 and 1932, football has been in every summer Olympics, with the women's tournament added in 1996.
Sixteen teams contested the men's tournament in Brazil, with 12 sides in the women's competition in which Germany beat Sweden in the final.
Following legal action by Barcelona, Werder Bremen and Schalke in 2008, clubs are no longer required to automatically release their players because the competition falls outside the recognised Fifa international calendar.
Take part in our new Premier League Predictor game, which allows you to create leagues with friends.
Subscribe to the BBC Sport newsletter to get our pick of news, features and video sent to your inbox.