Huawei has announced plans to pre-install its own Harmony operating system on its smartphones from next year.
The Chinese company said it would also offer the software to other manufacturers to use as an alternative to Android.
Huawei is currently the world's second bestselling handset-maker, after a brief time in the top spot.
But others have tried and failed to challenge Google and Apple's dominance.
Google's Android accounted for 85.4% of smartphones shipped last year, according to research company IDC, and Apple's iOS the remaining 14.6%.
And those figures reflect the fact Samsung's Tizen, Amazon's FireOS, Microsoft's Windows Phone, and Canonical's Ubuntu have all failed to make headway on handsets.
Huawei's move has been prompted by the fact it can no longer offer Google's apps and services on its latest devices, because of a US trade ban - although the restriction does not prevent it from offering Android itself.
Within China - where consumers do not use the Google Play store and many of Google's services are blocked - this limitation has not caused Huawei problems.
But in other countries, demand for its most recent phones has been weak because Google's tools are popular.
One expert said the fate of the new operating system might depend on how many other technology companies Huawei convinced to come on board.
"This move will be supported by the Chinese government because it fits into its wider Made In China 2025 strategy," Marta Pinto, from IDC, said.
"But it will only take off elsewhere if other Chinese vendors, such as Xiaomi and Oppo, adopt it.
"Even then, it will still be a challenge in geographies like Western Europe and Latin America, where so many people and businesses rely on Google's products."
Huawei announced its plan at the start of a three-day developer conference near the city of Shenzhen.
The original version of Harmony OS was unveiled a year ago, when it was pitched for use in smartwatches, TVs and other smart home gadgets.
But the company now intends to release a fresh version - Harmony OS 2.0 - that can be tested on handsets from December, ahead of a formal launch in October 2021.
In parallel, it will shortly release EMUI 11 - a version of its mobile phone user interface based on Android 11.
After October, some of its smartphone models will be offered with Harmony OS.
But it will continue to offer EMUI as an alternative.
Part of Huawei's challenge is it will need developers to code their apps specifically for Harmony if the software is to run natively and thus get the best performance.
The company has indicated it will be relatively easy to recode apps already written for Android.
But this has proved a sticking point for other failed operating-system challengers.
Developers either decided the extra work was not worth it or did not make it a priority, meaning the apps typically lacked the latest features available on Android and iOS.
"Huawei has the engineering talent, the ambition, and a home market advantage," Ben Wood, from the consultancy CCS Insight, said.
"And if the Chinese government got fully behind Harmony and made support for it a condition for other companies to offer their products and services in the country, that would provide an added catalyst for Huawei to establish a third smartphone platform.
"But in terms of global aspirations, history shows it's still a very difficult thing to achieve."
Huawei faces other immediate concerns.
From 15 September, the company will be unable to get more of its Kirin processor chips manufactured, because of further US trade restrictions.
Samsung and SK Hynix will no longer be able to to sell it memory chips.
And it has been reported Samsung and LG will be unable to continue selling it OLED displays for its smartphone screens.
Huawei has indicated it has built up stockpiles of components to get through the immediate future.
But the fate of its consumer division may rest on Washington changing its position after the US presidential election.