"To start with a clean slate and begin to instil methods to ensure England [are] moving in the right direction will be incredibly exciting," he said.
"But I do not feel like I am in a position to undertake that challenge."
Jonathan AgnewBBC cricket correspondent
"For some time, this England team have carried an arrogant attitude. That's not necessarily Andy Flower's fault, but if you are looking for change then there were two major candidates - Flower and captain Alastair Cook.
"There are very few candidates to replace Cook. Matt Prior was left out for the final two Ashes Tests and Stuart Broad is too much of a loose cannon to be considered for the top job just yet.
"Therefore, when you are looking for change - and a simplification of an England set-up that has become bloated with various captains, coaches and backroom staff - Flower's departure and replacement with one man to oversee all teams seems a clear choice."
Flower, who led England to three Ashes wins and the World Twenty20 in 2010, informed the England & Wales Cricket Board of his decision following a meeting on Thursday with new managing director Paul Downton.
He believes the next coach should be given control over all three England sides, a position Flower held before handing over responsibility for the one-day and Twenty20 sides in November 2012.
Former spinner Giles, who will lead England on their limited-overs tour of West Indies in February and March, is favourite to take the new role.
Ex-South Africa and India coach Gary Kirsten is thought to be another candidate for the job.
A decision is expected within two weeks.
Flower said: "Following the recent very disappointing Ashes defeat it is clear to me that this is now the time for England cricket, led by Alastair Cook, to rebuild with a new set of values and goals.
"In order for England cricket to make significant progress I believe that the team director, together with the respective captains, needs to be responsible across all formats in order to positively influence the rebuilding process.
"This will ensure complete clarity and continuity across the squads. Having stepped aside from the limited-overs squads 14 months ago, that is not something I am able to do and I do not therefore feel that starting the process would be in the best interests of all involved at what is a pivotal time for England cricket.
England should get Kirsten - Vaughan
"This has been a very difficult decision to make and I remain committed to England cricket."
Flower's team completed a third consecutive Ashes series victory with a
3-0 home win
last summer but despite arriving down under as favourites this winter, they were heavily beaten in all five Tests and have now slipped to fourth in the Test rankings.
Having seen England
bowled out for 51 in Jamaica
in his first match, Flower then moulded his team into one of the most polished and professional units in international cricket.
After winning the Ashes back later that year, and retaining them in the 2010-11 series down under, he steered England to number one in the world Test rankings in August 2011 following a 4-0 home victory over India.
Flower also improved England's performances in the shorter forms of the game, coaching them to their first global limited-overs success at the 2010 Twenty20 World Cup, although the team slumped at the 2011 one-day World Cup.
After handing over one-day and Twenty20 duties to Giles, Flower remained in charge of the Test side, overseeing their first series win in India since 1984.
Flower had two stints as Zimbabwe captain during a decade-long international playing career in which he established himself as a world-class batsman and wicketkeeper, scoring 12 centuries, and averaging 51.54 in 63 Tests.
After quitting the international arena, Flower dedicated himself to county cricket and Essex, where he played for five seasons between 2002 and 2006.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.