The allegations made by Mr O'Neill are that the President was a bad listener, that his tax-cutting agenda was being foisted on him by ideologically driven advisors and that his war against Saddam Hussein was fought in spite of there being no evidence that Iraq possessed weapons of mass destruction.
In response, Mr Bush tried to be genial and statesmanlike, thanking Mr O'Neill for his years of service and pointing out that regime change in Iraq had been President Clinton's policy as well. But behind the scenes the reaction has been vitriolic. The Treasury Department, which Mr O'Neill once ran, has begun what is described as an internal inquiry to see whether any of the documents used by the former minister in support of his case have been improperly removed from the archives.
It's likely that the row will die down without further action being taken and without great damage to the President but the episode is a reminder that not all senior Republicans think Mr Bush has made the right choices at home or abroad.