Parts of the civil service will be moved out of London to other English regions, Cabinet Office Minister Michael Gove has said.
Speaking to the BBC he said it was "vitally important that decision makers are close to people".
He also said proposals to move the House of Lords to York were "a matter for Parliament".
The Institute for Government estimates that 83,500 out of a total of 430,000 civil servants are London-based.
More civil servants work in the capital than any other part of the UK.
Speaking on the BBC's Andrew Marr Show, Mr Gove said moving more branches of the service out of London would help to distribute "opportunity, jobs and investment fairly" across the country.
"We've already got civil servants in Scotland, who are working for the Department for International Development, and in Wales, working for the Department for Transport - but we can do more," he added.
Asked about suggestions the House of Lords could be moved to York, Mr Gove said: "As far as the legislature goes, that is obviously a question for the House of Commons and the House of Lords.
"But my own view? I think that, if people were to see Parliament closer to different parts of the United Kingdom, then I don't see there are any reasons why we can't have more operations of the UK Parliament in Scotland, Wales and Northern Ireland."
The government has made clear its desire to change the civil service and, in a speech in June, Mr Gove said major changes to the service were needed to tackle "group think".
Other parties have also called for more parts of Whitehall to be moved out of London. In its 2019 manifesto, Labour proposed moving part of the Treasury to northern England.