A historic 1830s railway works is due to be demolished to make way for new homes after a High Court ruling.
Last year Milton Keynes Council granted planning permission for 375 homes to be built on the site in Wolverton.
Historic England objected saying it was "very concerned" about the impact on a nearby conservation area.
Judge Mr Justice Dove dismissed the challenge and said Historic England had "not demonstrated any illegality in the council's decision".
Wolverton railway works are still active today and consist of large train sheds and workshops - some of which date back to the mid-19th Century.
As part of the redevelopment plans, all structures on the site will be demolished apart from an old lifting shop and a retaining wall.
Housebuilders, St Modwen Developments Ltd, have agreed to keep gable ends on three buildings as a "vestigal" reminder of the site's history.
Mr Dove said the council's planning committee voted in favour of the plans by a majority of six councillors to four.
A planning officer's report stated the site is "too large for a modern railway or business".
Several buildings on the site are said to be derelict, or nearly so, and "beyond their operational lives".
The planning officer acknowledged the harm to Wolverton Conservation Area, but said there was "no other means of delivering the same benefits".
The case was taken to the High Court after Historic England argued the council had unlawfully failed to give full reasons for its "irrational" decision to grant planning permission,
Milton Keynes Council admitted it had made a "technical error" in failing to inform objectors of their right to seek a judicial review of the decision.
Mr Dove said that was not a good enough reason for overturning the permission.