The Flying Scotsman could be banned from main line tracks after people trespassed to catch a glimpse of it.
There was chaos between Derby and Birmingham last Sunday as fans vied to spot the legendary loco on its UK tour.
The situation was blamed for a string of delays to normal services, with reports of people refusing to move when challenged by drivers.
Network Rail said a ban would be a "move of last resort" but could not be ruled out if lives were being risked.
It would not let "a few thoughtless lawbreakers" cause dangers and delays, it said.
Nearly 60 services were delayed for a total of 1,000 minutes as Flying Scotsman complete its tour of the Midlands last weekend, British Transport Police (BTP) said.
The force has issued an image of two photographers it wants to trace, saying it was "extremely disappointing that a small minority of rail enthusiasts put their lives in grave danger".
One passenger service driver, who asked to remain anonymous, said he saw trainspotting trespassers every 200 yards.
Describing it as "probably the most stressful experience I have ever had", he said it was "only a matter of time before someone is seriously injured or killed" trying to get a photo.
Nick Brodrick, editor of Steam Railway magazine, said the problem was "deeply troubling".
"When you have drivers having to stop, get out and tell trespassers to move and even then be ignored, the situation is simply unacceptable," he said.
He said it would be hard to argue with a ban if someone was injured or killed.
A Flying Scotsman spokesman said a ban "would not be a surprise" but every effort, such as CCTV and extra police on the train, was being taken to avoid the situation.
The tour ends in Scotland on Friday.