Youngsters have told Newcastle City Council bosses they would be unable to use a community library if a drug and alcohol recovery hub opens there.
They spoke out during a public meeting over plans for Fenham Library.
One said she "would not feel safe" using it, while another said she would be "scared" walking past it to school.
The authority said the proposed recovery services would not pose a risk to the community and were vital to keep the library from closing.
Kim McGuiness, cabinet member for sport, culture and public health at the council, said: "This is the sort of place you'd expect someone to be referred at the end of their treatment or by a GP to see a counsellor.
"There are no clinical services whatsoever in this building, I think that's really really important. No pharmacy, no needle exchange; this is a therapeutic centre."
The public meeting at the Robert Stewart Memorial Church on Thursday evening was described as "heated", and the council has already faced criticism for lack of consultation.
One 11-year-old girl said: "I really like using the library, but after you change it my mum will be too busy to bring me and she would have to if it is exclusively for people recovering from drug addiction."
Another young girl asked why schools had not been consulted on the plans, and a sixth-form student said she would be scared of walking to and from school for the first time in her life.
Eugene Milne, the council's director of public health, said he regretted the lack of consultation with residents and schools, but said: "Right now we are dealing with swingeing cuts.
"This is a way to deal with that, otherwise the library is going to go."