Justice Secretary Chris Grayling says new prison policies are not denying inmates access to books, despite criticism from high-profile authors.
Regulations introduced in November stop people sending books and other items to prisoners in England and Wales.
Prison campaigners say it is effectively a "blanket ban" that restricts education and rehabilitation.
But the government says prisoners can still use prison libraries or earn money to buy books.
Under the changes, prisoners are no longer allowed to receive small parcels from outside containing items such as underwear and magazines.
Mr Grayling, writing for the politics.co.uk website, stressed the new changes had been brought in to encourage good behaviour.
The justice secretary was responding to a piece by the chief executive of the Howard League for Penal Reform, also for politics.co.uk.
Frances Crook described the "book banning" policy as the "most despicable and nastiest element of the new rules", adding it was part of an "increasingly irrational punishment regime orchestrated by Chris Grayling".
"The ban on receiving books is a blanket decision, so no matter how compliant and well behaved you are, no prisoner will be allowed to receive books from the outside," she wrote.
Authors have also criticised the policy, while an online petition has been set up, receiving almost 13,000 signatures.
Mark Haddon, author of The Curious Incident of the Dog in the Night-Time, said he thought he was "reading things from another century" when he saw Ms Crook's article.
"Even prisoners in Guantanamo Bay can get given books as gifts," he told BBC Radio 4's Today programme.
"We give children books because they are unequivocally good things which make them better people.
"But we're apparently trying to make prisoners into better people by actually restricting their access to books as if we're dealing with two entirely different species of human beings."
He dismissed the government's argument that prisoners could still buy books, saying average earnings were £8 a week and that prisoners "have to use the money to buy everything - coffee, clothing, toothpaste, phone calls".
Philip Pullman, author of the His Dark Materials trilogy, told the Guardian it was "one of the most disgusting, mean, vindictive acts of a barbaric government".
'Privileges and incentives'
Juliet Lyon, director of the Prison Reform Trust, said: "Banning prisoners from receiving books in prison is just one of a number of mean and petty rules introduced by the justice secretary which add to the stress and strain of imprisonment while doing nothing to promote rehabilitation."
And the charity Booktrust said in a statement: "Surely our efforts should be on encouraging more people in prison to read rather than punitive action to stop those that want to?"
But Mr Grayling said prisoners were still allowed up to 12 books in their cells and also had access to prison libraries, if they behaved well.
"We believe offenders need to behave well and engage in their own rehabilitation if they are to earn privileges and incentives," he said.
He said it was "never the case" that prisoners were allowed unlimited parcels and that the government had "introduced consistency across the estate".
Prisons minister Jeremy Wright, meanwhile, said a major reason restrictions were in place was to stop things like drugs being smuggled in.
"The brutal reality here, which I think we all need to recognise, is that just because a package comes into prison marked 'This contains books', doesn't mean we don't have to check it to make sure that it does, in fact, just contain books," he told Today.
"What's being suggested is that we should have very few limits on the packages that come into prison - that clearly isn't feasible."
He added: "It's also sensible because we're trying to change the system so prisoners earn the creature comforts that they have in their cells."
Shadow justice secretary Sadiq Khan said the policy was an example of "the skewed priorities of a justice secretary who has no solutions to the problems in our prisons".
And John Podmore, a former prison governor and university professor, told BBC Radio 5 live that books were being treated as "luxuries" when prisons should be "encouraging their use not restricting them".