There is no reliable data to identify EU nationals in the UK or the length of their stay in the country, immigration minister Robert Goodwill has said.
He told MPs this lack of detail would not affect Brexit negotiations as he could not foresee a situation in which all EU nationals were told to leave.
Ministers say they cannot guarantee EU nationals living in the UK the right to stay without reciprocal assurances.
But Labour's Chuka Umunna said this stance was now just a "pretence".
Prime Minister Theresa May has said she would expect to guarantee all EU citizens currently living in the UK the right to remain after the UK leaves the EU but this will depend on other EU countries offering similar assurances to British citizens living there.
Campaigners say this means the two million EU citizens estimated to be living in the UK have been effectively "left in limbo" and they must not be used as "bargaining chips" in Brexit discussions over issues such as free movement and access to the single market.
Giving evidence to the Home Affairs Select Committee for the first time since he was appointed in July, Mr Goodwill said EU nationals made a major contribution to the UK economy and society - in areas such as the health service and agriculture - and he could not envisage a circumstance in which they were told to go home.
Pressed by Mr Umunna, he said the government did not have the means of tracking down all EU nationals were it deemed necessary to remove them.
"No we are not in a position and I could not foresee a circumstance where we would want to be in that position," he said.
"I can't see a situation in which we would even think of that."
Citizens of other EU countries living in the UK who are not working do not necessarily have national insurance numbers, resulting in gaps in the official records.
Mr Umunna said the disclosure that the UK government "did not have a clue" about the number or identity of EU nationals undermined the government's whole public position and ministers should stop "carrying on with the pretence" that they had the ability to ask all EU nationals to leave.
"If you can't identify all the EU nationals in our country and be in a position to remove them, what on earth is the point carrying on with the pretence that somehow if you weren't to guarantee them the right to stay, you could get rid of them," he said.
Mr Goodwill replied: "I can see the sort of route you are trying to take me down but it is not a route I think we are ever going to be going down."
The unwillingness of the government to offer a concrete guarantee to EU nationals was not a "bargaining ploy" but a reflection of "how things are" pending the start of official negotiations, he said.
But he added that he couldn't see countries like Poland clamouring "to have their nationals returned" as part of the Brexit deal.
Earlier, Mr Goodwill said the government had not decided yet whether to introduce a cut-off point after which any automatic right to stay would no longer apply.
He said a number of options existed, including the date of the EU referendum, the day on which Article 50 was triggered or the day on which the UK actually left the EU.
Officials have said that anyone who has been living in the UK for five years would have a virtual guarantee as they would be able to apply for permanent residence.