The number of people living in England and Wales who could not speak any English was 138,000, latest figures from the 2011 census show.
After English and Welsh, the most reported main language was Polish, with 546,000 speakers, followed by Punjabi and Urdu.
Some 4 million - or 8% - reported speaking a different main language other than English or Welsh.
Meanwhile, the percentage of the population that is married fell from 51% in 2001 to 47% in 2011.
English or Welsh was the main language for 92% - or 50 million - of residents aged three and over. Of the 8% who spoke a different main language, most were proficient in speaking English.
Of those with a main language other than English, 1.7 million could speak English very well, 1.6 million could speak English well, and 726,000 could speak English, however not well. The remaining 138,000 could not speak English at all.
In Wales, 97% - 2.9 million of residents - reported English or Welsh as their main language and 19%, 562,000, reported that they could speak Welsh.
In London, 22% - or 1.7 million of residents - used a main language other than English.
The top 10 reported languages were English, followed by Polish, Panjabi, Urdu, Bengali, Gujarati, Arabic, French, Chinese (excluding Mandarin and Cantonese, which were the 27th and 40th most commonly-used languages) and Portuguese.
The figures also showed that not all languages were spoken - with 22,000 people using sign language.
This was the first census to ask how well the population could speak English when it was not a person's main language.
Rise in households
The latest statistics offer a more detailed snapshot of the population in 2011 and a new level of geographical detail relating to the characteristics of those living in England and Wales.
On the make-up of homes in England and Wales, there was a total of 23.4 million households in 2011, an increase of 8% from almost 22 million in 2001.
In 2011, there were 14.4 million one-family households, 7.1 million one-person households and 1.9 million "other households," which includes households of unrelated adults or more than one family.
The largest percentage increase was for "other households", which rose 28% between 2001 and 2011.
The census revealed that in 2011, the majority of 16 to 74-year-olds drove a car or van to work - 58% or 15 million - while a further 5%, or 1.4 million, travelled to work as a passenger in a car or van.
The second most reported method of travel to work was by foot, 11%, or 2.8 million, followed by bus, minibus or coach, 7%, or 1.9 million.
In London, 50% - or 2 million - of the population used public transport to get to work, compared with 31% - 1.3 million - travelling by car, motorcycle or taxi.
In terms of health, 81% reported their general health as either "very good" or "good".
People living in London and south-east England recorded the highest percentages of "very good" or "good" general health, and those in Wales and north-east England the lowest.
Meanwhile, more than 10 million people reported their daily activities in 2011 were limited because of a health problem or disability, with the North East recording the highest percentage at 22% and London the lowest at 14%.
The London borough of Wandsworth had the lowest percentage of people with limitations on their activity at 11%, and Neath and Port Talbot in Wales the highest at 28%.
The census - which is used to plan public services - is carried out every 10 years, during which the public are asked questions about their jobs, health, education and ethnic background.
This year's questionnaire was sent to about 26 million households in England and Wales on 27 March 2011 and was compulsory to fill in.