Google says its Translate app can now act as an interpreter, with the addition of a real-time voice-translation mode.
It said the updated app would automatically recognise languages being spoken and translate them.
The update, launched on Wednesday, also allowed users to instantly translate messages using their phone's camera.
But one academic said it would fail to understand the more complex linguistic tools.
"For basic things, it might be very useful. My mother, for example, does not speak any other languages, but loves travelling, so she could find her way around a town.
"But it is never going to pick up the nuances, the cultural references or the humour," said Ariane Bogain, a senior lecturer in modern foreign languages at the University of Northumbria.
Prior to the update, Google's app could translate spoken or typed phrases and repeat them out loud. But it worked phrase-by-phrase, rather than in real time, and the pronunciations and rhythm of speech could sometimes be problematic.
Users of the new app can also use their phone's camera to instantly translate phrases, using the Word Lens feature, which works without a wi-fi or a data connection.
Google said: "The Translate app already lets you use camera mode to snap a photo of text and get a translation for it in 36 languages. Now, we're taking it to the next level and letting you instantly translate text using your camera.
"While using the Translate app, just point your camera at a sign or text and you'll see the translated text overlaid on your screen - even if you don't have an internet or data connection."
It said the feature was available for English to and from:
Google said users could also "tap the mic to get into voice translation mode, tap the mic again, and the Google Translate app will automatically recognise which of the two languages are being spoken, letting you have a more fluid conversation".
The company said the update took people "one step closer to turning [their smartphone] into a universal translator and to a world where language is no longer a barrier".
While she acknowledged the app's usefulness to holidaymakers struggling with the language, Ms Bogain said that major events, such as an EU summit, were not likely to begin using it.
She added that online translations conveyed the rough message - "but you are going to lose a lot in the translation".
Ms Bogain said: "One word can have various meanings, depending on the context, and I do not think that online translation tools are there yet.
"As lecturers, when we ask students to hand in translations, we can immediately spot who has used a translation tool because the phrasing is not natural. The message is roughly there, but something is lost.
"I do not think it is going to replace 15 years of training."
Other voice translation apps are already on the market, and Skype launched the function last month.
According to Google, more than 500 million people use its version every month to make more than one billion translations per day.
Google said the update would be available to both Android and iOS over the course of this week.