Theta Android News: Here’s a first, real look at Android 10 Live Caption feature in action

Tuesday, October 1, 2019

Here’s a first, real look at Android 10 Live Caption feature in action

Live Caption is arguably the coolest Android 10 feature around, delivering captions for local and web videos thanks to on-device machine learning. Best of all, it doesn't require an internet connection to work.

Android 10 is out already, but we haven't seen Live Caption hit devices yet. Fortunately, XDA-Developers has obtained the Pixel 4's "Device Personalization Services" app and tweaked things to get Live Caption running on the Pixel 2 XL.

A setup screen also gives us a better idea of how it works, revealing that the feature can censor profanity and can appear in the volume control section. Furthermore, you can hold on the caption to drag it around or double-tap the caption to expand it. The setup page also notes that songs aren't fully supported — although this is an understandable move. We also get a screenshot of the feature's toggle in the quick settings menu. Check out the screens below:

XDA noted that they successfully tested Live Caption in YouTube, Amazon Prime Video, Google Photos (seen above), Google Podcasts, and Netflix. The feature also works in both portrait and landscape mode — check out Amazon Prime Video in landscape mode below.

XDA-Developers

The outlet added that it provided reasonably accurate results without needing to turn the volume up. Unfortunately, a Google API restriction reportedly means this feature won't work in phone/VoIP/video calls, but hopefully this changes in the official release.

We've got to assume that the Pixel 4 series will be first to gain Live Caption, as Android 10 is already out for several devices but the feature isn't present in these releases. Here's hoping the likes of the OnePlus 7 series — which have already received Android 10 — get an update to enable this feature then.

It's also pretty encouraging to see that the Pixel 2 XL is able to run Live Caption. The phone uses a flagship processor that's almost three years old, and it still seems to have enough grunt to run the required on-device machine learning for the service. This bodes well for current mid-range phones, such as the Pixel 3a. Hopefully Google optimizes Live Caption to the point where even low-end devices are able to access it.