You can record speech on a Pixel phone with Google’s new Recorder app, and it will transcribe the words into text in real time. You can save the text and search for words later.
You can see captions on any video or spoken audio on your phone. (Not phone calls or video calls – those bring up thorny privacy issues.)
Sounds simple, doesn’t it? It’s not. It’s another example of Google using its AI prowess to do something that looks easy; something that can be genuinely transformative; and something that is simply not possible for any other company to do as well.
For some people, the real-time transcription is a sufficient reason by itself to buy a Pixel 4 phone.
What is “real-time transcription”?
Pixel 4 phones come with a Recorder app. It’s simple, just an empty screen with a big Record button. Push the button to start recording.
The microphones on Pixel phones are quite good. You don’t have to hold the phone up to your face like all the people walking around with bizarre horizontal iPhone slabs next to the corner of their mouths. If Recorder is turned on and the phone is not too far out of sight, it will happily record a lecture or all the people in a conversation.
The recording is saved as an audio file, of course.
But it is also transcribed on the fly, nearly instantaneously. You can push the Transcript button and watch the text appear as it’s spoken. The text is saved alongside the audio file. The transcript does its best to insert punctuation, and it puts in paragraph breaks at pauses.
(Side note: Google is working on it, but the transcription doesn’t distinguish multiple voices yet. It’s up to you to separate the speakers if you record a conversation. The Recorder team is working on this for a future update.)
You’re a student. You sit in a classroom and push the record button when the professor starts the lecture. The phone sits on the desk in front of you.
When the professor has finished, you have a recording and a transcript of the lecture.
This is Google, so add one more thing: the text is immediately searchable. Type in a word or a phrase and it will pop up highlighted in the transcript.
Click on a word or phrase and the audio will jump immediately to that point in the recording.
Are you starting to see how this can be used? Students recording lectures. Journalists recording interviews. Business people recording meetings. Creating notes for yourself if you have a fallible memory. Saving family memories from a relative. Making the world more accessible to hearing-impaired people.
This is magic.
What about “Live Caption”?
There’s a similar feature on Pixel phones for video. A single click turns on live real-time captions for any video playing on the phone (except live conversations with Google Duo or Skype).
YouTube has had captions for all videos for some time – click the CC (“Closed Caption”) button under each video.
This new feature adds captions to every other kind of video you might play – Amazon Prime movies, videos on websites, videos in Google Photos, anything.
Examples from TechCrunch: “If you’re in a loud environment, like a commuter train, or trying to watch content privately and forgot your headphones, you may need to just use the captions. Or maybe you don’t want to blare the audio, which disturbs others around you. Or perhaps, you want to see the words appear because you’re having trouble understanding the audio, or just want to be sure to catch every word.”
On a Pixel 4, the volume control that slides in from the right now has an extra button at the bottom to turn Live Caption on or off.
What about security and privacy?
Although Google gathers more data about us than any other single company, it also continues to be more careful about security and privacy than almost any other tech company (*cough* Facebook).
100% of the real-time transcription is done on the phone. No cloud communication is required. You can turn on transcription or live captions with the phone in airplane mode.
This is an engineering marvel. No one but Google is even close to being able to do this. I cannot imagine how they got the code so small and efficient. Reportedly early versions would run down the battery and bog down the processor. Now it’s featherweight in memory and processor load.
That’s not all. Recordings are encrypted and stored only on the phone. A recording never leaves the phone unless you explicitly share it. You have the option to save a recording to Google Drive or share it with any of the usual Android sharing options. If you email a recording, the recipient gets a .m4a audio file plus a .TXT file.
To recap: Google’s cloud servers do not process a recording while it is in progress, and Google never gets a copy of your recording unless you use their services to store or share it.
What phones does it work on?
For the moment, Recorder is only available on Google Pixel phones.
Previously, Android didn’t have a built-in Recorder app, although many were available from third parties in the Play Store. The new Google Recorder app is included on Pixel 4 phones, and in the last three weeks, it has been made available in the Play Store for earlier models of Pixel phones.
It is not available on other Android phones. There is no clear word on whether it will be extended to all Android devices, or if Google will keep it exclusive to the Pixel line.
There are other Android apps that transcribe recordings. Reportedly Otter does transcriptions as well as or even slightly better than Google Recorder. Otter is free for basic services; there is a paid subscription for more than 600 minutes per month, or if you want some extra services. The difference is that Otter and the other services upload your recordings to the cloud, which introduces some security and privacy concerns.
Otter or another similar app is also your only option if you have an iPhone. Google Recorder is not available on an iPhone, of course, and Apple’s built-in Voice Memos records audio but doesn’t do anything to make it useful or searchable.
If you have a Pixel phone, check out Recorder. Download it from the Play Store if you have a Pixel 2 or 3.
If you don’t have a Pixel phone, but you think this could change your life – well, Pixel 4s are being sold for less than $500 on sale this month. Nice phones. Amazing cameras. Something to think about, eh?
