The more powerful your phone gets, the more privacy you have. Strange but true! Things have taken an unexpected turn when you weren’t looking.
For years “the cloud” has been a magical phrase that means all the heavy lifting in our tech lives is happening online. Our phones and computers are barely more than dumb terminals communicating with online megabrains that have all the data and all the processing power. We’ve been taught to believe that’s where “artificial intelligence” happens, whatever it is, in huge buildings stuffed with servers run by Google and Amazon and Microsoft.
The cloud services make our lives better. We talk to our phones and our homes, we navigate and store our photos and do business and communicate, yay 21st century. But there’s a tradeoff that we try not to think about: the cloud can only do all the coolness for us if everything we say and do is sent online to Google and Amazon and Microsoft. All the words we speak to Alexa and Google (and in our paranoid fears, all the words we speak anywhere because we’re afraid the devices are always listening) (they’re not) – only a couple of years ago those words had to be sent online for the massive cloud servers to interpret and translate and create a response to be beamed back to our phones.
Three important things have happened in the last few years.
The first is the global backlash against the tech companies – criticism of the vast amounts of data hoovered up as they handle our searches and use their devices. Governments all over the world are threatening to take action against the companies for privacy violations – which tend to be vaguely described and perhaps not all that meaningful when you get down to the pesky details, but our world does not do well with details these days.
The second is the success Apple has had with its claims to be the privacy company. Again, those claims are a bit overblown when scrutinized closely, but Apple’s emphasis on privacy has clearly resonated with consumers. Google is determined to build its own reputation as a privacy-focused company.
The third is the new part that is changing our world. Phone processors have become so incredibly powerful that they can do a lot of work on the phones without sending anything to the online super-servers.
The result is that the tech companies are starting to collect less data about us. Google and Apple are finding ways to get just as much revenue without quite as much individualized data.
When you speak to a device – to dictate a text or ask for a translation, say – the audio used to be sent online where huge trained servers would try to figure out what you said in your impenetrable accent slurred by alcohol, then send back an answer – the words of the text or the translation into Finnish.
Sundar Pichai announced big steps forward in privacy on Google devices two years ago. Everything that could be done locally would stay on the phone. Apple has been taking the same approach on iPhones, although Google is leading the way.
Today, on some devices, the phone does all the work. Nothing is sent online if it’s possible to do it locally.
Pixel phones have extraordinarily accurate voice recognition, all done by the processor on the phone. It’s a combination of processor power, smart AI built into the phone, and clever work to shrink databases and access them more quickly. When you dictate a text message on a new Pixel phone, Google doesn’t just pinky swear not to store a copy of what you said. Instead, your words never leave the phone, period.
The translation dictionaries have been boiled down to impossibly small databases that can be stored on the device.
Facial recognition on Google Home devices is done on the device. When you walk in the kitchen with a Next Home Max on the counter, the device recognizes your terrifying morning face but your picture is not sent to Google.
Live real-time transcription of videos, lectures, and meetings is done on the phone almost instantaneously. You can turn on transcription during a lecture and see the words appear like closed-captions, ready to be saved as notes and searched.
When you speak to Google Assistant, as much as possible is done on the device. If you ask Google for information, it gets the answer online, but it does not send a recording of your voice to get the answer.
Here’s an amazing example. A Pixel phone has a database of every popular song released in the last 50 years. The “Playing Now” feature listens to the song playing in the background and puts the song name and artist on the Android lock screen. It does that without sending a single note online to Google – all the identification is done on the phone. The music database was already down to 500 Mb – megabytes! – in 2017 and might have been shrunk further since then. Google then continuously updates the database:
Now Playing uses a privacy-preserving technology called federated analytics. It figures out the most frequently recognized songs across all Pixel phones by region, without revealing which songs are heard by any individual phone. Using this aggregated data, it updates the on-device song database with the songs people are most likely to hear, without Google ever seeing what you listen to.
As of a couple of weeks ago, Android 12 adds a search button when a song is playing that the phone doesn’t recognize. And it still doesn’t upload a recording to Google! The phone creates a digital fingerprint that its online servers can analyze without knowing what the song is or associating anything with you individually.
(Oh, and a fun fact. If you bring up Google Search and tap the microphone, you can hum a song and Google will try to identify it. It will be more successful than you expect. Google engineers are very smart.)
Google just announced a new security feature coming to Android phones: a separate partition, the “Private Compute Core,” to work with all of this data on the phone while keeping it encrypted and heavily guarded. “This means that all sensitive audio and language processing happens exclusively on your device and isolated from the network to preserve your privacy,” Google explained.
Google’s commitment to security and privacy is pretty impressive. There’s more information here about the safety of Pixel phones and Nest home devices.
This is the real significance of Apple’s customized processor for iPhones and Google’s customized Tensor chip introduced with the new Pixel phones. The processors are more powerful and use less power, sure, but the important thing is that the companies can use the power to keep your information on the phone instead of scooping it up into their cloud databases.
As our phones and home devices become more powerful and have more storage space, Google and Apple are committed to doing more on the device with less information transmitted to the cloud. It’s the exact reverse of what we expected the cloud future to be. And it permits protection of our privacy in profound ways. Isn’t that nice?
