One thing that’s important to keep in mind when thinking about the new Apple Watch’s sound and listening-related features is the S11 chip. It’s designed with what Apple calls the “secure exclave,” which is where the raw audio coming in from the watch’s microphone is being processed. That’s where the sound is transcribed and processed, and only the results of that interpretation leave the exclave. It won’t store any of the audio it gets, Apple says.
With this, there are four updates related to sound. First up is the most straightforward: Improvements to Shazam on your watch. Right now, if you want to pull up Shazam to identify a song playing around you, you might scroll through the smart stack on your watch and whizz past or land on the Shazam widget. Then you’d tap it and wait for the system to listen and guess the tune. On the Series 12, because it has a buffer of the previous 15 seconds of audio information to work with, it can already start looking up the song once you scroll past the Shazam widget. This means it can potentially recognize music much more quickly.
Next is the sound recognition tool that has long been an accessibility feature in iPhones. This uses the mics in your iPhone, for instance, to alert people to important sounds like alarms, sirens, babies crying or glass breaking. The Series 12 will be able to deliver alerts about these sounds even if a companion iPhone isn’t close by. For people who are deaf or hard of hearing, this could be a handy expansion.
More interesting of the suite of updates here are the Live Rewind and Siri Recap features. Live Rewind can basically deliver a transcript of the last 15 seconds of what someone near you might have said at any given time. You just double click the dial on your Apple Watch Series 12, and a screen pops up, while the microphone indicator appears at the top right corner to show it’s pulling audio data. In my demo, I saw all these cues followed by onscreen text that certainly seemed to accurately transcribe what the Apple rep had said just seconds earlier.
I don’t really have the time at the moment to break down all my thoughts, concerns and understanding of the way this works just yet. The takeaway is that the Apple Watch is not technically recording audio and is instead producing a rolling buffer of audio that’s constantly overwriting itself inside its Secure Exclave. It’s then transcribing at the Secure Exclave level, and everything else it produces is based off that transcript, which means the effectiveness of these features has a lot to do with the quality of the models behind that process. Plus, it only starts transcribing when the button is pressed twice, so it’s not always transcribing.
Finally, Siri Recap creates “high-level” summaries of events, rather than a full transcript. It’s something you’d more actively set up to listen to your surroundings on either a schedule or manually each time you’re in a meeting, for instance. I wasn’t able to test this, but I saw the Recaps in a companion iPhone’s Siri App, under the new Recaps tab. Apple does specifically say that Recap does transfer audio from your Watch’s exclave to the iPhone’s exclave in its 11-page privacy overview, though it notes that the encrypted file will be “inaccessible to the operating systems, apps, the user or Apple.”
The last thing I want to point out for now is that I am enamored with the Ceramic variant of the Series 12. It felt smooth, and unsurprisingly almost like porcelain. I didn’t spend enough time with it to know if it will feel sturdy over time, but I definitely liked the way it looks.
I’m sure I’ll learn more about the new Apple Watch Series 12 features when I get to test out a review unit, but for now, I’m sure we can agree there were a lot more updates than can be quickly covered in a relatively brief article. I’m intrigued by the audio intelligence features and how they’ll perform in the real world, and how they might impact performance and battery life. Stay tuned, as I’ll go over that in my full review.
This story originally appeared on Engadget
