What "On-Device AI" Actually Means — And Which Meeting Assistants Really Do It

“On-device” has become the most abused phrase in this category. Over the past eighteen months it has appeared on the marketing pages of tools that stream every second of your audio to a cloud server, and it is doing real damage to buyers who read it as a privacy guarantee.

The phrase is not usually a lie. It is usually true about one stage of a pipeline that has four, and the vendor lets you assume it covers all four. This guide is about learning to ask which stage.

The four stages

Every meeting assistant does roughly the same four things, in order.

Capture. Getting audio off the microphone or out of the meeting application.

Transcription. Turning that audio into text.

Analysis. Producing a summary, action items, decisions, and whatever else the tool sells.

Storage. Keeping the results so you can find them later.

A tool can perform any of these locally or remotely, and the combinations matter enormously. When a vendor says “on-device,” they are almost always talking about stage one.

Why capture-only is the common case

Local capture is genuinely useful, and it is the reason bot-free tools exist. Instead of sending a bot into your Zoom call as a visible participant, the app records from your machine’s own audio. Nothing joins the meeting. Nothing appears in the participant list. Nothing has to be admitted from a lobby, which matters more since Google Meet and Microsoft Teams both began restricting third-party bots earlier this year.

But local capture says nothing about where the audio goes next. In most of these tools, the answer is: straight to the vendor’s servers, for transcription and analysis, within seconds.

Granola captures locally on Mac and processes in the cloud. Jamie captures on your device and processes in Germany — genuinely good for EU residency, and genuinely not on-device. Circleback captures locally and analyses in its cloud. Wispr Flow, whose notetaker launched on 5 August, describes Flow as working “right on your device”; that describes capture and its dictation path, not the meeting analysis pipeline.

None of those vendors is behaving badly. All four are decent products with reasonable privacy postures. But if you bought any of them believing your meeting content never leaves your laptop, you bought something other than what you thought.

The test that settles it

Here is the whole diagnostic, and it takes about two minutes.

Turn off your internet and record a meeting.

If you get a transcript and a summary, the processing is genuinely local. If you get an error, a spinning indicator, or a note saying it will finish later, it is not. Everything else — the marketing copy, the architecture diagram, the privacy page — is downstream of that single result.

There is a softer version if you cannot easily test: ask the vendor, in writing, “does transcription run locally, and does summarisation run locally?” as two separate questions. Vendors who do it locally answer immediately and specifically. Vendors who do not tend to answer a different question, usually about encryption.

Encryption is not the same thing, and the conflation is deliberate often enough to be worth naming. Encryption protects data in transit and at rest on someone else’s computer. Local processing means there is no someone else. Those are different security models with different failure modes: encryption fails when a vendor is breached, subpoenaed, or acquired by someone with different values. Local processing does not have that failure mode, because there is nothing at the other end to breach.

Which tools actually pass

Of the 28 tools we rank, the honest list is short.

MacWhisper runs Whisper and Parakeet models entirely on your Mac. No account, no server, no network dependency. It is a transcription tool rather than a full meeting assistant — there is no calendar integration, no team features, no cross-meeting search — but what it does, it does entirely locally. Pricing depends on where you buy: the developer’s Pro licence is a one-time purchase of roughly €59–64, while the Mac App Store edition is sold separately on a subscription (about $29.99 a year). For a journalist transcribing a sensitive interview, it is close to unimprovable.

Hedy is the only tool in our rankings that can run the whole pipeline — transcription, summarisation, and its real-time layer — on-device on supported hardware. The qualifier matters and we keep repeating it: this depends on the hardware being capable enough, and older devices fall back to cloud processing. But on supported hardware it passes the airplane test with the full feature set intact, which nothing else in the category does.

Krisp processes its audio layer — noise cancellation and voice isolation — on-device, which is what earned it a strong privacy score. Its transcription and meeting notes are a different matter.

Plaud.ai and TwinMind both have meaningful on-device components without being fully local.

Everything else in our rankings processes in the cloud. That includes every tool from Otter and Fireflies through to the built-in assistants from Microsoft, Google, and Zoom.

When cloud is the right answer

This is not an argument that local processing is always better. It plainly is not.

Cloud processing gets you better models than most laptops can run, cross-device sync, team-wide search across everyone’s meetings, CRM integration, and analytics over hundreds of calls. If you are running a sales organisation and want to know which objections correlate with lost deals, that computation has to happen somewhere central. Gong cannot be an on-device product, and it would be a worse product if it tried.

The reasonable position is that local processing is the right default for sensitive conversations, and cloud is the right default for collaborative ones. A therapy session, a board discussion, a performance review, and a privileged legal conversation belong in the first category. A weekly sales standup belongs in the second.

Most organisations need both, and there is nothing wrong with running two tools.

What to check before you buy

Run the airplane test. Everything else is inference.

Ask about transcription and analysis separately. They are frequently split, with transcription local and analysis remote, or the reverse.

Ask what happens on unsupported hardware. Tools with an on-device mode usually have a cloud fallback. Find out when it triggers and whether you are told.

Ask whether local mode is the default or an option. A local mode nobody enables protects nobody.

Check the training default separately. Local processing and no-training are different guarantees, and after the August Otter.ai ruling the training question carries its own legal weight.

The phrase “on-device” is not going to become more precise on its own — it is too useful to vendors in its current vagueness. But the airplane test is not something marketing can argue with, and it costs you one meeting to run.