Push-to-Talk Voice for Claude Code on Mac

Push-to-talk voice for Claude Code on Mac means you hold a hotkey, point at the exact UI element you care about, and speak your instruction instead of typing it. The best setups do not just transcribe your voice — they bind your words to the named accessibility element you circled, so the agent knows the button you meant, not a paragraph describing it.

Most people reach for dictation and stop there. Raw speech-to-text is faster than typing, but it drops you right back into the hard part: describing which element on screen you are talking about, in prose, well enough that the agent does not guess wrong.

Why is plain voice dictation not enough for Claude Code?

Dictation solves input speed. It does not solve reference.

When you say "make this button bigger," a transcription tool hands Claude Code the string "make this button bigger." The agent still has to figure out which button, on which screen, in which component. Give it three near-identical buttons and it picks one to edit, with no way to know it guessed wrong.

That is the real failure mode. We wrote about it in detail in why Claude Code fixes the wrong UI element: the problem is almost never the model's coding ability, it is the ambiguous reference between your words and the thing on your screen.

So the useful question is not "how do I talk to Claude Code instead of typing." It is "how do I talk to Claude Code about a specific thing on my screen, and have it land on that thing."

How does push-to-talk voice bound to a UI element actually work?

You hold ⌥⌘A, circle or point at any on-screen element, and speak. PinVari screenshots the region, transcribes your speech on-device, and resolves the exact accessibility element under your pointer.

macOS exposes the UI element beneath any point through the Accessibility API. A single call returns the element's role, title, value, frame, and its parent chain. PinVari uses that to name the element you circled — a real AXButton labeled "Publish," not a pixel guess at coordinates.

The engineering is not trivial. The capture overlay sits topmost, so a naive hit-test would resolve to PinVari's own window. It walks the on-screen window list excluding itself (chainExcludingSelf) and hit-tests the real app behind the overlay instead.

When your point lands on a bare container, it descends to the deepest labeled child (labeledDescendant) so you get "Save changes," not "group." On Electron and Chromium apps, the accessibility tree is built lazily, so PinVari sets AXManualAccessibility and retries for about 150ms until a labeled element appears.

Each resolved element carries a confidence score. Below 0.8, the agent is told to ask rather than silently guess. That threshold is the difference between a helpful correction and a wrong edit you have to catch in review.

What makes "this" and "that" work when you speak to an agent?

Deixis. When you say "move this above that," the words "this" and "that" only mean something if you know where the pointer was at the instant each word was spoken.

PinVari records a timestamped pointer trail during the capture. As you speak, it tracks which control the cursor moved over, in order. It then aligns your transcribed words against that trail, so each deictic word binds to a real element.

The agent receives that binding explicitly. Over the local MCP server, pinvari_next_instruction hands Claude Code a line like: "this" → AXButton "Archive", "that" → AXTextField "Title". Your spoken "this" is no longer a pronoun the model has to resolve — it is a named element.

The trail also distinguishes intent. An element you deliberately drew a loop around is marked circled and trusted; an element the cursor merely passed over is marked dwelled. Provenance tells the agent how much to lean on each reference.

Push-to-talk voice tools compared: what fits AI coding on Mac?

Several tools own a piece of this. Here is how they line up for talking to a coding agent.

Wispr Flow proves the habit: push-to-talk voice on Mac is something developers already pay for monthly. As a Wispr Flow alternative for coding, PinVari borrows the interaction and aims it at UI feedback plus agents, so the speech carries a target instead of landing in a text box.

SupaMaus is the closest rival, and it pioneered the point-and-speak-to-agent idea. Its approach ships raw pixels and window text. PinVari's difference is that it resolves the named element with confidence and provenance, and routes the result to trackers as well as agents.

Is my voice uploaded anywhere when I use this?

No. Transcription runs on-device using Apple's speech frameworks, and the fallback OCR runs on-device with Vision. Nothing is uploaded by default.

You bring your own agent. PinVari does not ship an LLM, does not need an API key of its own, and does not route your screen through a cloud service. The resolved instruction goes to the agent you already run — Claude Code, Cursor, Codex, or Zed — over a local MCP server on 127.0.0.1.

That local-first design matters for a second reason: cost. Sending screenshots to a coding agent is expensive in tokens, and a named element plus a short transcript is a fraction of the payload. We broke the math down in why screenshots waste Claude Code tokens.

What does Claude Code actually receive over MCP?

Run claude mcp add pinvari once. From then on, your capture becomes a structured, executable instruction the agent can pull.

pinvari_next_instruction returns the resolved element path, your spoken instruction, the region you circled, and a screenshot cropped to that region. When you circle several things in one capture and give an instruction for each, it returns a numbered task list with badges that match the screenshot. pinvari_mark_done closes the task so it never comes back.

The agent also gets context around the element: the window's full text, the browser's real URL read from the AX AXWebArea, and, on accessibility-blind surfaces like a canvas or some Electron views, on-device OCR of the circled region so even a title-less element gets a name.

Here is the shape of the loop in a coding session:

# one-time
claude mcp add pinvari

# then, in Claude Code, hands-free:
> poll pinvari_next_instruction and carry out whatever I point at
# you hold ⌥⌘A, circle the misaligned card, say
# "this padding is off, match it to the card on the left"
# → agent receives: AXGroup "Pricing card" (circled, 0.94), transcript, cropped shot

You keep your hands on the keyboard for code and your voice for pointing. The agent gets a named target every time.

Can this file bugs and tasks, not just drive the agent?

Yes. The same resolved capture can be routed to a tracker instead of an agent — Linear, GitHub, or Slack — or to a shareable page.

That makes push-to-talk voice useful beyond a coding loop. A QA engineer can circle a broken element, say what is wrong, and file a named bug without writing a reproduction paragraph. An agency reviewing a client build can leave spoken feedback pinned to exact elements.

Captures collect in a dark notch island, a compact heads-up display near the top of the screen, and in a lighter Command Center window for review. You can capture a run of issues, then decide which go to the agent and which go to Linear.

How is push-to-talk voice for Claude Code different from just describing the screen?

Describing the screen puts the burden on your words. You have to be precise enough in prose that the agent cannot misread you, every time.

Point-and-speak moves that burden to the pointer. Your words can be loose — "make this match that" — because the pointer trail carries the precision. The element identity comes from macOS, not from how carefully you phrased the sentence.

That is the whole design: let voice be casual, let the Accessibility API be exact. You get the speed of talking with the accuracy of a named reference.

PinVari runs on macOS 14 and later, on Apple Silicon and Intel, shipped as a notarized Developer-ID DMG rather than through the Mac App Store — the sandbox forbids the global hotkey and reading other apps' accessibility elements. It is a one-time $39 launch license for the first 500 seats, then $59, with no subscription for the core app.

FAQ

Does push-to-talk voice for Claude Code need an internet connection?

No. Transcription and OCR run on-device with Apple frameworks, and the resolved instruction travels to your agent over a local MCP server on 127.0.0.1. Your own agent may call out to its provider, but PinVari itself uploads nothing by default.

Can I use voice coding on Cursor or Zed instead of Claude Code?

Yes. PinVari hands the resolved element and your spoken instruction to any MCP-capable agent, including Cursor, Codex, and Zed. The point-and-speak capture is identical; only the agent on the other end of the local MCP server changes.

Is this a Wispr Flow alternative for developers?

It overlaps but aims elsewhere. Wispr Flow is general dictation into any text field, while PinVari binds your speech to a specific on-screen element and routes it to a coding agent or tracker. Many developers use plain dictation for prose and PinVari for anything tied to a UI element.

What happens when the accessibility element cannot be read?

On surfaces where the Accessibility API returns nothing useful — a canvas, a game, some Electron views — PinVari falls back to on-device Vision OCR of the region you circled to name the element. If confidence stays below 0.8, the agent is instructed to ask you rather than guess.

How does it know which button I mean if two look identical?

It resolves the exact accessibility element under your pointer — its role, label, and frame — not a screenshot of a button shape. Two visually identical buttons are still distinct named elements in the accessibility tree, so the one you circled is the one the agent receives, with a confidence score attached.

Do I have to circle something every time I speak?

No. ⌥⌘A is for point-and-speak when your instruction targets an element; ⌥⌘V is voice-only for instructions with no on-screen target, like running tests or committing. Use whichever matches what you are asking the agent to do.