Voice Dictation for AI Coding Agents
Speak your instructions, review the transcription, then send. amux ships a Dictation tab built for phones: 16kbps mobile path, offline queuing, custom dictionary, phonetic session matching, and BYO Gemini key. Control your AI engineering team with your voice — from anywhere.
Why Voice Control for AI Agents?
Running a fleet of AI coding agents means sending a lot of messages. When you're debugging a production issue at midnight, pacing around your office, or reviewing your agents' progress from your phone during a commute, typing is slow and error-prone. Voice is the fastest way to direct agents in those moments — but only if the transcription is accurate and you review before sending.
amux v0.9.197 ships a built-in Dictation tab on every session's Peek panel. You speak, Gemini transcribes, and the cleaned text lands in the message composer. You review, edit if needed, then send. Your agents never see anything you didn't explicitly approve.
How Dictation Works (Step by Step)
-
Open a session's Peek panel
Click any active session in the amux dashboard. The Peek panel slides in with tabs: Messages, Files, Schedules, and Dictation.
-
Add your Gemini API key (one-time setup)
In the Dictation tab, tap Settings and enter your Gemini API key. Get one free from Google AI Studio in under 2 minutes. The key is stored server-side — it is never sent to the browser or logged.
-
Tap Dictate and speak your instruction
Tap the microphone button in the Dictation tab, or open the session's ⋯ menu and select Dictate from anywhere in the dashboard. A live recording popup appears. Speak your instruction clearly.
-
Gemini transcribes and cleans the text
When you stop recording, the audio uploads to
/api/dictate(server-side). Gemini transcribes it and applies any custom dictionary corrections you've defined. The cleaned text drops into the message composer. -
Review, edit if needed, and press Send
Read the transcription. Fix any errors — technical term mishearing is the most common. Once it looks right, press Send. The instruction goes to your agent.
The Mobile-First Path (v0.9.199)
Most voice control happens on mobile — you're walking, your hands are busy, or you're away from your desk. amux v0.9.199 adds an explicit low-bandwidth path optimized for phones:
| Property | Browser Default | amux Mobile Path | Reduction |
|---|---|---|---|
| Audio encoding | Stereo WebM (Opus or VP8) | 16kbps mono Opus | ~9× smaller |
| Transfer encoding | Base64 (33% overhead) | Raw binary | 33% less |
| Upload timeout | Browser default (~30s) | 120s | Works on slow LTE |
| Offline behavior | Upload fails, lost | IndexedDB queue, auto-flush | Zero lost recordings |
A typical 10-second coding instruction at 16kbps mono uses about 20KB of upload bandwidth — well under 1 second on any LTE connection. On weak signal, the extended 120-second timeout means uploads still complete where the browser's default would time out.
Dictation Features
History tab
All past dictations organized by date. Copy a transcription to the clipboard, re-insert it into the composer, trigger an AI Edit, or delete it. Searchable across the full history.
Dictionary tab
Teach Gemini your project's vocabulary. Add correction pairs: spoken form → intended text. For example, "Reack" → "React", "auth dot js" → "auth.js", or a function name that sounds like a common word.
BYO Gemini key
Your Gemini API key is stored server-side in amux's database. It powers transcription and AI Edit. It is never shipped to browser clients. The free Gemini tier handles hundreds of short dictations per day.
Phonetic session matching
Say "send this to session three" or "the auth agent". amux matches what you said to your actual session names phonetically — even if the names are slugs like auth-agent-3.
AI Edit
After transcription, tap AI Edit to refine the instruction with another Gemini pass — add precision, reformat as a checklist, or expand a shorthand note into a full request.
Offline recording queue
Dictate while in airplane mode, underground, or with a dropped connection. Recordings save to IndexedDB and auto-upload the moment connectivity returns. Nothing is lost.
Practical Use Cases
"Agent three: the deploy is live. Run the smoke tests against production and report back."
"You've been on the auth refactor for an hour. Commit what you have and start on the rate limiter next."
"All agents: pause and wait for instructions. Auth session: push your branch when ready."
"The tests are failing in the JWT module. Check the expiry logic in verify-token.js and show me the diff."
"What did each agent finish overnight? Give me a one-line summary per session."
Dictate a series of instructions before going underground. The queue uploads all at once when you surface.
Setup Guide
Dictation is built into the amux dashboard — no separate install, no CLI flags.
-
Install amux
pip install amuxor clone from GitHub. Runamux serveto start the dashboard at localhost:8822. -
Start a session
amux start my-project. The session appears in the dashboard. Click it to open the Peek panel. -
Get a free Gemini API key
Go to Google AI Studio (aistudio.google.com), click Get API key, copy it. The free tier is generous for dictation use.
-
Add the key in the Dictation tab Settings
In any session's Peek panel, open the Dictation tab. Tap Settings and paste your Gemini key. This is a one-time setup — the key persists across sessions.
-
Grant microphone permission in your browser
When you first tap Dictate, your browser will ask for microphone access. Grant it. On iOS Safari, you may also need to allow mic access in Settings › Safari.
-
Optionally: build your dictionary
Open the Dictionary sub-tab and add any terms Gemini consistently mishears. Start with your project's entity names, custom function names, and technical jargon that sounds like common words.
Technical Details
For developers who want to understand the implementation:
- Client recording: MediaRecorder API with a constraints object requesting
audioBitsPerSecond: 16000and mono channel. Produces Opus-encoded blobs. - Upload endpoint:
POST /api/dictate— accepts raw binary audio asapplication/octet-stream. Returns{"text": "cleaned transcription"}. - Transcription: Server passes the audio to the Gemini API with a system prompt that instructs it to transcribe accurately, apply the user's dictionary corrections, and return clean text ready for a coding agent.
- Key storage: Gemini API key stored in the
dictation_dict/ settings table in amux's SQLite database. Never returned to the client. - Offline queue: Recordings saved as IndexedDB entries with a
pendingflag. On reconnect, the client iterates the queue and POST-uploads each entry, deleting it on 200 OK. - Phonetic matching: Session names are converted to a phonetic representation (Double Metaphone or similar) and matched against the spoken name extracted by Gemini.
Frequently Asked Questions
Does amux auto-send what I dictate?
Never. The transcription always lands in the message composer for your review first. You can edit it, extend it, or discard it. Nothing reaches your agents until you press Send. This is a deliberate design choice — voice-to-text errors can corrupt instructions, so the review step is non-negotiable.
What AI model transcribes the audio?
Google Gemini, via your own API key. The key is stored server-side and never exposed to the browser. You can get a free key from Google AI Studio (aistudio.google.com) in about two minutes. The free Gemini tier handles hundreds of short dictations per day well within its limits.
Can I dictate while offline?
Yes — recording works offline. The audio clip saves to IndexedDB in your browser. When you reconnect, amux automatically uploads the queued recordings and transcribes them in order. You never lose a dictation to a dropped connection.
How much mobile data does a dictation use?
Very little. The amux mobile path encodes at 16kbps mono Opus — roughly 9× smaller than the browser's default stereo WebM. A 10-second instruction is about 20KB. It also uploads raw binary instead of base64, saving an additional 33%. On any LTE connection, a typical coding instruction uploads in under a second.
How do I handle technical terms and project-specific names?
Use the Dictionary sub-tab in the Dictation panel. Add correction pairs: the spoken form on the left, the intended text on the right. For example: "racks" → "React", or a component name that sounds like a common word. The dictionary is applied server-side before the text lands in your composer.
Does it work on iPhone/iOS Safari?
Yes. The dictation UI is a PWA that runs in iOS Safari. You'll need to grant microphone permission in Settings › Safari › Microphone the first time. The low-bandwidth mobile path (16kbps mono) is specifically tuned for iOS Safari's WebAudio behavior.
Can I route a dictated instruction to a specific agent session by name?
Yes, using phonetic session matching (v0.9.199+). Say "send this to the auth session" or "session three" and amux matches it to your actual session names phonetically. You don't need to pronounce exact slugs — natural spoken names are matched to auth-agent-3-style session IDs automatically.