Self-Healing AI Agent Configuration

How to configure amux so your Claude Code agents automatically recover from crashes, context exhaustion, and stuck tool-approval prompts — and run unattended overnight.

The premise of running AI coding agents while you sleep is simple. The execution is not. Claude Code sessions fail in ways that are subtle, frequent, and varied: context windows fill, thinking blocks corrupt, agents hang waiting for tool approval that never comes. Without a recovery layer, every overnight run is a gamble.

amux's self-healing watchdog is that recovery layer. It monitors every registered session, detects each failure mode by its specific signature, and applies the appropriate fix — automatically, without waking you up.

Why AI agents crash and get stuck

Before configuring recovery, it helps to understand what you're recovering from. Claude Code sessions fail in four distinct ways, each requiring a different response:

1. Context window exhaustion

Every Claude Code session has a finite context window. As the conversation grows — messages, tool outputs, file reads — the window fills. When it hits ~95%, Claude Code either errors out or produces degraded output. The fix is compaction: Claude Code's built-in /compact command summarizes the conversation and clears space. But compaction only works if something sends the command, and there's nobody at the keyboard at 3am.

2. Thinking-block corruption

Claude's extended thinking mode writes internal reasoning to a structured block before responding. Occasionally — more often with very long reasoning chains — this block is malformed. The Claude Code loop detects the malformed output and throws an error, terminating the session process. The agent had work to do. Now it's dead and the work is sitting in its queue.

3. Tool-approval blocking

By default, Claude Code pauses before executing certain tools — shell commands, file writes — and waits for the user to press Enter to approve. This is sensible for interactive development. For unattended overnight runs, it means the agent stops and sits idle indefinitely, blocking all downstream work. The fix is automatic approval — but only if the watchdog can reliably distinguish a tool-approval prompt from every other kind of Claude Code output.

4. Silent idle hangs

Sometimes an agent completes a subtask and then stops. Not an error — just done, waiting for another message. If the task was structured as "do this one thing," that's fine. But if there's more work on the board, the agent should pick it up. Without a nudge, it just sits there. The work sits there. And nothing happens until morning.

How the amux watchdog handles each failure mode

The watchdog runs inside the amux server process. It polls each registered session on a configurable interval, reads the tmux pane output, and applies pattern-matched recovery actions:

The key insight behind YOLO mode is specificity: the Esc to cancel string appears in Claude Code output only when waiting for tool approval, never in normal agent output or open-ended questions. That specificity makes automatic approval safe — the watchdog isn't blindly pressing Enter on everything, it's responding to a known, unambiguous state.

Configuration reference

Step 1 — Install amux

git clone https://github.com/mixpeek/amux
cd amux
./install.sh   # requires Python 3.8+ and tmux

Step 2 — Register a session with self-healing enabled

# Basic registration (self-healing on, tool-approval prompts will block)
amux register myproject --dir ~/Dev/myproject

# With YOLO mode (unattended-safe — auto-approves tool prompts)
amux register myproject --dir ~/Dev/myproject --yolo

# Start the session and the amux server
amux start myproject
amux serve  # dashboard at https://localhost:8822

The --yolo flag is per-session. You can run some sessions with it and others without, depending on how much you trust the project's CLAUDE.md constraints.

Step 3 — Configure watchdog thresholds (optional)

Thresholds are set in ~/.amux/server.env. This file is loaded at startup and survives auto-restarts:

# ~/.amux/server.env

# Context threshold for triggering /compact (default: 20)
WATCHDOG_CONTEXT_THRESHOLD=20

# Seconds of idle output before sending a continue signal (default: 120)
WATCHDOG_IDLE_TIMEOUT=120

# Watchdog polling interval in seconds (default: 10)
WATCHDOG_POLL_INTERVAL=10

After editing server.env, touch amux-server.py to trigger an auto-restart:

touch /path/to/amux/amux-server.py

Step 4 — Verify the watchdog is running

The amux dashboard shows watchdog status in the session detail view. You can also confirm via the API:

curl -sk https://localhost:8822/api/sessions | python3 -c "
import json, sys
sessions = json.load(sys.stdin)
for s in sessions:
    print(s['name'], '|', 'yolo:', s.get('yolo', False), '|', 'status:', s.get('status'))
"

Recovery patterns in practice

Pattern 1 — Simple overnight run

Queue tasks on the board before bed. Register one session per major workstream. Enable YOLO. Start everything. The watchdog handles all four failure modes. In the morning, open the dashboard from your phone:

# Before bed
amux register feature-auth --dir ~/Dev/app --yolo
amux register feature-payments --dir ~/Dev/app --yolo
amux register fix-bugs --dir ~/Dev/app --yolo

# Queue tasks via dashboard or API
curl -sk -X POST -H 'Content-Type: application/json' \
  -d '{"title":"Implement OAuth flow","session":"feature-auth","status":"doing"}' \
  https://localhost:8822/api/board

# Start all sessions
amux start feature-auth
amux start feature-payments
amux start fix-bugs
amux serve

Pattern 2 — Monitor agent watching a fleet

For larger fleets, register a dedicated monitor session whose sole job is to check on other agents. The monitor can do things the watchdog alone can't — like reading an agent's actual output and deciding whether to send a corrective message or re-queue a task that looks stuck:

# Monitor CLAUDE.md prompt
You are a fleet monitor. Every 5 minutes:
1. Peek at all sessions via the amux API
2. If any session's output ends with an error, re-queue its current board task
3. If any session has been idle for 10+ minutes with tasks in doing, send it a "continue" message
4. Log observations to the amux board with session=monitor

Never stop. Never wait for input. Repeat.

Pattern 3 — Self-healing with git safety

A crashed or replayed session might re-attempt work that was partially completed — including git commits. Protect against duplicate commits by having each agent check for its task ID in the recent commit log before starting work:

# In each agent's CLAUDE.md
Before starting any board task:
1. Read the task ID from the board
2. Run: git log --oneline -20 | grep "TASK-{id}"
3. If found, mark the task done and move to the next one
4. If not found, proceed with the work

Diagnosing self-healing failures

Self-healing is not foolproof. Knowing when it fails is as important as configuring it:

Running amux serve as a persistent service

The watchdog only runs while amux serve is running. For true unattended operation, run it as a persistent background service. On macOS with launchd:

# ~/Library/LaunchAgents/io.amux.server.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>       <string>io.amux.server</string>
  <key>ProgramArguments</key>
  <array>
    <string>/usr/local/bin/amux</string>
    <string>serve</string>
  </array>
  <key>RunAtLoad</key>   <true/>
  <key>KeepAlive</key>  <true/>
  <key>StandardOutPath</key> <string>/tmp/amux.log</string>
  <key>StandardErrorPath</key> <string>/tmp/amux.log</string>
</dict>
</plist>

launchctl load ~/Library/LaunchAgents/io.amux.server.plist

On Linux with systemd:

[Unit]
Description=amux agent orchestration server
After=network.target

[Service]
ExecStart=/usr/local/bin/amux serve
Restart=always
RestartSec=5
User=%i
EnvironmentFile=/home/%i/.amux/server.env

[Install]
WantedBy=multi-user.target

What self-healing does NOT handle

Being precise about the limits helps you design around them:

• Logic errors — if an agent makes a wrong architectural decision, the watchdog won't notice. It only detects process-level failures and specific output patterns, not semantic correctness.
• Rate limits — if your Claude API key hits a rate limit, sessions will error. The watchdog will restart them, but they'll hit the same limit. Use backoff-aware task design.
• Infinite loops in agent logic — if an agent is making steady progress (producing output, not erroring) but doing the wrong thing, the watchdog sees it as healthy. Use the monitor agent pattern to add semantic monitoring on top of process monitoring.
• Git merge conflicts — if two agents edit the same file and both push, the second will fail. This is a coordination problem, not a crash recovery problem. Use the worktree isolation pattern to prevent it.

Self-healing + the amux board

The watchdog and the board are designed to work together. When a session crashes and restarts, it can re-read the board to find its task — which is still marked doing — and resume. When the session completes the task, it marks it done. When a monitor agent detects a zombie session, it can move its task back to todo for another session to pick up.

The board is the single source of truth for what work exists and who owns it. The watchdog is what keeps the workers alive. Together, they give you the same guarantees a managed job queue provides: tasks run to completion, failures are retried, and nothing falls through the cracks.

See also: Kanban board for AI agents · Running 10+ agents in parallel · AI coding while you sleep

git clone https://github.com/mixpeek/amux && cd amux && ./install.sh
amux register myproject --dir ~/Dev/myproject --yolo
amux start myproject
amux serve  # → https://localhost:8822