Thesis
The conversation about AI agents often assumes a binary: either the agent acts autonomously or a human does the work. In practice, autonomy is a spectrum, and the most effective systems are those that place each decision at the right point on that spectrum.
I've developed a framework for making these placement decisions based on two years of deploying agents in operations environments.
The Five Levels
I use a five-level autonomy model for every process I design:
Level 0 — Human Only. The agent has no role. The decision is too consequential, too novel, or too political for any AI involvement. Examples: firing decisions, major strategic pivots, crisis communications.
Level 1 — Agent Suggests. The agent analyzes data and presents recommendations, but a human makes every decision. Examples: budget reallocation suggestions, vendor selection shortlists, anomaly flagging.
Level 2 — Agent Drafts, Human Approves. The agent does the work and presents a completed output for human review before it goes live. Examples: report generation, email drafts, contract summaries.
Level 3 — Agent Acts, Human Monitors. The agent executes autonomously within defined bounds. Humans review in batch (daily/weekly) rather than approving each action. Examples: routine data entry, standard notifications, pipeline updates.
Level 4 — Full Autonomy. The agent handles the entire process end-to-end with no regular human oversight. Humans are only involved when the agent escalates (confidence < threshold). Examples: log monitoring, basic data validation, scheduling.
The Placement Framework
To decide where a process falls on the spectrum, I evaluate three dimensions:
Reversibility. How easily can a mistake be undone? Sending a Slack message is low-consequence and partially reversible (you can correct it). Processing a wire transfer is high-consequence and irreversible. The less reversible the action, the more human oversight required.
Variability. How much does the input vary? If every instance looks the same (standard invoice format, known vendors), high autonomy works. If every instance is unique (custom contract terms, unusual requests), the agent needs more human backup.
Organizational risk. What's the blast radius if it goes wrong? An internal process error is annoying. A customer-facing error is damaging. A compliance violation is catastrophic. Higher organizational risk → more human oversight.
Score each dimension on a 1-5 scale. The average determines the default autonomy level. Then adjust based on the agent's observed performance over time.
Progressive Autonomy
The most important principle: autonomy should increase over time based on demonstrated reliability, not based on assumptions about agent capability.
Every agent I deploy starts at Level 1 or 2 regardless of how confident I am in its design. It earns higher autonomy levels through measured performance:
Week 1–2: Level 1 (suggest only). Collect accuracy data. Week 3–4: Level 2 (draft and approve). Measure approval rate. Month 2–3: Level 3 (act and monitor). Track error rate. Month 4+: Level 4 if error rate stays below threshold (typically < 2% for non-critical processes, < 0.5% for critical ones).
This progression builds organizational trust. The team isn't being asked to trust an AI system — they're being asked to trust their own data about the system's performance.
The Patterns I See Most Often
After deploying this framework across six organizations, some patterns recur:
Teams underestimate what should stay at Level 0. Not every process should involve an agent. Some decisions are inherently human — political, emotional, or novel enough that agent involvement adds complexity without value. It's okay to have human-only processes.
The jump from Level 2 to Level 3 is the hardest. Going from "human reviews everything" to "human reviews in batch" requires a psychological shift. The team has to accept they'll occasionally miss something in batch review that they'd have caught in individual review. The math usually works out (batch review saves 10x the time), but it feels riskier.
Level 4 is rare and should be. I have maybe 15% of agent-managed processes at full autonomy. These are always the most standardized, lowest-risk, highest-volume tasks. If someone tells you their entire agent system runs at Level 4, they're either lying or not measuring errors.
Ideas to take with you
- Autonomy is a 5-level spectrum: Human Only → Suggest → Draft & Approve → Act & Monitor → Full Autonomy
- Placement is based on reversibility, variability, and organizational risk — scored 1-5 on each
- All agents should start at Level 1-2 and earn higher autonomy through measured performance
- The Level 2→3 jump (individual review to batch review) is the hardest organizational shift
- Only ~15% of processes should reach full autonomy — and that's fine
References
- SAE Levels of Driving Automation (adapted framework) — SAE International
- NIST AI Risk Management Framework — NIST
- Anthropic: Core Views on AI Safety — Anthropic
Thanks for reading.
Keep reading ↗