The Agent Stack Is a Permission System, Not a Prompt
How agent harnesses, tool boundaries, and evals turn LLM demos into dependable systems.
Most agent demos begin with the model.
Reasoning ability, conversational fluency, and planning depth all matter.
Those questions matter. But when an agent enters a real workflow, the central issue becomes authority.
The operational question is the system’s allowed scope: what it can see, change, and approve.
A production agent is not just an LLM with a prompt and a few tools. It is a system made of a model, an agent harness, context, tools, policy, state, and evaluations.
The model generates a proposed next step. The harness is the runtime around it. It assembles context, manages tool calls, controls memory, handles retries, records traces, and creates approval gates. A prompt can describe a rule. The harness makes the rule observable and enforceable.
A practical system map
Intent
The intended outcome must be explicit. “Reconcile this month’s expenses” is an intent. It is not permission to change the ledger, contact a vendor, or move money.
Context
The agent needs the smallest useful set of information. That may include retrieved documents, database records, conversation history, memory, and tool schemas. More context is not automatically better. Irrelevant or sensitive context increases risk and can reduce accuracy.
Model
The LLM can classify a request, propose a plan, extract fields, select a tool, or draft an explanation. Its output is a proposal until the surrounding system validates it.
Tools
Each tool needs a contract: required inputs, allowed scope, expected output, failure modes, and side effects. Reading a document, drafting an email, updating a database, and initiating a payment should not share the same permission level.
Authority
Authority determines what may happen without approval. Useful controls include read and write separation, approval thresholds, scoped credentials, tenant boundaries, expiry, reversibility, and audit logs.
Harness and evals
The harness runs the loop:
observe state
generate or select a step
validate the step
call a tool
record the result
continue, hand off, or stop
Evals test whether that loop behaves correctly across normal tasks, edge cases, and failure conditions.
A concrete use case: invoice reconciliation
An invoice agent can read invoice records, purchase orders, and approved vendor data. The model can classify mismatches and explain them. The tools can draft a review queue, but cannot pay an invoice. A human approves any ledger change.
A useful test set includes:
- an invoice total that differs from the purchase order
- a duplicate invoice
- a missing approval record
- a vendor outside the current organization
- an ambiguous request that should produce a handoff
This is more valuable than a happy-path demo because it tests the boundary between assistance and authority.
What good evals measure
A final answer that sounds plausible is not enough. Measure:
- task success and final workflow state
- tool selection and argument correctness
- grounding in permitted data
- policy compliance and access boundaries
- recovery from timeouts and malformed results
- human handoff when authority or confidence is insufficient
- trace completeness, latency, and cost
Store expected outcomes, permitted tools, forbidden actions, and acceptable handoff behavior in a small versioned eval set. Run it when the model, prompt, retrieval logic, tool schema, or policy changes.
If a model upgrade improves task completion but increases unauthorized tool calls, the system did not improve overall.
A daily operator loop with Codex
Codex is most useful when it speeds up the loop without removing judgment:
1. Inspect the current repository, instructions, and runtime state.
2. Define one bounded task and a clear stop condition.
3. Let the model propose or implement the smallest useful change.
4. Run tests and inspect the diff.
5. Verify the result in the real browser or product surface.
6. Record what changed, what was verified, and what remains open.
The same pattern applies to browser automation, terminal tools, issue trackers, and AI assistants:
propose, approve, execute, verify, record.
That separation turns AI from a source of plausible output into a dependable productivity system.
The most common failures are system failures. A tool is too broad. Retrieval returns irrelevant information. A retry repeats a side effect without an idempotency key. Approval is vague. The audit trail is missing. Or the eval suite only tests the happy path.
The fix is not to remove autonomy. It is to shape it.
Make high-consequence actions visible and reversible. Separate read, draft, and execute tools. Use thresholds, approval checkpoints, scoped credentials, idempotency controls, and logs that a real operator can read.
The teams that win with agents will not be the ones that add an agent first. They will be the ones that make its boundaries clear enough to use every day.
Reply with one tag to steer the next field note:
[MAP] for a system diagram
[EVAL] for a test-case walkthrough
[CODEX] for a practical automation workflow
I will use the most repeated tag for the next edition
.