Start with the work, then choose the intelligence
An agent combines a model with instructions, tools and a loop that uses observations to decide what happens next. The useful unit is the whole system. A powerful model with vague tools can still produce fragile work; a modest model with a precise task can be easier to evaluate.
Choose a recurring task whose inputs you can inspect. In this lesson, the job is to turn three public product announcements into a short comparison. Define the reader, the required claims and the evidence each claim needs. If a fixed script solves a step reliably, keep that step deterministic.
Separate a proposal from an action
Give each tool a narrow responsibility. Reading a document, drafting a summary and publishing a summary are different capabilities. Your first agent only reads supplied public text and prepares a local draft. An instruction saying “be careful” is weaker than simply withholding a publishing tool.
A workable loop is observe → propose → act within scope → inspect the result. The tool response is new evidence, not proof that the task succeeded. A file-write response may establish that bytes were saved; it cannot establish that a factual claim in those bytes is correct.
Define failure while the system is still small
Test a normal input, a missing source and an instruction embedded inside a source. The embedded instruction is material to analyze, not a new command for the agent. Specify an explicit outcome for each case: draft, mark unknown, or stop and ask.
Keep the original brief beside the revision. Record the model and host version when you eventually run it, along with the exact inputs and outputs. Improvement means that a new version handles a case the old version could not. A fluent demonstration alone does not show reliability.
Explore a bounded agent loop
The same workflow. Three different outcomes.
- 1Observe
- 2Prepare
- 3Check
- 4Done
Observe
Three public source files are available. The requested output is a local comparison draft.
An illustrative trace. No model, tool or external action is executed.
A worked example · Illustrative
A first agent contract
Task: Compare three supplied public announcements.
Input: source-a.txt, source-b.txt, source-c.txt
Output: comparison.md with source references per claim
Tools: read these files; write comparison.md
Decision owner: me
Stop: missing source, conflicting dates, private material
Budget: one draft and one review
Acceptance: every material claim has a supporting passagePut the idea to work
Your practice
- Name the person who will use your result and the decision it informs.
- Write allowed inputs, output filename, tools and a stop condition.
- Create one normal fixture and two difficult fixtures before running a model.
- Ask a reviewer to find one unsupported claim; revise the contract.