The Discipline Behind Good Agents

It's all about evaluations. And in this article I'll explain why.

We've all seen many cool agent demos out there. To me though, a good agent demo just proves that an LLM and a harness together can produce an impressive result. On the contrary, a good agentic system must accumulate enough evidence against explicit acceptance criteria to justify shipping the agent.

And I've got the feeling this is the gap where agent projects can fail. The first few outputs during agent development can look promising, so the team goes and improves the prompt, gives the agent tools, and expands its responsibilities. Capabilities grow quickly, but team confidence ... is it really there? Traditional software gave us a convenient mental model for confidence: given the same code and input, a function usually returns the same result, which allows us to describe expected behavior, encode it in a test, and rerun that test whenever the system changes via a CI/CD pipeline. The test says little about whether we chose the right product problem, of course, although it can tell us whether a known functionality still works.

Agents complicate this model because their behavior depends on a larger and less predictable surface including the model, prompt, context, available tools, and previous decisions with sprinkles of chaos. Change one element and the agent may improve on the example in front of us while quietly regressing somewhere else.

This is why I think evals are like the CI/CD pipeline of agent development. Not in a strict sense, but in that they turn uncertainty into something a team can observe and continuously improve. The analogy is useful if we build the same discipline for agents around repeatable checks with explicit gates but it will become misleading if we expect it to remove uncertainty completely.

In any case, the terminology is unimportant here. The real deal is that quality can't be played down to a final glance at an output. It has to become an inherent part of the system that produces the output.

A convincing answer can still be wrong

An agent can produce something plausible while violating almost every rule we care about. They can be confidently wrong.

I use agents for drafting my articles. Then, more often than not, I find a draft reads well but it is inventing a source (I can reject citations outside the source set, though whether a source supports the claim still requires judgment; more on that later), flattening my viewpoint embarrassingly, or falling back on the same rhetorical patterns I have already rejected time after time. I also use agents to create videos, and I can see the same pattern: while a generated video may compile, there are many failures I need to recover from, such as narrations that outrun the animation, unreadable pieces of text, and visual arguments dissolving into a collection of presentation slides rather than my desired cinematic motions.

These failures are harder to see than a runtime exception or a broken build because I still get an artifact, and it looks finished! This plausibility hides defects, but not to trained eyes!

Taken to agent development at scale, when a team has no eval pipeline, it is trading quality gates with anecdotal inspection. And that's probably because we still don't know what quality gates look like in agent development, so when we look at a failure in retrospective, we shake our heads and wondered how we could have been so stupid! I have seen it (and done it myself): someone blindly tries a handful of prompts or Matt Pocock skills, skims through the results, and decides whether the new version just feels better. That approach is good for exploration (which benefits from speed and loose rules) but breaks down as we iterate our agent. Like in traditional software, every improvement to the agent expands the surface that could regress, but we get blinded by plausibility.

The economic problem is equally important. Agents increase build throughput, which moves pressure (not the bottleneck) toward review. If generating ten artifacts becomes cheap while establishing confidence in each one remains expensive, the organization has accelerated the wrong part of the system because more output may delay confidence rather than accelerate delivery.

In any case, Agent Evaluations (or evals) address that constraint by making the criteria, evidence, and findings of these inspections explicit and comparable, although assessments that depend on LLM judgment are still probabilistic by nature. We can increase this probability by evaluating the judge itself, but more on that later. At the end, an eval is simply a test of an agent’s output or behavior against an expected standard, but the useful unit is the pipeline around it:

task → agent run → candidate result → checks and judgment → repair or reject → release → observe real outcomes → add discovered failures to the next evaluation cycle

The concept of pipeline is key because one passing score tells us very little. An eval pipeline earns its value by accumulating evidence through repeated agent tests, capturing the results, blocking failures, and letting us learn from the ones we didn't catch.

CI/CD for a different kind of release

CI/CD works because software teams decided that releasing source code should produce actionable evidence. A successful build that respected all contracts, a green test suite, a package bundle with the right checksum, and a completed deployment each contribute a different piece of that evidence. Although no individual check proves that the software is valuable, their combined result lets a development team make a release decision without manually reconstructing the entire system every time.

Agent development needs the same discipline, but around outcomes, not outputs. I mean, around behavior.

I find it useful to think of an agent run as producing two artifacts:

1. The result we intend to use.
2. The evidence that gives us the confidence to use it.

In my case, for my agent-generated articles, the first artifact is the draft. The evidence may include proof that the required sources were consulted, links are valid, the requested ideas are present, and prohibited patterns are absent, so later I can review if the argument is faithful to my style (like I am doing now ☺️). For my videos, the first artifact is the rendered composition and its evidence includes valid scene metadata, non-overlapping narration segments, representative frames, and a visual review against explicit instructions.

This evidence changes our perception of delegation. In a manual workflow, confidence comes from inspecting almost every decision made along the way, an approach that quickly consumes the productivity an agent was supposed to create. Automated verification through evals gives us another interesting route: define intent, establish boundaries, delegate execution, verify the result, and then steer the next iteration. I have described this broader loop as Explain, Challenge, Delegate, Verify, Steer. Evals occupy the verification-and-steering half, allowing the agent to act more autonomously while preserving an accountable release decision. Human attention can then concentrate on ambiguity and taste instead of rechecking everything.

The CI/CD analogy has an important implication worth mentioning. A perfect pipeline can ship the wrong feature with extraordinary efficiency, just as an eval suite can show that an agent followed instructions that expressed the wrong intent. Strong evals expose regressions, but they cannot compensate for confused goals. Rigor therefore moves upstream, as we spend less effort manually producing every intermediate step and more effort expressing intent and deciding what level of delegation the task requires.

Let mechanically verifiable rules descend into code

As I argue in Agent-ready Software vs. Software-ready Agents, the most useful design decision in an eval pipeline is deciding which requirements are stable and mechanically verifiable.

For example, in my article generation agent, some requirements have objectively verifiable answers:

• Does the expected file exist?
• Is the article's JSON metadata valid against the schema?
• Did the build and typecheck succeed?
• Do two timeline ranges overlap in my scrolly-telling?
• Does the recorded word count match the text?
• Was a protected source modified?
• Did the agent use a prohibited tool?
• Does every citation resolve to a source that was actually consulted?

Because these checks are reproducible and inexpensive (they run on CPU), they have become code once I was confident about the rule. Repeatedly asking a model to infer whether JSON is valid wastes tokens and creates another opportunity for interpretation when a parser already knows the answer.

This connects with a broader principle I have written about as owning the deterministic core: encode the stable parts of a decision that software can enforce reliably, leaving the agent to handle the parts that require synthesis, adaptation, or contextual judgment.

On the other hand, there are other qualities that require judgement:

• Is the argument coherent?
• Does the draft preserve my voice?
• Are its claims properly supported by the supplied material?
• Does the visual hierarchy direct attention to the right place?
• Does a scrolly-telling animation communicate cause and consequence?
• Is the artifact useful to the intended reader?

For generated articles, a deterministic check can confirm that a heading exists, but it can't tell whether the heading belongs in the argument. For my videos, code can measure the distance between two objects on screen, yet geometry alone cannot decide whether the composition has visual appeal or whether the viewer understands what is going on. Those qualities require judgment, which may come from more expensive sources like a person, an LLM (they run on GPU), a vision-language model, or some combination of them. The right choice depends on the criterion an the appetite for risk.

The constraint-descent spectrum in Agent-ready Software vs. Software-ready Agents needs this qualification. Part of a prose instruction may become a schema or executable test, while another part becomes a rubric that still requires judgment. That rubric can remain useful long after the team understands the principle. A pipeline can require an agent or human to apply it and block delivery when the review fails, although enforcing the review does not make its conclusion deterministic.

Encoding provisional preferences too early can freeze the wrong decisions into the platform, while leaving mechanically verifiable requirements in prose creates avoidable interpretation and review. Established principles that need context deserve a different response: better guidance, examples, and evaluation. As failures reveal gaps, we should strengthen whichever mechanism can address them.

Before release and after reality

A release pipeline needs evidence from two environments.

• Design-time: This is the controlled side of the discipline, where an agent development team can test a change against known configurations (e.g., changing the model or the system prompt), inspect differences, and reject a candidate without making users participate in the experiment. These are known as offline evals, and they run before an output is accepted or a new agent version reaches end users. They include regression suites executed against representative rubrics, which can be deterministic or non-deterministic:
  • Deterministic rubrics are for requirements that have objectively verifiable answers. For example, in my article generation agent, this includes things like checking word count against a goal, validating the article's metadata against the schema, or verifying that there was no unintended Web search or API call during generation.
  • Non-deterministic rubrics evaluate qualities that require judgment (both LLM and/or human) rather than a binary check. The challenge here is that these rubrics need reference datasets coming from real (or synthetic) tasks, edge cases, or sampled judgment. This is an example from my article-generation agent showing a rubric that uses LLM-as-a-judge (via a cheaper/faster LLM) to check whether an article generated from synthetic intent data follows my writing style:

{ 
   "rubric_name": "author-style",
   "description: "Checks whether the generated article follows Pablo's writing style",
   "dataset": "/data/intent.md",
   "article" : "/output/generated/draft.md",
   "output" : {
      "good": "The article features a thesis-first, systems-based approach that connects technical mechanisms to broader implications with clear contrasts, pragmatic authority, curiosity, and a personal voice",
      "bad": "The article features a neutral, meandering, impersonal tone that catalogs facts without a clear thesis, systems lens, meaningful contrasts, or authorial point of view."
   }
}

• Runtime: These are known as online evals, and they can also be implemented as deterministic or non-deterministic checks (which in runtime are usually known as guardrails). These evaluations observe what happens under real use through tool failures, rejected outputs, user corrections, approval rates, and downstream evidence such as how much repair an output needs before it can be presented to the user. This is important because real users out there combine inputs in ways the original agent developers did not anticipate, so runtime agent runs always reveal the edges of the specification.

  If agent development teams think a guardrail failure is useful, it should travel backward through the pipeline towards offline evals, and sometimes all the way into the code as I argued earlier. And that's why agent observability is so important. Suppose my article-generation agent cites a source that does not support its claim. That runtime failure should become an offline rubric built from the same prompt, source, and output I should be able to grab from execution logs.

  When an online eval fails, the pipeline can ask the agent to repair the output and evaluate it again. Each retry adds latency, tokens, tool calls, and another opportunity for drift. We should therefore track first-pass success and repair attempts, limit how many retries are allowed, and turn repeated user repairs into new offline tests.

This is where the CI/CD analogy becomes more concrete. The citation failure above can be captured from the logs, reproduced offline, and then added as a release offline eval. Any candidate the eval flags for repeating the failure is automatically blocked, while a stronger rubric may improve detection without catching every recurrence. If the failure still escapes at runtime, the agent gets a limited chance to repair it before handing it to the user (in this case, me). I believe users should not have to discover the same failure twice.

The result is a pipeline built from four complementary kinds of evidence: deterministic and judgment-based evaluation, each operating offline and online. I can't resist treating this as a grand taxonomy but ... it is a practical reminder that checks differ both in how they reach a conclusion (deterministic vs. non-deterministic) and when they encounter the artifact (design-time vs. runtime). In any case, the pipeline needs to be balanced across these dimensions: relying entirely on deterministic checks produces structurally valid nonsense, while judgment alone makes review expensive and inconsistent. Also, an offline-only pipeline misses reality, whereas an online-only pipeline makes users become regression testers.

My article eval pipeline

The article workflow I use is already leaning on an eval pipeline (online only at this point), even though I did not begin by giving it that name.

1. It starts with selected source material, often fragmented notes and thoughts living in a Notion workspace. Before drafting, a retrieval step searches my existing post corpus from this personal website for related arguments and examples, producing an evidence that distinguishes direct corpus support from pure inference. It also records a no-match result when the corpus offers little signal.
2. The draft then passes through a voice and argument specification. A separate writing-quality rubric inspects it for patterns such as unsupported claims, loss of nuance, weak progression, repetitive cadence, canned AI vocabulary, and prohibited rhetorical formulas. As I explained earlier, any findings trigger targeted repairs before the automated stages hand the article to me for review.

You can tell several properties in this workflow are naturally deterministic. For example, the pipeline can record which issues were consulted and preserve Markdown during revision, thus preventing publication before my explicit approval. Additionally, very few and simple text checks can also detect some known filler phrases and forbidden syntactical patterns deterministically (e.g., replacing em-dashes with parenthesis using regex).

Conversely, other deeper qualities of this workflow require judgment (LLM and human). For example, there is an online LLM evaluator that must decide whether the retrieved material genuinely supports the inferred point of view and whether the article develops a differentiated angle rather than remixing a previous post. It also has to judge whether the thesis organizes the piece and the draft still sounds like me, including the curiosity and occasional human rough edge that an LLM-based polishing pass tends to erase. This needs explicit rubrics because just “hey, make it good” gives an evaluator almost nothing to work with. I've learned that this LLM-as-a-judge is more useful when we feed it with the original intent of my article as part of the eval dataset because it can find evidence in the draft and recommend a targetted repair, even when the judgment remains subjective.

However, this approach also reveals two important weaknesses.

• First, the independence of an LLM evaluator is limited when the same model interprets the original intent data, drafts the article, evaluates it, and repairs its own findings. Separate skills create different roles, which helps, although a role playing approach inside one model does not create an independent observer. The generator still carries its own assumptions into the judgment. We now know this happens because LLM evaluators tend to bias their own generations.
• Second, the pipeline checks fidelity to supplied Notion sources more strongly than factual, universal truth. It can ask whether a claim appears in the Notion note or retrieved issue, but establishing whether that source is correct requires external fact-checking and remains a separate responsibility.

My final review is therefore more than a liturgy of approval. The edits and objections I make on the generated draft are signals I capture, so repeated changes (rewriting openings, rejecting a certain cadence, etc., ...) can be captured in new rubrics that influence the next evaluation cycle. Over time, I predict some of that feedback will descend into deterministic checks (e.g., via powerful regex) ... yet some will remain mine because they express my personal taste and style.

That division is exactly what I want. The pipeline should shift human attention and preserve it for the decisions where human responsibility adds the most value.

My video eval pipeline

The video workflow makes the same idea easier to see because its output combines code (i.e., these are Remotion videos), data (i.e., video metadata), still images, motion rules, and narration. It also leans on an eval pipeline (only only as well):

1. The process begins with a finished (and typically approved) article and extracts a visual argument. In other words, the transformation or causal relationship the video should make visible. It then develops a visual world, a storyboard, and a cinematic scene description before an anti-infographic gate asks whether the concept reads like a film or like a glorified slide deck.
2. Once that direction passes, the system implements a Remotion project and evaluates several layers in parallel.

On the one hand, the deterministic layer checks whether the generated Remotion project is complete, internally consistent, and safe to render. Those are very important constraints because video generation creates a surprising number of ways to produce something that renders while violating the contract. A successful render only proves that frames were generated. On the other hand, the judgment layer asks harder questions: did the system find a visual argument worth showing, preserve it across the video, and make its visual language and motion carry the article's meaning? Together, both layers separate a renderable artifact from a storytelling video I would actually publish.

Today, some checks live in code, others in model instructions, and others in human review. In ay case, each failure tells us which layer needs to become stronger.

Evaluating the Evaluator

LLM judges are attractive because they make open-ended, non-deterministic evaluation more repeatable, and arguably, cheaper. They apply a rubric, identify evidence, and return structured findings at a scale that manual review cannot match.

However, we must not forget that their conclusions remain probabilistic, which means they may misunderstand the rubric or agree with the generator because both share similar model priors. Attaching a numerical score does not make that judgment objective. Instead, we should test whether explicit criteria and a separate evaluator (a different model from the generator) improve agreement with expert annotations, since a different model may still share the generator's biases or be less capable at the task. An excellent QCon 2026 talk, “Beyond Benchmarks: How Evaluations Ensure Safety at Scale in LLM Applications” explains this through a useful two-tier structure.

• The main agent is evaluated on a dataset representing its task, paired with a rubric and expected behavior
• The judge is evaluated on a different dataset of example conversations annotated by human experts and paired with its own rubric and expected judgments.

That second tier is interesting because evaluating only the agent quietly assumes that the judge is reliable. Once the judge participate in release decisions, its consistency become part of the product. How is that second-tier dataset produced? It's basically human-annotated example conversations with the main agent, which establish the reference point against which the LLM judge can be calibrated. This introduces a hierarchy in which the LLM judge extends the reach of human experts.

Evaluation quality is therefore recursive: we test the agent, test the judge, inspect disagreements, and refine rubrics, with human review at the center of the loop.

Trust is an engineering output

Technical leaders often ask how much autonomy an agent should have. I suspect all depends on what evidence the system produces during autonomous work and wha do we do with that evidence.

A low-risk, reversible task with strong deterministic checks can move quickly, while a high-impact action with subjective success criteria needs a tighter gate and more human judgment. As Grady Booch said:

All architecture is design, but not all design is architecture. Architecture represents the significant design decisions that shape a system, where significance is measured by the cost of change.

Therefore, the architecture of the eval pipeline should follow the cost of being wrong.

This is why evals belong in product and engineering strategy rather than in a final quality-assurance phase. They influence everything: how teams define tasks, store evidence, version agent behavior, and release changes. Yes, our confidence has to be engineered.

As agent capabilities improve, engineering work shifts toward specifying executable constraints and calibrating trus. Human judgment, meanwhile, concentrates where ambiguity and consequence meet. My article and video pipelines are small, evolving examples whose value comes from making intent visible. Deterministic checks validates verifiable rules, models inspect meaning through explicit rubrics, and I remain accountable for intent, taste, and the final publish decision.

As I said earlier, an agent should generate an artifact together with the evidence needed to trust it, and evals should turn that evidence into a repeatable decision.

Then the agent can remain probabilistic while we remain confident.