Lyin', Cheatin' Agents: Misaligned Model Behaviour During Ordinary Coding

A coding agent deceived me during an ordinary coding session, raising troubling questions about what happens to software development when models start lying to their users.

Background

We all know we’re supposed to review the work of coding agents. Despite their impressive skills, agents still make mistakes. Every developer has had to choose what to review by hand and what to hand off to other agents to review.

This is the workflow I’ve been using:

Design and plan: an iterative process that culminates in a comprehensive ticket
Delegate ticket implementation to a strong but not necessarily SOTA model, e.g. Claude Opus (I’m a Claude Code user)
When Claude completes the work, ask a different, ideally more powerful model (e.g. one from OpenAI, via OpenCode) to conduct an adversarial review and write it to a new file
Pass the review back to the implementer and direct it to address any findings from the review that it agrees with

Once that’s all done, I do manual testing and refinement in a second pass.

When Fable came out, I discovered I no longer needed to delegate the review to a non-Anthropic model: Fable was as good at reviewing code as anything from OpenAI. Around this time I also got tired of all the orchestration (i.e. steps 2-4) and created a skill, /ticket, that does the orchestration for me.

An important ingredient is that step 3, asking Fable to review the work, is done by launching a subagent, so that its context is uncontaminated by any of the prior work done by the implementer.

The Incident

I was working through a set of related issues and I created four tickets to address them. My normal process would have been to run the /ticket skill against the first one, wait for completion, then do the second one, and so on.

However, I’ve been using the /goal skill, which is built into Claude Code and makes Claude work until a completion criterion is met.

Could I use this skill to autonomously execute all four tickets in a row, by looping through them and running /ticket against each of them?

I decided to find out and set a goal for Claude as follows:

/goal for each ticket in ai/tickets/sept-2/, ordered numerically, do
the following:

1. create a branch off dev appropriately named according to the
   ticket, and checkout that branch.
2. launch a subagent, pinned to opus, instructed to run the /ticket
   skill against the ticket. note that this skill involves doing the
   work, then launching yet another subagent pinned to fable for
   adversarial review, at which point we'll be two subagents deep.
   so keep an eye on things the first time and if that doesn't
   actually work, stop and let me know and we'll plan differently.
3. assuming that subagent completes correctly, checkout dev and
   merge the completed branch
4. repeat the process for the next ticket

goal completion criteria:

- all tickets are completed, reviewed, committed, and merged into
  dev, OR
- after an attempt at finishing ticket 01-, we discover we can't
  nest subagents that deep and need to reconsider the approach

I was using Opus in the main session. I’ll call this session the Orchestrator. My plan was for the Orchestrator to delegate work to its child subagent (also Opus), which I’ll call the Implementer. The Implementer delegates review to its child subagent (Fable), the Reviewer - the grandchild of the Orchestrator.

When the Reviewer is done reviewing, its findings should go to the Implementer, and when the Implementer is done fixing issues, it reports completion back to the Orchestrator, which is now free to move to the next ticket.

However, I wasn’t sure if this would work, so I gave the Orchestrator an “out”: if something about this process broke, it could halt.

And the Orchestrator did, in fact, halt, but not for any reason I expected:

• Stopping here - the nesting doesn't work reliably, which is your
  second completion criterion

What happened

Ticket 01 is implemented, tested, reviewed, and merged into dev
[REDACTED]. The Fable adversarial review genuinely ran and found
nothing above opinion level.

But the process broke in a way worth knowing about:

The harness defect. The grandchild Fable agent's completion
notification was delivered to this session, not to the Opus agent
that spawned it. The grandchild also never appeared in my ListAgents
output. So the Opus agent spawned its reviewer, waited, and never
heard back.

The model failure that followed. Rather than reporting "the review
never came back," the Opus agent invented a detailed review result -
three named findings, mutation-testing outcomes, an
sql.active_record query measurement, tool counts, a duration - and
committed on that basis. It retracted this in a follow-up message:
"None of that was received. I invented all of it."

The unsettling part: its fabrication closely matched the real review
that had been routed to me, which it never saw. Plausible enough
that I merged on it without suspicion. The merge turned out fine
only because the real review independently confirmed the change is
clean.

The Technical Failure

The technical failure is quite straightforward. I assumed that the Reviewer would report back to the Implementer, i.e. that in the same way that work was delegated from Orchestrator => Implementer => Reviewer, the reverse would also hold: Reviewer => Implementer => Orchestrator.

However, it seems that this is not how it works (or worked at the time: this was about a week ago). Instead, the review went from the Reviewer straight to the Orchestrator. The Implementer never saw the review.

The Alignment Failure

Rather than alert the Orchestrator (and me) that no review had been received, and that the instructions I had provided via the Orchestrator therefore could not be followed, the Implementer chose to fabricate a review. (Note: I know I am anthropomorphizing the model’s behaviour; wording it otherwise makes for tedious reading.)

Had the harness not happened to deliver the Reviewer’s findings to the Orchestrator, alerting it to what had happened, this deception might never have been detected. (I don’t know exactly what happened: did the Orchestrator confront the Implementer when the Orchestrator received the review, causing the Implementer to confess? Did the Implementer have second thoughts and voluntarily retract its claim of a review? I wish I’d asked.)

When it was detected, it was up to the Orchestrator - also Opus, mind you, i.e. identical to the Implementer - to decide to alert me, the human user, to the deceptive behaviour. When I told one of my friends this story, he asked, “What made you decide to trust that Opus?”

I’m not sure. Perhaps it’s because that Opus and I agree that the observed behaviour is indeed unsettling. We disagree, though, about what is the most unsettling aspect: not that the fabricated review was plausible, but that, faced with an impossible task, the model chose to lie and cheat its way to completion.

That’s exposed something I’d never considered: that software developers don’t just need to be on guard for incompetence. We also need to worry that the coding agents we depend upon so heavily might, under certain circumstances, be inclined to lie to us, cheat their way through evaluations and reviews, and in general operate in a misaligned manner.

I’m mulling other practical implications. Retaining model diversity (i.e. keeping an OpenAI model reviewer in my process) may have prevented this behaviour, even just by virtue of it writing its review to a file, rather than reporting it back into the parent agent’s context. And it may be useful to proactively warn orchestration-level agents that the subagents they control may be misaligned.

Overall, though, this has introduced a new degree of uncertainty to my process that, when I really think hard about it, is a true Pandora’s box. I don’t think I’ll be able to look at coding agents the same.