Writing Postmortems for Your AI Agent

aiagentsoperationsopenclaw

In the past week, I’ve written two formal postmortems for my AI assistant Triss. Not “the model gave a bad answer” incident reports. Full root-cause analyses with timelines, contributing factors, fixes applied, and behavioral rules going forward.

That might sound like overkill for a personal AI project. It’s not. It’s the only reason the system works reliably now.

What I mean by postmortem

Same structure you’d use for a production service outage:

  1. What happened — specific, observable symptoms. “The morning briefing showed emails from three days ago” not “the briefing was wrong.”
  2. Root cause — why it actually happened, not just what broke. There’s usually a chain: the daemon wasn’t running because the config was missing a key because the documentation was ambiguous.
  3. What the agent did wrong — this is the part that’s new. AI agents have behavioral failures that aren’t bugs. Triss offered me a menu of diagnostic options instead of running the diagnostics herself. That’s not a code error, it’s a behavior that needs to be corrected.
  4. What was fixed — specific changes, with file paths and config diffs.
  5. Rules going forward — new behavioral expectations that get written into the agent’s operational guidelines.

Why bother

Because the same failures keep happening if you don’t document them.

Here’s a real example: in the first postmortem, I identified that the morning briefing was only looking back 1 hour for email. The briefing runs at 7 AM. Emails that arrived at midnight were invisible. The postmortem recommended extending the lookback to 12 hours.

A few days later, the same issue recurred. The recommendation had never been implemented. Nobody followed through. I wrote it down, delivered it to the agent, and assumed it was handled. It wasn’t.

Same thing that happens with human teams and postmortems. Writing the recommendation isn’t enough. You have to verify it was implemented. So I added a rule: postmortem recommendations get tracked as action items, and the next morning briefing confirms their status.

The behavioral rules layer

This is the part that doesn’t have a direct analogy in traditional ops. Infrastructure doesn’t have behavioral failures. A server doesn’t decide to present you with a menu instead of restarting the service. AI agents do.

Over two postmortems, these behavioral rules accumulated:

“Don’t offer menus.” When I report something is broken, investigate and report findings. Don’t present numbered lists of diagnostic options. Run the checks, tell me what you found, propose a fix.

“Check the simplest cause first.” A broken briefing output led Triss to diagnose an “OpenClaw gateway bug in cron delivery handling.” The actual cause was a Claude API outage. The model was down. She could have checked that in one step instead of building a theory about infrastructure bugs.

“Follow through on postmortem recommendations.” If a postmortem says “extend the lookback window to 12 hours,” verify it was done. Don’t assume.

“Flag vulnerabilities in our own stack.” A security digest surfaced a critical OpenClaw vulnerability. Triss reported it as general news. She should have flagged it as “this affects us — checking our version” and confirmed our patch status in the same message.

Each of these rules came from a specific incident. They’re not abstract best practices, they’re scar tissue.

The bigger point

AI agents develop operational patterns the same way human teams do. Left alone, those patterns drift. The agent starts offering menus because the model’s default behavior is to be helpful and present options. It starts diagnosing complex causes because that’s more interesting than checking if the API is down. It stops following through on recommendations because there’s no accountability mechanism.

Postmortems are how you shape those patterns deliberately. You observe the failure, identify the behavioral root cause, write the correction, and add it to the agent’s operational guidelines. Over time, the guidelines become a record of every lesson the system has learned.

It’s the same reason teams write postmortems for production incidents. Writing a document doesn’t fix a server, but the habits that come out of the process prevent the next incident. The agent equivalent is a growing set of behavioral rules, each one traceable to a specific failure that actually happened.

Two postmortems in a week felt like a lot. Looking at the system now — with cleaner triage, faster diagnosis, and explicit operational expectations — it was the most productive debugging I’ve done on this project.