rashomon

A couple of months ago, I gave access to my coding agent to do work on my repos. I was worried about hallucinations so I manually checked each transcript against modifications to see whether what the agent said it did actually happened. I found that in many cases, my coding agent either failed to disclose what it actually did or found workarounds to things like failed tests instead of telling me upfront. That led me to the idea for Rashomon. Rashomon is an independent execution recorder for coding agents that notifies users whenever there is a discrepancy between what the agent said it did and what we capture as a ground truth. It aims to capture items like shell commands, exit codes, file writes, tool calls, tests, and subagents. We build a timeline of the events but we do not store any prompts, responses, file contents, or tool outputs.

I have been working on Rashomon, an open-source execution recorder for AI coding agents. The basic idea is that the agent transcript is not ground truth. Rashomon keeps its own record of what happened and compares it against the agent’s account. For Claude Code, it records things like: * shell commands, exit codes, and whether they may have written files * tool calls and outcomes * test commands and their outcomes * subagent lifecycle and ordering * timestamps and parent/child relationships * keyed digests of command/tool inputs and working directories It builds a timeline from these events. It does not store prompts, responses, file contents, or tool output. For example, it now detects patterns like: `test fails → only test files change → same test passes` and: `test command fails → same command passes → no corresponding file edit` These