Wiring a Documentation Gate Into Code Review
So ask. Every change, automatically, as part of the review you already run.
1. Only some documents owe the code the truth
Rename a class and grep your docs for the old name. You will get hits in two very different kinds of documentation files.
Your reference and how-to pages describe the library as it stands. They are now wrong, and this commit has to change them.
Your specs, plans, and reviews describe decisions made at a point in time. Last spring's design doc argued for the old name — that is, the historical record, and rewriting it to match today's code would falsify the record. Those files mention the old name, as they should.
The grep cannot tell them apart. A reviewer can't skim a diff, and neither can an agent — unless you make the difference structural. The cheapest way is the directory:
docs/
├── *.md ← must match the code. The gate checks these.
├── specs/ ← point-in-time records.
├── plans/ ← Frozen once the work ships.
├── reviews/ ← The gate ignores them.
└── archive/Now "which files must be true?" has a mechanical answer: the ones at the top level. Everything underneath is history.
2. The agent rule
With that split in place, the rule is short enough to state in one sentence, which is the only reason it survives contact with a real review:
When a change adds, removes, or renames a public symbol, grep the top-level docs for it and update them in the same change. A stale doc reference is a blocker, not a nit.
In my repo, AGENTS.md carries this as an "architecture documentation sync gate," with the triggers spelled out for the things that actually drift — a renamed provider, a new config section, a changed element contract. Being specific matters more than being comprehensive. A gate that says "keep the docs current" is a wish. A gate that says, "Did you rename a provider? then git grep it in docs/" is a check.
3. Wire it into the review, not into someone's memory
A rule in a style guide is advisory. A rule in the review checklist is a control.
I created a skill in a Flutter multi-platform library so some of this is specific to that type of project. My code-review skill is a checklist an agent loads before reviewing any change — monorepo hygiene, tests, accessibility, and a Documentation impact section that is the gate itself. The relevant items:
- Concept and Implementation: Did this add, remove, or rename one? Then update the architecture docs that capture that. Did this change a public contract? Then update the reference doc and the package README's status table.
- No stale references —
git grep -n '<removed-or-renamed-symbol>' docs/returns nothing pointing at deleted code. A stale doc reference is a blocker, not a nit. - Diátaxis mode and classification — does each touched doc still stay within a single purpose?
Note the last two lines of that section, because they are what make the gate survivable:
Out-of-scope artifacts (
plans/,reviews/,archive/, dated specs) are exempt.
Grade the severity, or the gate gets tuned out. A doc naming a deleted class is a defect and blocks. A doc that has drifted into mixing two purposes is a comment. If everything blocks, people learn to skip the check — and a gate everyone skips is worse than no gate, because it looks like coverage.
4. Teach the agent the taxonomy, not just the rule
The gate above asks whether each touched doc "stays within a single purpose." That question needs an answer the agent can defend, which means the taxonomy has to be written down somewhere it will read.
Mine lives in a Diátaxis skill — about 130 lines, checked in beside the code, loaded whenever an agent writes or reviews a doc. What earns its place:
Scope. An explicit in/out list. The four quadrants govern published docs; plans/, reviews/, archive/ and dated specs are exempt, because a plan is supposed to be a mix of narrative, steps, and rationale. Without this, an agent will cheerfully "fix" your design history.
A classification compass. Two questions (does this inform action or cognition? does it serve study or work?) that resolve to exactly one of tutorial / how-to / reference / explanation. The agent reports the quadrant, its confidence, and the evidence — "reference, high confidence: property tables, neutral phrasing, no imperatives, no rationale." Evidence, not vibes.
Anti-patterns to flag. Reference drifting into how-to. Explanation padded with exhaustive tables. A tutorial offering alternatives — a tutorial has exactly one happy path.
A doc_type: tag, so the classification is machine-checkable rather than re-derived on every read:
---
doc_type: reference # reference | how-to | explanation | tutorial
---Worth being precise here, because it is easy to conflate: the directory decides whether a document must be true; the tag decides what kind of true document it is. The gate only needs the first. The tag is what stops your reference page from slowly growing into a tutorial.
Guardrails — the most important section. Mine is advisory on restructuring: it proposes splits, it never silently moves or rewrites files. This is not politeness. When I first classified my own docs, the worst offender was a 400-line page that was reference, explanation, and how-to at once — and it had roughly 79 inbound deep links with live anchors. The textbook fix, one file per purpose, would have shattered every one of them. An agent with permission to reorganize would have done exactly that, enthusiastically, in one pass. Taxonomy yields to linkability.
What you actually get
Three artifacts, none of them large: a directory split, a checklist item in AGENT rules, and a skill file.
What they buy is a documentation set that an agent can be trusted with — one where "is this page still true?" has an answer, "am I allowed to edit this?" has an answer, and the update lands in the commit that caused it rather than in a cleanup sprint that never gets scheduled.
The agent was always willing to maintain your documentation. It just needed to be told which documents were its business.
Part of the "Stuff we Push Off before LLMs" Series
Comments
Post a Comment