AI code assistants have made documentation fast to produce and easy to ignore.
The issue is not speed; it's trust.
Models can write a polished docstring that describes a function that no longer exists, or an example that fails on the first run.
The more docs we generate, the more stale those docs become if nothing checks them.
This article describes a test-first documentation workflow.
It treats documentation like code: each claim passes an automated check before it is considered done.
You will find a small Python artifact that extracts docstring stubs, a validation script that catches broken examples, and a decision table that separates what a model may draft from what a human must own.
Why AI-Generated Docs Fail Silently Consider a typical flow: you ask a language model to document a function.
It returns a docstring with an example.
The example contains a parameter name that was renamed in the last commit.
The docstring looks plausible, so no one questions it.
The problem is the absence of a feedback loop.
Code has compilers, linters, and tests.
Documentation only has the cursor and the reader's patience.
The fix is to give documentation the same feedback loop.
Run the examples.
Check that documented names exist.
Compare the documented behavior with the actual behavior.
The Three Test Gates Gate 1: Presence Every public function should have a docstring.
This is simple to enforce with or a tiny AST script.
If a new function lands without a docstring, CI fails.
Gate 2: Accuracy Docstring examples must be executable.
Python's is the classic tool, but you can also build custom checks.
For example: Running turns those examples into tests.
If the function changes behavior, the docs fail loudly.
Gate 3: Freshness When a function signature changes, its docstring should be flagged.
You can write a CI script that compares the set of public names defined in the code with the set of names mentioned in the documentation.
Here is a minimal version: This is a heuristic, not a proof.
A docstring could mention a function name and still be wrong.
But it catches the most common drift.
Using a Free-Tier Model to Create the Draft Drafting docstrings is a good job for a language model because it is repetitive and low-risk.
The high-risk part is reviewing the result.
That's why the workflow separates drafting from ownership.
MonkeyCode is an open-source project that provides free model access (currently advertised as 10 million tokens, check the official README for the latest quota) and a free server option, which is useful for this kind of batch job.
Instead of paying per request, you can run a nightly script that scans new public functions and sends their signatures to the model for a draft.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
A typical prompt might be: The model output should be treated as draft.
It goes into the docstring, but the CI gates decide whether it survives.
What a Human Must Own Even with three test gates, some information cannot be verified by executing examples.
A human reviewer owns: Public API contracts: exact parameter semantics, ordering guarantees, exceptions raised.
Breaking changes: migration steps from old to new behavior.
Security and safety invariants: mention of auth, data handling, or resource limits.
Rationale: why a design decision exists, not just what the code does.
The table below summarizes what can be delegated to a model and what should stay with a human.
Documentation content Model can draft?
Human must review?
Repeating parameter descriptions Yes No, if example passes Usage examples for stable functions Yes Yes, check for outdated calls Return-value units and ranges No Yes Exception and edge-case behavior No Yes Migration and deprecation notes No Yes "Why" explanations No Yes The Full Pipeline On every pull request, a CI job extracts changed public functions.
A script generates empty docstring stubs for those functions.
A free-tier model (e.g., via MonkeyCode's free model access) fills in the stubs. runs every example in the docstrings.
A coverage checker asserts every public function appears in the docs.
A human reviews only the rows marked "Yes" in the table above.
This keeps the model's contribution useful but bounded.
It also gives reviewers a checklist instead of a blank page.
Limitations The custom freshness check is a heuristic.
It cannot understand meaning, so a wrong yet plausible explanation can pass.
Doctests fail on functions with non-deterministic output or heavy I/O.
You may need to exclude those examples from the test set.
The free tier on any service may have rate limits and uptime constraints.
Verify the current quotas on the official project before building a production pipeline.
For teams with strict compliance requirements, this workflow is not a substitute for expert review.
If a wrong docstring could cause harm, do not rely on automated checks alone.
Who Should Not Use This You should not use this workflow if: Your documentation must meet regulatory audit standards or document safety-critical systems.
Your team already has a maintainer who writes docs by hand and the docs are never stale.
Automating will add process overhead.
You cannot tolerate the small risk that a generated example passes doctest but misleads readers.
The Takeaway Documentation is code.
Give it tests, give it CI gates, and give a model only the parts it can fail safely.
The free tier of a tool like MonkeyCode makes the experimentation cheap, but the discipline comes from your pipeline, not from the model.