Epic: make the codebase agent-legible — fix misleading self-description, ambiguous contracts, and legacy leftovers

"This issue was filed by an AI agent on a human's behalf. The human submitter may not have independently verified the report."

Goal

Coding agents (and new human contributors) navigate this codebase by reading it locally: file names, doc comments, option structs, docs, and error messages are their map. Today several of those signposts are inaccurate, ambiguous, or point at code that no longer exists, so a reader who trusts them lands on wrong conclusions and produces plausible-looking regressions.

This epic tracks making the codebase self-describing and truthful: fix documentation and comments that lie, make implicit invariants explicit, remove legacy leftovers, and add guardrails so the map stays accurate.

The findings below come from a systematic legibility review (5 parallel deep dives: lifecycle backend, CLI layer, pkg/api contract, build/watch, cross-cutting), with each claim spot-verified against the current code. Every item lists file references. Items marked are arguably user-facing bugs discovered along the way and may deserve extraction into standalone issues.

A. The code misdescribes its own structure — split into focused PRs (supersede #14075)

• pkg/compose/convergence.go is a false friend: the convergence type was removed (fbea647b9), but the file keeps its name and its doc comment ("convergence manages service's container lifecycle") now sits on top of the unrelated getScale() (convergence.go:47-52). The file is a grab-bag (naming, dependency waiting, container creation, start helpers). Rename/split it. → PR #14129 (merged)
• 8 comments reference deleted code or stale line numbers: reconcile.go:195, :245, :349, :378, :719 ("matching the previous ensureNetwork/ensureVolume/convergence.go behavior"), reconcile.go:1012 ("same way as convergence.go:138-160" — those lines now hold unrelated code), observed_state.go:154-155, :369. Replace historical references with actual behavioral statements. → PR #14130 (merged)
• docs/sdk.md documents a progress package that does not exist (progress.NewTTYWriter/NewPlainWriter/NewJSONWriter/NewQuietWriter, sdk.md:148-155; WithEventProcessor(progress.EventProcessor) at :110 — actual type is api.EventProcessor). The real renderers live in cmd/display, which an SDK consumer cannot reasonably import. → PR #14131 (merged)
• AGENTS.md test instructions are misleading: "Test unit: go test ./pkg/..." actually starts the e2e suite (no file in pkg/e2e carries an e2e build tag; the documented -tags e2e is a no-op; CI excludes e2e by grep in Dockerfile:112). Document the real local-unit command and gate e2e behind a build tag or testing.Short. → PR #14132 (merged)
• 5 error messages say "set DOCKER_BUILDKIT=1 to use BuildKit" (e.g. build_classic.go:133-147) but the internal BuildKit builder was removed (af579ebd4); the real remedy is installing buildx. Only two build paths remain (bake via buildx subprocess vs. classic daemon API) — update messages and any remaining "three builders" wording. → PR #14133 (merged)
• AGENTS.md has no architecture map. A ~30-line section would prevent most wrong turns: the two lifecycle engines and who calls which (see C below), the Run(ctx, …, "op") wrapper convention distinguishing exported operations from internal helpers, the stdout/stderr conventions, and Docker Desktop integration points (internal/desktop, pkg/compose/up.go:88-92, publish.go:100, cmd/formatter/shortcut.go:249). → PR #14134 (open)

B. pkg/api promises things the implementation does not honor

• Services has three different semantics across option structs: real project filter (ProjectLoadOptions, read at pkg/compose/loader.go:125,146), recreate-policy selector that does not filter (CreateOptions — Create with Services: []string{"web"} still creates the whole project; reconcile.go iterates all of project.Services), and never read at all (StartOptions.Services — unused in start.go). The CLI compensates by shrinking *types.Project upfront (cmd/compose/up.go:70), an invariant written nowhere. Document per-struct, or fix. → PR #14078 (merged)
• ScaleOptions has no replica count: the CLI mutates the model (cmd/compose/scale.go:100 service.SetScale) before calling the backend, while the interface doc promises "Scale manages numbers of container instances running per service". → PR #14077 (merged)
• Half of StartOptions (Attach, OnExit, ExitCodeFrom, Watch, NavigationMenu) is only honored by Up, never by Start (pkg/compose/start.go:37-95 reads only Project/AttachTo/WaitTimeout/Wait). Nothing in api.go says "Up-only". → PR #14078 (merged)
• RunOptions embeds CreateOptions but only 4 fields are propagated (pkg/compose/run.go:283-288); Exec reuses the same struct while reading only Service/Index. The embedding suggests completeness that does not exist. → PR #14078 (merged)
• Dead surface that describes a nonexistent contract: api.STARTING/RUNNING/… constants never produced (Stack.Status is actually "running(2)" from pkg/compose/ls.go:61,95), Stack.Reason never populated but read in cmd/compose/list.go:140, ConfigOptions orphaned, ServiceStatus dead (and shadowed by an unrelated type in dependencies.go:33), api/errors.go sentinels (ErrAlreadyExists, ErrForbidden, …) never returned, BuildOptions.Attestations written once and never read, PsOptions.Project / AttachOptions.Project never read, api.go:748 doc comment describes the wrong constant.
• DownOptions.Images is stringly-typed; its legal values (ImagePruneNone/Local/All) live in pkg/compose/image_pruner.go:41-47, not in pkg/api, and validation happens mid-down after containers are already removed. → #14149
• --dry-run does not intercept every mutating operation: pkg/dryrun/dryrunclient.go:330 delegates ContainerCommit to the real client, so compose commit --dry-run actually creates an image. More generally the set of intercepted operations is defined only by method position relative to the comment at dryrunclient.go:308 — add a declarative list/test. → #14150
• No stability marking: nothing distinguishes the public SDK surface (pkg/api, NewComposeService) from incidentally-exported internals (pkg/compose exposes ImagePruner, ReconcileOptions, InDependencyOrder, …). Close() isn't even on the api.Compose interface (pkg/compose/compose.go:226). Mocks are in sync today but nothing in CI regenerates and diffs them (Makefile:97-99 still passes the pre-rename Service arg).

C. Lifecycle invariants are implicit (highest regression risk)

• Two lifecycle engines coexist and the boundary is undocumented: the plan-based reconciler has exactly one caller (create.go:122); start/stop/restart/down use the imperative InDependencyOrder engine. The plan almost never emits OpStartContainer — startup is a second, disjoint pass (start.go:52-68) with a different dependency traversal. Document (or converge) before anyone "fixes" the reconciler's case StateCreated: // nothing to do. → convergence tracked in #14081 (redesigned; lot 0 merged: tests PR #14104, startMx fix PR #14106)
• Label taxonomy is undocumented and inconsistent:
  • the de-facto project-membership invariant is carrying com.docker.compose.config-hash (containers.go:74-88 adds it to every default filter), not project+service labels;
  • hook helper containers carry project/service labels but no config-hash — their own comment (pre_start.go:140-147) claims compose down can find them; it cannot;
  • oneoff semantics differ between the Go predicate (containers.go:160-163: missing label ⇒ not one-off) and the daemon filter (filters.go:44-50: missing label ⇒ excluded), and the literal "True" is compared in ≥5 files with no constant;
  • com.docker.compose.depends_on is a serialized mini-language (svc:condition:restart) written from map iteration (non-deterministic order, create.go:593-597) and parsed without validation (compose.go:385-406, Required silently forced to true);
  • container-number: absent on one-offs, parse failures silently become 0 (observed_state.go:322) while being used as identity in plan ResourceIDs and container names;
  • a second, undeclared label registry exists for OCI publish (internal/oci/push.go:77-91, publish.go:134-211) and bridge (transformers.go:32). Document the full taxonomy in pkg/api/labels.go (who writes, who reads, membership invariant, compat policy).
• ~~ waitDependencies swallows its timeout: convergence.go:186-189 returns nil on ctx.Done(), making the DeadlineExceeded translation at :257-261 (and start.go:87-93's "application not healthy after %s") mostly unreachable. Four callers (up --wait, restart, run, start) depend on an error the callee doesn't reliably produce; no test covers either message.~~ → PR #14105 (merged)
• ObservedState documents 2 buckets but has 4: one-off running containers are silently dropped, and disabled-profile services land in Containers yet are never reconciled (observed_state.go:164-177 vs reconcile.go:587) — emitRunningEvents was already patched around this symptom (observed_state.go:371-373, #13882).
• isOrphaned (containers.go:145-158) conflates "service absent from model" with "exited compose run container", so up --remove-orphans deletes exited one-offs of perfectly declared services; down --remove-orphans also removes running one-offs through a different branch (down.go:47-50 vs :86-94). Intentional or not, it deserves a name/doc that says so. → PR #14142 (open)
• Load-bearing subtleties with no local warning: sortContainers (reconcile.go:1012-1034) — an uncommented double inversion is the scale-down policy; three deliberately-divergent views of observed containers (r.observed.Containers mutated as a communication channel at reconcile.go:502-505, the memoized snapshot, and the executor's live view) with the "do not fix" comment present at only one of the three sites; getCreateConfigs (create.go:252-425, 172 lines) mixes hash computation, project mutation via network-map pointer, network I/O and file reads — any added field silently invalidates the config hash of every container in the field.

D. Environment variable resolution is inconsistent and unregistered

• COMPOSE_REMOVE_ORPHANS has three resolution mechanisms: up reads it in PreRunE after the project .env is injected (up.go:124-126); down/kill read it at cobra-tree construction time, before .env injection (down.go:66-67, kill.go:52-53) — so the variable set in the project's .env works for up but not down/kill; run never reads it. Its sibling COMPOSE_IGNORE_ORPHANS is read from a different source (project.Environment, up.go:130), and create reads neither despite sharing the options struct (createOptions.ignoreOrphans never assigned, still sent to the API). → PR #14139 (open)
• setEnvWithDotEnv (compose.go:692-728) — the mechanism that re-injects COMPOSE_* keys from the project .env into the process environment — is documented nowhere, is skipped for remote (OCI/Git) configs, and only benefits variables read after PersistentPreRunE. Which variables are read when is currently unknowable without reading RootCommand line by line.
• No central env-var registry: ~25 recognized variables are scattered across 5+ constant files plus compose-go-handled ones (COMPOSE_FILE, COMPOSE_PROFILES, …); docs/reference/compose.md documents 6 of them. A single documented table (name → where read → process-env or project-env → default) would remove a whole class of wrong guesses. → PR #14145 (open)
• Legacy/experimental leftovers: COMPOSE_BAKE is read by no production code (only a stale e2e env); COMPOSE_EXPERIMENTAL_WATCH_TAR=0 now hard-breaks watch ("no available sync implementation") since the non-tar syncer was removed (d20340299); COMPOSE_EXPERIMENTAL_GIT_REMOTE/_OCI_REMOTE default to true despite the name; internal/experimental is dead code (no importers) describing a feature-flag mechanism that is not wired. → PR #14085 (open)

E. Global mutable state and output-channel sprawl

• display.Mode package global: written from 4 files, not assigned in the default interactive case (compose.go:657-667), compared once against a string literal instead of the constant (up.go:350). → PR #14103 (merged)
• pkg/compose/watch.go:82 — the Watcher mutex is package-level (all watchers in a process serialize), and the Start/Stop state machine leaks a stale error + non-nil stopFn on failed start; exercised by the interactive w shortcut (shortcut.go:286-310). → PR #14116 (community, open)
• swarmEnabled is a package-level sync.Once cache (compose.go:474-479) shared across service instances and never reset in tests (errors are cached forever, unlike the instance-level runtimeVersionCache next to it).
• --quiet is implemented by reassigning os.Stdout (build.go:104-108, config.go:96-100); the default prompt and publish print via bare fmt.Println bypassing WithStreams (compose.go:78-82, publish.go:86).
• The shared *ProjectOptions singleton is mutated by subcommands (build.go:161 sets opts.All = true; completion.go sets Offline), and ps declares its own All field shadowing ProjectOptions.All — same identifier, two meanings, neighboring files. build also re-registers a hidden --progress flag bound to the same variable as the root flag with a different default (build.go:144).
• At least 7 output channels with no written convention (progress→stderr, TTY detection probing Err() in one place — with a good comment at compose.go:651-655 — and Out() in another (colors.go:80), interactive menu and ANSI cursor sequences hardcoded to os.Stdout in cmd/formatter); the EventProcessor contract (Start→…→Done, non-reentrant) is undocumented and already violated by nested publish→push operations, with a blocking unbuffered done channel in the TTY renderer (tty.go:44,177-186).

F. Dual code paths where only one is complete

• Bake vs classic build: the decision matrix exists only as code (build.go:99-106 + buildWithBake); classic silently ignores cache_from/cache_to, no_cache_filter, shm_size, ulimits, entitlements, provenance/sbom, dockerfile_inline (compare imageBuildOptions() with bakeTarget); classic --push pushes the whole project once per built service (build_classic.go:102-106); classic-built images lack the compose labels bake applies (getImageBuildLabels has one caller), skewing everything that filters images by project (down --rmi local, watch prune); dry-run bake returns a differently-keyed map than the real path (build_bake.go:585-596 vs :412-424); api.ImageBuilderLabel is write-only.
• Watch has per-platform implementations chosen by a build tag set only in Dockerfile:87: dev builds on macOS run the naive watcher while shipped binaries run FSEvents — debugging a macOS watch bug locally does not exercise shipped code; watcher_naive.go:35-38's platform comment is wrong; go build -tags fsnotify on Linux doesn't compile.
• handleWatchBatch (watch.go:533-608) has a load-bearing implicit order (rebuild → sync → restart → exec) and stores indices into the rules slice for exec hooks — any reordering of rules silently corrupts hook execution.
• Project resolution: projectOrName vs toProjectName (compose.go:246-289) give opposite precedence, and a loading error is silently swallowed when COMPOSE_PROJECT_NAME is set — commands fall back to label-based reconstruction without saying so. Service-name validation is inconsistent: restart/wait silently no-op on unknown names while stop/ps/etc. error, and two commands re-implement validation by hand (ps.go:101-106, volumes.go:66-72). → #14151
• Backend construction happens three ways (withBackend helper, inline NewComposeService(dockerCli, backendOptions.Options...), and bare NewComposeService(dockerCli) in bridge.go:68/config.go) — the bare form silently drops --dry-run, --parallel and the progress mode; a new command written by imitation inherits the bug.

G. Guardrails (keep the map accurate)

• CI check: make mocks + git diff --exit-code (and fix the stale mockgen invocation). → PR #14102 (merged)
• Compile-test the docs/sdk.md examples. → PR #14136 (merged)
• Unit test the .env → env-var → flag resolution order (the COMPOSE_REMOVE_ORPHANS divergence in D is currently only pinned by one e2e test on up). → PR #14139 (open)
• A short "legibility" section in CONTRIBUTING/AGENTS.md: comments must state behavior, not history ("matching the previous X behavior" is how items in section A were born). → PR #14135 (merged)

Suggested sequencing

1. Documentation-only truth fixes (A, label taxonomy, env registry) — no runtime risk, immediate payoff for any reader.
2. Mechanical deletions/renames (dead API surface, convergence.go, internal/experimental, stale messages).
3. Extract the items as standalone bug issues (commit --dry-run, classic --push, waitDependencies timeout, COMPOSE_REMOVE_ORPHANS divergence, COMPOSE_EXPERIMENTAL_WATCH_TAR).
4. Contract honesty in pkg/api (docs first, then field removal/deprecation).
5. Structural work (single lifecycle engine story, env resolution unification) — each large enough to deserve its own design discussion.