kata-deploy: install never removes artifacts absent from the new payload → stale confidential initrd persists across in-place upgrades

Summary

On an in-place upgrade where the artifact set changes, kata-deploy's install action never removes files or symlinks that exist in the install dir from a prior install but are absent from the new payload. They are neither overwritten (not in the new payload) nor garbage-collected, so a stale, wrong-version artifact persists indefinitely.

This is distinct from the already-fixed cp -au overwrite bug (fixed in 3.26.0 by the Rust rewrite's remove-then-copy). This one is a missing GC, present on main.

Root cause (source)

tools/packaging/kata-deploy/binary/src/artifacts/install.rs:

• install_artifacts() (L82) — the Install / InstallStageArtifacts action — does not clean the install dir first: fs::create_dir_all(host_install_dir) (keeps existing contents) → extract_component_tarballs → copy_artifacts.
• copy_artifacts() (L664) walks the source only: for entry in WalkDir::new(src). It creates/overwrites a dst entry for each src entry and never enumerates dst to find entries absent from src.
• remove_artifacts() (full removal, L138) is invoked only on Cleanup / Reset / CleanupStageRemoveArtifacts, not on Install.
• The reconcile_* helpers (debug tools, debug-variant handler dirs, devkit, stale drop-ins) are targeted — none covers a dropped guest image/initrd. (cf. #13141, a targeted scrub for a different stale item — same pattern, no general GC.)

Reproduction

1. Install Confidential Containers 0.18.0 (kata 3.25.0, which ships kata-containers-initrd-confidential.img).
2. Upgrade in place to 0.22.0 (kata 4.0.0, which is image-mode for the confidential shim and does not ship that initrd).
3. Compare mtimes/targets of the kata-containers-*confidential* symlinks in <prefix>/share/kata-containers/.

Observed on a live node (upgraded 0.18.0 → 0.22.0):

kata-containers-initrd-confidential.img   symlink mtime Aug 22 (0.18 install)  -> target contains kata-agent 3.25.0   # STALE, never removed
kata-containers-confidential.img          symlink mtime Aug 23 (0.22 install)  -> target contains kata-agent 4.0.0     # refreshed (shipped by new payload)

The symlink whose artifact the new payload ships got a fresh mtime; the one it no longer ships kept the prior install's mtime and 3.25.0 contents. Independently reproduced on two SEV-SNP setups.

Orthogonal to installationPrefix / #13697

One of the setups uses a custom prefix, but nothing here is prefix-dependent — the missing GC applies at the default /opt/kata too. Please don't fold this into the installation-prefix drop-in issue (#13697); they are unrelated.

Impact

A stale, wrong-version guest artifact is left on the host. Whether it bites depends on whether a shim consumes it: image-mode confidential shims (qemu-snp) ignore the leftover initrd, but any confidential shim/config that references the initrd path would boot a version-mismatched agent under a newer shim (a silent guest-agent/runtime skew — e.g. streaming-RPC/ReadStream incompatibilities). At minimum it is incorrect state and wasted disk; at worst a hard-to-diagnose skew. In practice the only thing that caught it was a version-equality guard on the artifact.

Proposed fix

On the Install path, GC managed-artifact paths present in the install dir but absent from the payload — e.g. track an install manifest (the payload already carries versions.yaml / shim-components.json) and remove share/kata-containers/* entries (and their symlinks) not present in the new payload, before/after copy. This generalizes the existing targeted reconcile_* / #13141 pattern into "reconcile the shipped artifact set."

Environment

Upgrade path CoCo 0.18.0 → 0.22.0 (kata 3.25.0 → 4.0.0), in-place on a live node; SEV-SNP bare metal. kata-deploy main (install.rs as cited).