Fix partially-installed policy packs being treated as installed forever
Problem
If a required policy pack's installation fails partway through, the CLI treats the pack as installed on every subsequent run and never re-downloads it. Every later update fails with a confusing error that points at the symptom rather than the cause:
error: failed to load Pulumi policy project located at "/Users/me/.pulumi/policies/myorg/pulumi-analyzer-hitrust-google-cloud-v2.1.1": open /Users/me/.pulumi/policies/myorg/pulumi-analyzer-hitrust-google-cloud-v2.1.1/PulumiPolicy.yaml: no such file or directoryThe pack directory on disk existed but contained only an empty node_modules — no PulumiPolicy.yaml, no package.json. There is no way to recover other than manually rm -rf-ing the pack directory, which requires knowing the on-disk layout.
Two distinct defects combine to produce this:
The installed-check is too weak.
GetPolicyPath(sdk/go/common/workspace/plugins.go:2001) reportsinstalled = trueif the pack path exists and is a directory. It never checks that the directory holds a usable policy pack. That result flows throughcloudRequiredPolicy.Installed()(pkg/backend/httpstate/policypack.go:92) toinstallPolicyPack(pkg/engine/policypacks.go:44), which returns early and skips download and extraction. The analyzer is then loaded from that same directory and fails atpkg/resource/plugin/analyzer_plugin.go:141.Failed installs leave the directory behind. In
installRequiredPolicy(pkg/backend/httpstate/policypack.go:499) thedeferonly removestempDir. Onceos.Rename(tempPackageDir, finalDir)succeeds at line 539, any later failure — theLoadPolicyPackat line 544, orInstallDependenciesat line 567 — returns an error while leavingfinalDirin place. Combined with defect 1, that poisons the cache permanently: the run that actually failed reports the real install error, and every run after it reports the misleading load error above.
Proposed Solution
- Make
GetPolicyPathreportinstalled = trueonly when the directory contains aPulumiPolicy.yaml. A directory without one is a partial install, not an installed pack. - Make
installRequiredPolicyclean upfinalDirwhen it fails after the rename, so a failed install leaves no trace and the next run retries from scratch.
Implementation Notes
GetPolicyPathis in thesdkmodule and is also used byInstalled()/LocalPath(); it must keep returning the computed path alongside the boolean, since callers rely on the path even when the pack is not installed.- Cleanup in
installRequiredPolicyshould not fire on the benign concurrent-install case already handled at line 539 (os.IsExist(err), which coversENOTEMPTY), where another process legitimately ownsfinalDir. - Both changes are needed. Fixing only the cleanup still leaves already-broken directories on users' machines wedged; fixing only the check means every failed install re-downloads but never reports why.
Acceptance Criteria
- Given a pack directory that exists but has no
PulumiPolicy.yaml, when an update runs, then the pack is re-downloaded and installed rather than reported as installed - Given a pack whose dependency install fails, when the install returns, then no pack directory is left behind and the error names the dependency failure
- Given a pack whose dependency install fails, when a second update runs, then it retries the install and surfaces the install error, not a
PulumiPolicy.yamlnot-found error - Given a fully installed pack, when an update runs, then it is still treated as installed and is not re-downloaded
- Tests added covering each scenario above
Scope
In scope: the installed-check in GetPolicyPath and failure cleanup in installRequiredPolicy.
Out of scope: validating pack contents beyond the presence of PulumiPolicy.yaml; the analogous install/cleanup paths for resource provider plugins; any change to the error text at pkg/resource/plugin/analyzer_plugin.go:141.
Source: pulumi/pulumi