The test suite writes to the developer's HOME directory and never cleans up
What happens
Running the spec suite leaves files in $HOME. It does not converge on a fixed set: a second run leaves more than the first.
Measured by pointing HOME at an empty directory and running the full suite:
after one run: 58 entries
after two runs: 83 entriesWhat gets written
~/.fastlane/.did_show_opt_info
~/Library/Developer/Xcode/Archives
~/Library/Developer/Xcode/UserData
~/Library/MobileDevice/Provisioning Profiles
~/Library/Logs/fastlane
~/Library/Logs/gym
~/Library/Logs/scan
~/Library/Logs/snapshot
~/.appstoreconnect
~/.cache/rubocop_cachePlus certificates in the default keychain, which is #30186.
Why it matters beyond tidiness
Specs also read that state, and a spec that depends on something a previous run left behind passes on every machine that has run the suite before and fails only on a clean checkout. That is a defect that can survive for a long time while looking like CI flakiness.
A concrete example from #30184: four examples in spaceship/spec/tunes/tunes_client_spec.rb counted HTTP requests, and one of those requests was Client#itc_service_key fetching a key it caches to disk. Whether the count was right depended on whether the cache file happened to exist. spaceship/spec/spec_helper.rb deleted that file around every example to force the count, which works in one process and races across several.
Reproducing
HOME=$(mktemp -d) bundle exec rspecNine examples fail, because security needs a keychain that a fresh home does not have. Seeding an empty one first makes the suite pass:
security create-keychain -p "" login.keychain
security default-keychain -s login.keychain
security list-keychains -s login.keychainNot covered here
Whether the fix is for specs to stop writing to $HOME, or for the suite to run against an isolated one. Both are defensible and it is a maintainer call.
Source: fastlane/fastlane