Atomic writes — how tempfile + os.replace prevent corrupted JSON

What happens if the power cuts out while a process is writing to a config file? Or if antivirus software on Windows briefly locks a file mid-write? If you naively overwrite a file with , whatever partial content existed at the moment of interruption is what remains on disk. For JSON, that usually means broken syntax — throws on the next startup, and the entire configuration is effectively lost. This article walks through a standard technique for preventing that: writing to a temporary file first, then swapping it in atomically. Note: "Atomic" here means an operation either completes entirely or doesn't happen at all — there's no partial, observable in-between state. It's the same sense of the word used for database transactions. Why direct overwrites are dangerous effectively truncates the file first and then writes the new content. If the process is interrupted during that window, the file is left empty or holding incomplete content. The causes vary: a , a power outage, antivirus software briefly blocking file access on Windows, or a backup tool grabbing the file mid-write. This rarely reproduces during local development, but in a long-running production environment, it will eventually happen with near certainty. The fix: write to a temp file, then swap it in The core idea is simple. Never touch the target file directly. Write the complete new content to a temporary file first, confirm that write fully succeeded, and only then replace the target file with that temp file. generates a collision-free temporary filename and returns its file descriptor. The argument matters here: placing the temp file in the same directory as the target file ensures the following call stays within a single filesystem. Why is atomic The core of this pattern is the final call. Python's official documentation states that "will be an atomic operation on Unix" and behaves atomically on Windows as well, as long as both paths are on the same filesystem. On POSIX systems (Linux/macOS), this maps to the system call. At the kernel level, swapping a directory entry is a single indivisible operation. There is no intermediate state to observe — from the outside, the file is either in its pre-swap or post-swap state, never something in between. On Windows, Python 3.3+ implements this so that overwriting an existing file is handled atomically (roughly equivalent to calling with the flag). Thanks to this property, if the process dies in the middle of , only the unnamed temp file is affected — the real config file remains untouched in whatever valid state it was in before. Forcing persistence to disk with Even with an atomic , there's a subtler risk: the content written by might still be sitting in the OS page cache rather than physically on disk when power is lost. In that case the rename itself could complete, but the renamed file's contents might not reflect what was actually written. addresses this. only pushes Python's internal buffer out to the OS — it may still sit in OS-level cache. goes a step further and asks the OS to wait until the data is actually written to physical disk. Combining both steps ensures the temp file's content is durably on disk by the time runs. Cleaning up after a failed write If the write to the temp file itself fails partway through (disk full, permission error, etc.), is never reached, so the target file is unaffected. But the half-written temp file is left behind on disk. Left unattended, these accumulate as clutter, so it's worth deleting them explicitly on failure. The nested accounts for the possibility that itself fails (already deleted, no permission, etc.). Re-raising the original exception () ensures the caller still learns that the write failed. Guarding against leftover files Since consumes the temp filename and swaps it into the target name on success, files normally don't linger under regular operation. But in an extreme edge case — the process gets 'd in the narrow window right after the temp file is fully written but