Skip to content

fsync the temporary .env file before replacing it - #754

Open
boriskozak wants to merge 2 commits into
theskumar:mainfrom
boriskozak:python-dotenv-713-fix
Open

boriskozak wants to merge 2 commits into
theskumar:mainfrom
boriskozak:python-dotenv-713-fix

Conversation

@boriskozak

Copy link
Copy Markdown

Problem

set_key and unset_key rewrite the .env file through rewrite(): they write a temporary file next to the target, then os.replace it over the target. The temporary file is closed (which flushes Python's buffer to the OS) and then renamed, with no fsync. If the machine loses power or the kernel crashes shortly after set_key returns, the rename can reach the disk before the file's data does, leaving an empty or garbage .env behind instead of either the old or the new contents.

Fix

In rewrite() (src/dotenv/main.py):

  • After the caller's writes finish and while the temporary file is still open, flush it and os.fsync it, so the new contents are on disk before the rename.
  • After a successful os.replace, best-effort fsync the containing directory on POSIX so the rename itself is durable. Failures are ignored: the contents are already safe, and some filesystems reject directory fsync.
  • A flush or fsync failure still discards the temporary file and reports the original error (a persistent flush error is re-raised by close(), so that path is folded in too).

Testing

  • Reproduced first on main: patched os.fsync/os.replace and recorded call order; os.replace ran with no preceding fsync for both set_key and unset_key. After the fix, fsync runs before replace for both.
  • Added tests: fsync-before-replace ordering for set_key/unset_key, directory-fsync failures ignored (both the os.open and EINVAL paths), and flush/fsync failures leaving the original file untouched with no temp file behind.
  • tests/test_main.py: 161 passed, 3 skipped. Ruff check and format clean.

Fixes #713

rewrite() wrote the new contents to a temporary file and renamed it
over the target without fsync, so a power loss shortly after set_key
or unset_key returned could leave behind an empty or garbage .env.
Flush and fsync the temporary file while it is still open, and
best-effort fsync the containing directory on POSIX so the rename
itself is durable. Directory fsync failures are ignored. A flush or
fsync failure still discards the temporary file and reports the
original error.

Fixes theskumar#713
…h typing

The directory-fsync tests assert on os.fsync calls, but
_fsync_directory returns early on non-POSIX platforms, so the
calls never happen on Windows. Skip them there.

Annotate the patcher in the flush/fsync failure test as
AbstractContextManager[Any]: the two mock.patch call shapes
return incompatible _patch generics and mypy 3.13 rejects the
reassignment.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

set_key and unset_key don't fsync before replacing the .env file

1 participant