[Core] Write session files atomically so a partial write cannot destroy them - #34060
[Core] Write session files atomically so a partial write cannot destroy them#34060om singhal (Om-singhaI) wants to merge 4 commits into
Conversation
save() opened the target with 'w', which truncates it in place and then streams the document into it. While that runs the file on disk is empty or half written, and a concurrent process that reads it fails to parse it, at which point load() overwrites it with defaults. A process that only meant to read the file destroys it. The same truncation loses the file outright if the write itself fails partway, with no second process involved. It now writes a temporary file in the same directory and renames it over the target, so a reader sees either the old document or the new one. The guards each cover a case the plain rename would regress: realpath so a symlinked config file is followed rather than replaced, chmod so mkstemp's 0600 does not silently tighten an existing file, the OSError fallback so a read only config directory still saves the way it used to, and BaseException so an interrupt does not leave a stray temporary file. This does not serialize concurrent writers. Two saves can still overwrite each other, cleanly rather than corruptly. That is Azure#14070 and needs a lock.
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
|
Thank you for your contribution om singhal (@Om-singhaI)! We will review the pull request and get back to you soon. |
There was a problem hiding this comment.
🟡 Changes recommended
The critical fallback can truncate files after non-permission errors, and test temporary directories need cleanup.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Updates Azure CLI session persistence to write JSON files atomically, preserving permissions and symlink targets.
Changes:
- Writes through same-directory temporary files before replacement.
- Adds fallback behavior and failure cleanup.
- Adds tests for atomic saves and edge cases.
File summaries
| File | Summary |
|---|---|
src/azure-cli-core/azure/cli/core/_session.py |
Implements atomic session writes. Critical (3 votes): fallback catches all OSError values and may reintroduce data loss. |
src/azure-cli-core/azure/cli/core/tests/test_session.py |
Adds session persistence tests. Nit (1 vote): temporary directories are not cleaned up. |
Review details
Suppressed comments (1)
src/azure-cli-core/azure/cli/core/tests/test_session.py:19
- Each test creates a new
mkdtemp()directory, but this fixture is never removed, so repeated runs leave eight directories behind in the system temp directory. Core tests using the same pattern clean up intearDown(for example,test_extension.py:79-94andtest_azlogging.py:18-23); register anaddCleanup/TemporaryDirectorycleanup here.
self.dir = tempfile.mkdtemp()
self.filename = os.path.join(self.dir, 'test.json')
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…sion The fallback caught every OSError from mkstemp. A full disk or an exceeded quota would land there too, and the in place write would fail the same way after having already truncated the file, which is the data loss this change exists to stop. It now falls back on EACCES, EPERM and EROFS and lets anything else surface. Note EROFS is a plain OSError rather than a PermissionError, so catching PermissionError alone would miss a read only file system. Also removes the temporary directory the tests were leaving behind.
|
Core |
There was a problem hiding this comment.
🔵 Needs a closer look
The test suite has unresolved Windows compatibility and read-only fallback coverage issues.
Review details
Suppressed comments (3)
Previously missed (1) — in code that hasn't changed since the last review.
src/azure-cli-core/azure/cli/core/tests/test_session.py:68
- This test asserts POSIX mode bits, but it is not skipped on Windows.
os.chmodon Windows does not preserve arbitrary0o644permissions, sostat.S_IMODE(os.stat(...).st_mode) == 0o644can fail on the Windows test runner; the existing permission tests inazure-cli-core/azure/cli/core/tests/test_azlogging.py:41skip Windows for this reason. Skip this test onos.name == 'nt'or assert only platform-independent behavior.
This issue also appears on line 76 of the same file.
src/azure-cli-core/azure/cli/core/tests/test_session.py:76
os.symlinkexists on Windows even when creating one requires the symlink privilege/Developer Mode, so checking onlyhasattrdoes not preventPermissionErroron standard Windows runners. The repository skips its other symlink test on Windows (azure-cli/azure/cli/command_modules/acs/tests/latest/test_custom.py:497); apply the same platform guard here or convert a permission failure into a skipped test.
@unittest.skipUnless(hasattr(os, 'symlink'), 'requires symlink support')
src/azure-cli-core/azure/cli/core/tests/test_session.py:123
- Mocking
mkstempto returnEROFSleaves the destination on the writable test filesystem, so the fallbackopen(..., 'w')succeeds. On a real read-only filesystem that open would also returnEROFS, so this test does not cover the claimed scenario and can give false confidence. TestEACCES/EPERMfor a non-writable directory with an existing writable file, or removeEROFSfrom the fallback contract.
with mock.patch('tempfile.mkstemp', side_effect=OSError(errno.EROFS, 'Read-only file system')):
self._session({'a': 1}).save()
- Files reviewed: 2/2 changed files
- Comments generated: 0 new
- Review effort level: Lite
EROFS does not belong in the fallback. If the file system is read only then the in place write cannot succeed either, so catching it only delays the same error while implying a recovery that does not exist. The real case is a directory that denies writes while the file itself is writable, which is EACCES or EPERM, and test_save_still_writes_when_the_directory_is_not_writable already covers it with a real 0500 directory rather than a mock. The permission and symlink tests assert POSIX behaviour, so they are skipped on Windows the way test_azlogging and the acs tests already do. os.symlink exists there but creating one needs a privilege, so hasattr was the wrong guard.
There was a problem hiding this comment.
🔵 Needs a closer look
Fix fallback serialization safety and make the permission test reliably exercise the fallback path.
Review details
Suppressed comments (2)
src/azure-cli-core/azure/cli/core/_session.py:79
- When
mkstempfails withEACCES/EPERM, this fallback opens the live file with'w'and then serializes directly into it. Ifjson.dumpraises after emitting a prefix (for example, because a later value is not JSON-serializable), the existing session file is left truncated, so the partial-save protection is lost on the read-only-directory path. Serialize to a string before opening the target; write failures can still be unavoidable in this fallback, but serialization errors should not destroy the old file.
with open(self.filename, 'w', encoding=self._encoding) as f:
json.dump(self.data, f)
src/azure-cli-core/azure/cli/core/tests/test_session.py:102
- This test relies on
chmod(0o500)to makemkstempfail, so a root or otherwise privileged test process can still create the temporary file and pass through the atomic path without exercising the permission-denied fallback. Mockmkstempto raiseEACCESwhile retaining the writable existing file, or otherwise assert that the fallback path was taken.
os.chmod(self.dir, 0o500)
try:
self._session({'a': 1}).save()
- Files reviewed: 2/2 changed files
- Comments generated: 0 new
- Review effort level: Lite
The fallback opened the live file and then serialized into it, so data that failed to serialize partway left the file truncated. That is the failure this change exists to remove, still present on the one path that cannot use a temporary file. It now builds the document first and only opens the target once there is something complete to write. The read only directory test relied on the directory mode stopping mkstemp, which root ignores, so on a privileged runner it silently took the atomic path instead. It is skipped for root now, and a mocked EACCES test covers the fallback everywhere.
Related command
Not command specific.
azure/cli/core/_session.pybacks~/.azure/azureProfile.json,commandIndex.jsonand the rest.Description
Fixes #9427.
save()opens the target with'w', so the file is truncated in place and is empty or half a document while the write runs. A process reading it in that window fails to parse it, andload()then overwrites it with defaults. The file is also lost if the write itself raises partway, with no second process involved.save()now writes a temporary file in the same directory and renames it over the target, so a reader sees either the old document or the new one.The rename follows a symlink rather than replacing it, carries over the existing file's permissions, and falls back to the in place write when the config directory is not writable, since that still works today.
It does not serialize concurrent writers. Two saves can still overwrite each other, cleanly rather than corruptly, which is #14070.
Testing Guide
Adds
test_session.pywith eight tests covering the atomic replace, a save that raises partway, permissions, symlinks and a read only directory.History Notes
[Core] Fix
~/.azurefiles being corrupted or emptied when a save is interrupted or another process reads the file mid write