Skip to content

[Core] Write session files atomically so a partial write cannot destroy them - #34060

Open
om singhal (Om-singhaI) wants to merge 4 commits into
Azure:devfrom
Om-singhaI:fix/session-atomic-save
Open

[Core] Write session files atomically so a partial write cannot destroy them#34060
om singhal (Om-singhaI) wants to merge 4 commits into
Azure:devfrom
Om-singhaI:fix/session-atomic-save

Conversation

@Om-singhaI

Copy link
Copy Markdown

Related command

Not command specific. azure/cli/core/_session.py backs ~/.azure/azureProfile.json, commandIndex.json and the rest.

Description

Fixes #9427.

save() opens the target with 'w', so the file is truncated in place and is empty or half a document while the write runs. A process reading it in that window fails to parse it, and load() then overwrites it with defaults. The file is also lost if the write itself raises partway, with no second process involved.

save() now writes a temporary file in the same directory and renames it over the target, so a reader sees either the old document or the new one.

The rename follows a symlink rather than replacing it, carries over the existing file's permissions, and falls back to the in place write when the config directory is not writable, since that still works today.

It does not serialize concurrent writers. Two saves can still overwrite each other, cleanly rather than corruptly, which is #14070.

Testing Guide

Adds test_session.py with eight tests covering the atomic replace, a save that raises partway, permissions, symlinks and a read only directory.

python -m unittest azure.cli.core.tests.test_session
Ran 8 tests in 0.004s
OK

History Notes

[Core] Fix ~/.azure files being corrupted or emptied when a save is interrupted or another process reads the file mid write

save() opened the target with 'w', which truncates it in place and then streams
the document into it. While that runs the file on disk is empty or half written,
and a concurrent process that reads it fails to parse it, at which point load()
overwrites it with defaults. A process that only meant to read the file destroys
it. The same truncation loses the file outright if the write itself fails partway,
with no second process involved.

It now writes a temporary file in the same directory and renames it over the
target, so a reader sees either the old document or the new one.

The guards each cover a case the plain rename would regress: realpath so a
symlinked config file is followed rather than replaced, chmod so mkstemp's 0600
does not silently tighten an existing file, the OSError fallback so a read only
config directory still saves the way it used to, and BaseException so an
interrupt does not leave a stray temporary file.

This does not serialize concurrent writers. Two saves can still overwrite each
other, cleanly rather than corruptly. That is Azure#14070 and needs a lock.
Copilot AI lite review requested due to automatic review settings September 10, 2026 23:12
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@microsoft-github-policy-service microsoft-github-policy-service Bot added the customer-reported Issues that are reported by GitHub users external to the Azure organization. label Sep 10, 2026
@microsoft-github-policy-service

Copy link
Copy Markdown
Contributor

Thank you for your contribution om singhal (@Om-singhaI)! We will review the pull request and get back to you soon.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The critical fallback can truncate files after non-permission errors, and test temporary directories need cleanup.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Updates Azure CLI session persistence to write JSON files atomically, preserving permissions and symlink targets.

Changes:

  • Writes through same-directory temporary files before replacement.
  • Adds fallback behavior and failure cleanup.
  • Adds tests for atomic saves and edge cases.
File summaries
File Summary
src/azure-cli-core/azure/cli/core/_session.py Implements atomic session writes. Critical (3 votes): fallback catches all OSError values and may reintroduce data loss.
src/azure-cli-core/azure/cli/core/tests/test_session.py Adds session persistence tests. Nit (1 vote): temporary directories are not cleaned up.
Review details

Suppressed comments (1)

src/azure-cli-core/azure/cli/core/tests/test_session.py:19

  • Each test creates a new mkdtemp() directory, but this fixture is never removed, so repeated runs leave eight directories behind in the system temp directory. Core tests using the same pattern clean up in tearDown (for example, test_extension.py:79-94 and test_azlogging.py:18-23); register an addCleanup/TemporaryDirectory cleanup here.
        self.dir = tempfile.mkdtemp()
        self.filename = os.path.join(self.dir, 'test.json')
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/azure-cli-core/azure/cli/core/_session.py Outdated
…sion

The fallback caught every OSError from mkstemp. A full disk or an exceeded quota
would land there too, and the in place write would fail the same way after having
already truncated the file, which is the data loss this change exists to stop.

It now falls back on EACCES, EPERM and EROFS and lets anything else surface. Note
EROFS is a plain OSError rather than a PermissionError, so catching PermissionError
alone would miss a read only file system.

Also removes the temporary directory the tests were leaving behind.
@yonzhan

Copy link
Copy Markdown
Collaborator

Core

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The test suite has unresolved Windows compatibility and read-only fallback coverage issues.

Review details

Suppressed comments (3)

Previously missed (1) — in code that hasn't changed since the last review.

src/azure-cli-core/azure/cli/core/tests/test_session.py:68

  • This test asserts POSIX mode bits, but it is not skipped on Windows. os.chmod on Windows does not preserve arbitrary 0o644 permissions, so stat.S_IMODE(os.stat(...).st_mode) == 0o644 can fail on the Windows test runner; the existing permission tests in azure-cli-core/azure/cli/core/tests/test_azlogging.py:41 skip Windows for this reason. Skip this test on os.name == 'nt' or assert only platform-independent behavior.

This issue also appears on line 76 of the same file.

src/azure-cli-core/azure/cli/core/tests/test_session.py:76

  • os.symlink exists on Windows even when creating one requires the symlink privilege/Developer Mode, so checking only hasattr does not prevent PermissionError on standard Windows runners. The repository skips its other symlink test on Windows (azure-cli/azure/cli/command_modules/acs/tests/latest/test_custom.py:497); apply the same platform guard here or convert a permission failure into a skipped test.
    @unittest.skipUnless(hasattr(os, 'symlink'), 'requires symlink support')

src/azure-cli-core/azure/cli/core/tests/test_session.py:123

  • Mocking mkstemp to return EROFS leaves the destination on the writable test filesystem, so the fallback open(..., 'w') succeeds. On a real read-only filesystem that open would also return EROFS, so this test does not cover the claimed scenario and can give false confidence. Test EACCES/EPERM for a non-writable directory with an existing writable file, or remove EROFS from the fallback contract.
        with mock.patch('tempfile.mkstemp', side_effect=OSError(errno.EROFS, 'Read-only file system')):
            self._session({'a': 1}).save()
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

EROFS does not belong in the fallback. If the file system is read only then the in
place write cannot succeed either, so catching it only delays the same error while
implying a recovery that does not exist. The real case is a directory that denies
writes while the file itself is writable, which is EACCES or EPERM, and
test_save_still_writes_when_the_directory_is_not_writable already covers it with a
real 0500 directory rather than a mock.

The permission and symlink tests assert POSIX behaviour, so they are skipped on
Windows the way test_azlogging and the acs tests already do. os.symlink exists
there but creating one needs a privilege, so hasattr was the wrong guard.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Fix fallback serialization safety and make the permission test reliably exercise the fallback path.

Review details

Suppressed comments (2)

src/azure-cli-core/azure/cli/core/_session.py:79

  • When mkstemp fails with EACCES/EPERM, this fallback opens the live file with 'w' and then serializes directly into it. If json.dump raises after emitting a prefix (for example, because a later value is not JSON-serializable), the existing session file is left truncated, so the partial-save protection is lost on the read-only-directory path. Serialize to a string before opening the target; write failures can still be unavoidable in this fallback, but serialization errors should not destroy the old file.
            with open(self.filename, 'w', encoding=self._encoding) as f:
                json.dump(self.data, f)

src/azure-cli-core/azure/cli/core/tests/test_session.py:102

  • This test relies on chmod(0o500) to make mkstemp fail, so a root or otherwise privileged test process can still create the temporary file and pass through the atomic path without exercising the permission-denied fallback. Mock mkstemp to raise EACCES while retaining the writable existing file, or otherwise assert that the fallback path was taken.
        os.chmod(self.dir, 0o500)
        try:
            self._session({'a': 1}).save()
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

The fallback opened the live file and then serialized into it, so data that failed
to serialize partway left the file truncated. That is the failure this change exists
to remove, still present on the one path that cannot use a temporary file. It now
builds the document first and only opens the target once there is something complete
to write.

The read only directory test relied on the directory mode stopping mkstemp, which
root ignores, so on a privileged runner it silently took the atomic path instead. It
is skipped for root now, and a mocked EACCES test covers the fallback everywhere.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

act-platform-engineering-squad Auto-Assign Auto assign by bot Core CLI core infrastructure customer-reported Issues that are reported by GitHub users external to the Azure organization.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Corruption of ~/.azure files

4 participants