Skip to content

Move upstream xz to vendor directory - #11

Merged
youknowone merged 1 commit into
mainfrom
remove-lzma
May 22, 2026
Merged

Move upstream xz to vendor directory#11
youknowone merged 1 commit into
mainfrom
remove-lzma

Conversation

@youknowone

@youknowone youknowone commented May 21, 2026

Copy link
Copy Markdown
Owner

Summary by CodeRabbit

  • Chores
    • Restructured internal crate organization by removing the local liblzma-sys crate and adopting the published version from crates.io.
    • Updated build scripts, CI workflows, and test configurations to reflect the new directory structure and dependency management.
    • Streamlined documentation and examples to match the updated codebase layout.

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 21, 2026

Copy link
Copy Markdown

Warning

Rate limit exceeded

@youknowone has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 41 minutes and 10 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 27174a35-da3a-44bf-8e3b-6d7bde491c89

📥 Commits

Reviewing files that changed from the base of the PR and between 46caf42 and 752303d.

📒 Files selected for processing (33)
  • .github/dependabot.yml
  • .github/scripts/update-xz.sh
  • .github/workflows/main.yml
  • .github/workflows/publish.yml
  • .gitmodules
  • Cargo.lock.msrv
  • Cargo.toml
  • README.md
  • docs/performance-workflow.md
  • examples/standard_files_probe.rs
  • liblzma-sys/Cargo.toml
  • liblzma-sys/LICENSE-APACHE
  • liblzma-sys/LICENSE-MIT
  • liblzma-sys/README.md
  • liblzma-sys/build.rs
  • liblzma-sys/config.h
  • liblzma-sys/src/bindgen.rs
  • liblzma-sys/src/bindgen_wrap.rs
  • liblzma-sys/src/lib.rs
  • liblzma-sys/src/manual.rs
  • liblzma-sys/src/wasm_shim.rs
  • liblzma-sys/wasm-shim/assert.h
  • liblzma-sys/wasm-shim/stdlib.h
  • liblzma-sys/wasm-shim/string.h
  • perf-probe/Cargo.toml
  • scripts/compare_all_trimmed.sh
  • scripts/compare_backends.sh
  • scripts/run_xz_c_tests_with_xz_sys.sh
  • systest/Cargo.toml
  • systest/build.rs
  • systest/src/main.rs
  • tests/sys_equivalence.rs
  • vendor/xz

Walkthrough

This PR restructures the xz-rs repository by publishing the liblzma-sys crate independently, moving xz sources to vendor/xz, and consolidating systest to exclusively use the xz-sys backend. Cargo dependencies, git submodules, CI workflows, benchmark scripts, and documentation are updated throughout.

Changes

Dependency Migration and Submodule Relocation

Layer / File(s) Summary
Submodule relocation and root manifest updates
.gitmodules, .github/scripts/update-xz.sh, Cargo.toml, perf-probe/Cargo.toml, Cargo.lock.msrv
Git submodule moved from liblzma-sys/xz to vendor/xz; root and perf-probe Cargo.toml switch liblzma-sys from path-based to published crates.io version 0.4.6; MSRV lock file updated with registry source and checksum.
CI/CD and workflow updates
.github/dependabot.yml, .github/workflows/main.yml, .github/workflows/publish.yml
Dependabot now monitors repository root instead of liblzma-sys directory; systest test step uses --features xz-sys with corresponding cleanup; publish workflow removes separate liblzma-sys step and updates xz publish condition.

Systest Backend Consolidation

Layer / File(s) Summary
Systest manifest and feature simplification
systest/Cargo.toml, systest/src/main.rs
Remove liblzma-sys dependency and bindgen feature from systest; enforce xz-sys requirement at compile time with simplified feature validation.
Systest build configuration consolidation
systest/build.rs
Hardcode xz-sys feature requirement; simplify ctest::TestGenerator to always use vendored headers and xz-sys API path; remove conditional branches for liblzma-sys and bindgen feature handling.

Performance Infrastructure Updates

Layer / File(s) Summary
Benchmark script restructuring
scripts/compare_backends.sh, scripts/compare_all_trimmed.sh, scripts/run_xz_c_tests_with_xz_sys.sh
Make root test measurements unconditional in compare_backends.sh; remove systest prebuild and measurement from compare_all_trimmed.sh; add CMake cache invalidation and update XZ_SRC_DIR to vendor/xz in run_xz_c_tests_with_xz_sys.sh.

Tests and Documentation Updates

Layer / File(s) Summary
Test suite and equivalence checks
tests/sys_equivalence.rs
Add xz-sys feature compatibility test; update exported function assertions; change test corpus path from liblzma-sys/xz/tests/files to vendor/xz/tests/files.
README and performance workflow documentation
README.md, docs/performance-workflow.md, examples/standard_files_probe.rs
Expand xz crate description highlighting pure-Rust derivation and c2rust translation cleanup; remove liblzma-sys from backend options; clarify C backend uses vendored sources via LZMA_API_STATIC=1; update corpus path in example; narrow documented regression-gate outputs to root-tests only.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • youknowone/xz-rs#10: Adds upstream C test job that directly uses scripts/run_xz_c_tests_with_xz_sys.sh, the script updated here.
  • youknowone/xz-rs#8: Updates publish workflow conditions to transition from liblzma-sys to xz-sys tag naming in the same workflow file.
  • youknowone/xz-rs#5: Introduces backend comparison tooling in scripts/compare_backends.sh and docs/performance-workflow.md, both further refined here.

🐰 A hop, a skip, a refactor so neat,
Our paths now dance where vendor and bindings meet.
From local to crates.io, one backend we choose,
Pure Rust in the spotlight—no C libs to lose!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main structural change: moving the upstream xz submodule from the liblzma-sys directory to a vendor directory, which is reflected across all configuration files (.gitmodules, .github scripts, Cargo.toml references) and examples.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch remove-lzma

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
README.md (1)

69-73: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Clarify README backend options: liblzma-sys is still supported
xz still defines an optional liblzma-sys feature (wired to the liblzma-sys dependency), so the README snippet at lines 69-73 is still valid.
The backend description (lines 63-68) lists only xz-core and xz-sys; update it to explicitly mention that features = ["liblzma-sys"] switches to the external C liblzma-sys implementation (and how that differs from the xz-sys backend).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@README.md` around lines 69 - 73, Update the README backend description to
explicitly mention the optional feature "liblzma-sys" on the xz crate and
explain how to enable it (features = ["liblzma-sys"]) to switch from the
Rust-based xz-core/xz-sys backends to the external C liblzma-sys implementation;
describe the practical difference (uses system/liblzma C library vs bundled Rust
implementation) and clarify compatibility/behavior differences compared to
xz-sys so readers know when to choose the C backend.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/publish.yml:
- Line 17: The publish workflow condition is too permissive; update the if
condition that references github.ref so it only triggers for tags explicitly
targeting the xz crate (e.g., tags beginning with "xz-") while still excluding
xz-sys tags. Locate the conditional line that currently uses
startsWith(github.ref, 'refs/tags/xz') and replace it with a more specific check
using startsWith(github.ref, 'refs/tags/xz-') combined with the existing
not-startsWith exclusion for 'refs/tags/xz-sys' so only tags like "xz-..." (not
generic tags like "v1.0.0") trigger the publish. Ensure the final expression
keeps github.ref as the referenced symbol and preserves the negation for
'xz-sys'.

In `@Cargo.toml`:
- Line 25: The liblzma-sys dependency is declared without disabling its defaults
so its default = ["bindgen"] will be pulled in whenever the xz backend is
enabled; update the Cargo.toml entry for liblzma-sys to opt out of default
features (add default-features = false) if you want bindgen to be opt-in, or
alternatively remove/adjust the xz crate's passthrough feature (bindgen =
["liblzma-sys?/bindgen"]) so it no longer implicitly enables liblzma-sys's
bindgen; target the liblzma-sys dependency declaration and the xz crate feature
named bindgen when making the change.

In `@tests/sys_equivalence.rs`:
- Around line 127-147: The test xz_sys_keeps_liblzma_sys_compatible_features
currently hardcodes an expected BTreeMap built from
include_str!("../xz-sys/Cargo.toml") which makes it brittle; change the test to
either (preferred) parse liblzma-sys's Cargo.toml at a known version (or the
published crate manifest) and compare the parsed feature maps from
parse_feature_table for both crates instead of asserting against the literal
`expected` BTreeMap, or (if you want minimal change) add a clear comment above
the `expected` BTreeMap explaining it must be kept in sync with liblzma-sys
0.4.6 and why; locate and update the test function
xz_sys_keeps_liblzma_sys_compatible_features and the use of
parse_feature_table/include_str!("../xz-sys/Cargo.toml") to implement the
dynamic comparison or the explanatory comment.

---

Outside diff comments:
In `@README.md`:
- Around line 69-73: Update the README backend description to explicitly mention
the optional feature "liblzma-sys" on the xz crate and explain how to enable it
(features = ["liblzma-sys"]) to switch from the Rust-based xz-core/xz-sys
backends to the external C liblzma-sys implementation; describe the practical
difference (uses system/liblzma C library vs bundled Rust implementation) and
clarify compatibility/behavior differences compared to xz-sys so readers know
when to choose the C backend.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6164ff71-67aa-47e8-8295-493435e4f5da

📥 Commits

Reviewing files that changed from the base of the PR and between 7c964d4 and 46caf42.

📒 Files selected for processing (33)
  • .github/dependabot.yml
  • .github/scripts/update-xz.sh
  • .github/workflows/main.yml
  • .github/workflows/publish.yml
  • .gitmodules
  • Cargo.lock.msrv
  • Cargo.toml
  • README.md
  • docs/performance-workflow.md
  • examples/standard_files_probe.rs
  • liblzma-sys/Cargo.toml
  • liblzma-sys/LICENSE-APACHE
  • liblzma-sys/LICENSE-MIT
  • liblzma-sys/README.md
  • liblzma-sys/build.rs
  • liblzma-sys/config.h
  • liblzma-sys/src/bindgen.rs
  • liblzma-sys/src/bindgen_wrap.rs
  • liblzma-sys/src/lib.rs
  • liblzma-sys/src/manual.rs
  • liblzma-sys/src/wasm_shim.rs
  • liblzma-sys/wasm-shim/assert.h
  • liblzma-sys/wasm-shim/stdlib.h
  • liblzma-sys/wasm-shim/string.h
  • perf-probe/Cargo.toml
  • scripts/compare_all_trimmed.sh
  • scripts/compare_backends.sh
  • scripts/run_xz_c_tests_with_xz_sys.sh
  • systest/Cargo.toml
  • systest/build.rs
  • systest/src/main.rs
  • tests/sys_equivalence.rs
  • vendor/xz
💤 Files with no reviewable changes (19)
  • liblzma-sys/README.md
  • scripts/compare_backends.sh
  • .github/dependabot.yml
  • liblzma-sys/build.rs
  • liblzma-sys/wasm-shim/assert.h
  • liblzma-sys/wasm-shim/string.h
  • liblzma-sys/LICENSE-APACHE
  • liblzma-sys/wasm-shim/stdlib.h
  • liblzma-sys/src/bindgen.rs
  • liblzma-sys/LICENSE-MIT
  • liblzma-sys/config.h
  • liblzma-sys/src/wasm_shim.rs
  • liblzma-sys/src/bindgen_wrap.rs
  • liblzma-sys/src/lib.rs
  • systest/Cargo.toml
  • liblzma-sys/src/manual.rs
  • scripts/compare_all_trimmed.sh
  • .github/workflows/main.yml
  • liblzma-sys/Cargo.toml

Comment thread .github/workflows/publish.yml Outdated
Comment thread Cargo.toml Outdated
Comment thread tests/sys_equivalence.rs
@youknowone
youknowone merged commit b80dfc2 into main May 22, 2026
49 checks passed
@youknowone
youknowone deleted the remove-lzma branch June 2, 2026 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant