chore(deps): update dependency lz4 to v1.10.0#416
Open
deckhouse-BOaTswain wants to merge 2 commits into
Open
Conversation
vporoshok
approved these changes
Jul 22, 2026
The lz4 1.9.2 -> 1.10.0 bump left the local patches unaligned with the new upstream sources, so patches 0001-0003 no longer applied. Rebase them against the 1.10.0 layout: - 0001: visibility is now controlled via the LZ4FLIB_VISIBILITY macro - 0002: relocate the LZ4F_allocated_memory hunks to the new context - 0003: offsetof rewrite adapts to the LZ4_ALIGN_TEST-guarded typedef Verified all four patches apply with `patch -p0` on a fresh v1.10.0 tree and that the patched sources compile warning-clean. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
Author
Edited/Blocked NotificationRenovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR. You can manually request rebase by checking the rebase/retry box above. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
v1.9.2->v1.10.0Release Notes
lz4/lz4 (lz4)
v1.10.0: LZ4 v1.10.0 - Multicores editionCompare Source
LZ4
v1.10.0introduces major updates, integrating 600+ commits that significantly enhance its capabilities. This version brings multithreading support to the forefront, harnessing modern multi-core processors to accelerate both compression and decompression processing. It's a good upgrade for users looking to optimize performance in high-throughput environments.Multithreading support
The most visible upgrade of this version is likely Multithreading support. While LZ4 has historically been recognized for its high-speed compression, the demand for even faster throughput has grown, particularly with the advent of
nvmestorage technologies that allow for multi-GB/s throughput.Multithreading is particularly beneficial for High Compression modes, which now perform dramatically faster. The following benchmark table showcases the performance improvements:
Multithreading is less critical for decompression, as modern
nvmedrives can still be saturated with a single decompression thread. Nonetheless, the new version enhances performance by overlapping I/O operations with decompression processes.Tested on a x64 linux platform, decompressing a 5 GB text file locally takes 5 seconds with
v1.9.4;this is reduced to 3 seconds in
v1.10.0, corresponding to > +60% performance improvement.Official support for dictionary compression (and decompression)
Starting from
v1.10.0, dictionary compression, previously tagged as "experimental", now receives full support. This upgrade assures stability and ongoing support for the feature, enabling developers to reliably use this functionality in their applications.The new symbols supported by
liblz4are :LZ4_loadDictSlow(): minor variant ofLZ4_loadDict(), which consumes more initialization time to better reference the dictionary, resulting in slightly improved compression ratios.LZ4_attach_dictionary(): use in-place a LZ4 state initialized with a dictionary, to perform dictionary compression (LZ4 Block format) without the initialization costs. Very useful for small data, where dictionary initialization can become a bottleneck. The dictionary state can be used by multiple threads concurrently.LZ4_attach_HC_dictionary(): same asLZ4_attach_dictionary(), but for LZ4HC dictionary compression.LZ4F_compressBegin_usingDict(): initiate streaming compression to the LZ4Frame format, using a Dictionary.LZ4F_decompress_usingDict(): decompress a LZ4Frame requiring a DictionaryLZ4F_createCDict(): create a materialized dictionary, ready to start compression without initialization cost. Can be shared across multiple threads.LZ4F_compressFrame_usingCDict(): one-shot compression to the LZ4Frame format, using materializedCDictLZ4F_compressBegin_usingCDict(): initiate streaming compression to the LZ4Frame format, using materializedCDictNew compression level 2
The new "Level 2" compression effectively fills the substantial gap between the standard "Fast Level 1" and the more intensive "High Compression Level 3." It provides a balanced option, optimizing performance and compression as evidenced in the benchmark results below (i7-9700k,
linux):Level 2 is ideal for applications requiring better compression than
lz4level 1, without the speed trade-offs associated with HC level 3.Miscellaneous
LZ4_CLEVELandLZ4_NBWORKERS, offering flexible control over its behavior in scenarios where direct commands are impractical, or when customized local defaults are necessary.GPL-2.0-or-laterto distinguish it fromGPL-2.0-only, enhancing transparency. Theliblz4library maintains its BSD-2 clause license.loongArch,risc-v,m68k,mipsandsparc), and are now continuously tested in CI, to ensure portability.cmakerecipe, in an effort to reduce manual maintenance of multiple Solutions.One-liner updates
LZ4_CLEVELandLZ4_NBWORKERSGPL-2.0-or-laterstdout(except when input isstdin)lz4fileAPI improvements, by @vsolontsov-volant and @t-matLZ4_compress_destSize_extState()build/directory, by @eli-schwartzcmakevia scriptsAutomated change log
basepointer by @Cyan4973 in #1163.gitignoreby @LocalSpook in #1288.clang-formatrule file by @LocalSpook in #1287-Wpedanticinstead of-pedanticfor consistency with other-W*options by @LocalSpook in #1294FREEMEM()by @LocalSpook in #1297LZ4_NBWORKERSby @Cyan4973 in #1430LZ4_CLEVELby @Cyan4973 in #1435-BD) can employ multiple threads by @Cyan4973 in #1436stdoutby @Cyan4973 in #1442cmakeby @Cyan4973 in #1440--listoutput is sent to stdout by @Cyan4973 in #1446New Contributors
Full Changelog: lz4/lz4@v1.9.4...v1.10.0
edit: the
lz4-1.10.0.tar.gzartifact has been updated because the initial version was embedding somemacosspecific stuff.edit 2: the windows binary packages have been updated to fix a bug affecting time measurement when compressing extremely large files.
v1.9.4: LZ4 v1.9.4Compare Source
LZ4
v1.9.4is a maintenance release, featuring a substantial amount (~350 commits) of minor fixes and improvements, making it a recommended upgrade. The stable portion ofliblz4API is unmodified, making this release a drop-in replacement for existing features.Improved decompression speed
Performance wasn't a major focus of this release, but there are nonetheless a few improvements worth mentioning :
ARM64platform is improved, by ~+20%. This is notably the case for recent M1 chips, featured in macbook laptops and nucs. Some server-class ARM64 cpus are also impacted, most notably when employinggccas a compiler. Due to the diversity ofaarch64chips in service, it's still difficult to have a one-size-fits-all policy for this platform.-BD4setting (small blocks, <= 64 KB, linked) decompressed block-by-block into a flush buffer, decompression speed is improved ~+70%. This is most visible in thelz4CLI, which triggers this exact scenario, but since the improvement is achieved at library level, it may also apply to other scenarios.lz4frameformat (native format oflz4CLI), it's possible to ignore checksum validation during decompression, resulting in speed improvements of ~+40% . This capability is exposed at both CLI (see--no-crc) and library levels.New experimental library capabilities
New
liblz4capabilities are provided in this version. They are considered experimental at this stage, and the most useful ones will be upgraded as candidate "stable" status in an upcoming release :lz4frameAPI to employ custom allocators for dynamic allocation.LZ4_decompress_safe_partial_usingDict()by @yawqilz4frameblocks which are intentionally uncompressed, usingLZ4F_uncompressedUpdate(), by @alexmohrlz4file, abstracting File I/O operations for higher-level programs and libraries, by @anjiahao1liblz4can be built for freestanding environments, using the new build macroLZ4_FREESTANDING, by @t-mat. In which case, it will not link to any standard library, disable all dynamic allocations, and rely on user-providedmemcpy()andmemset()operations.Miscellaneous
Makefilebug introduced inv1.9.3, in whichCFLAGSwas no longer respected when provided from environment variable. The root cause was an obscure bug inmake, which has been fixed upstream following this bug report. There is no need to updatemaketo buildliblz4though, theMakefilehas been modified to circumvent the issue and remains compatible with older versions ofmake.Makefileis compatible with-jparallel run, including to run parallel tests (make -j test).Changes list
Here is a more detailed list of updates introduced in
v1.9.4:-BD4setting in CLILZ4_decompress_safe_partial_usingDict()by @yawqilz4frame: ability to provide custom allocators at state creationlz4filefor file i/o API, by @anjiahao1LZ4F_uncompressedUpdate(), by @alexmohr--listworks onstdininput, by @Low-power--no-crcdoes not produce (compression) nor check (decompression) checksums--testand--listproduce an error code when parsing invalid input--test -mdoes no longer create decompressed file artifactsstdin, reported by @davidmankinMakefilerespectsCFLAGSdirectives passed via environment variableLZ4_FREESTANDING, new build macro for freestanding environments, by @t-matmakeandmake testare compatible with-jparallel runNew Contributors
v1.9.3: LZ4 v1.9.3Compare Source
LZ4
v1.9.3is a maintenance release, offering more than 200+ commits to fix multiple corner cases and build scenarios. Update is recommended. Existingliblz4API is not modified, so it should be a drop-in replacement.Faster Windows binaries
On the build side, multiple rounds of improvements, thanks to contributors such as @wolfpld and @remittor, make this version generate faster binaries for Visual Studio. It is also expected to better support a broader range of VS variants.
Speed benefits can be substantial. For example, on my laptop, compared with
v1.9.2, this version built with VS2019 compresses at 640 MB/s (from 420 MB/s), and decompression reaches 3.75 GB/s (from 3.3 GB/s). So this is definitely perceptible.Other notable updates
Among the visible fixes, this version improves the
_destSize()variant, an advanced API which reverses the logic by targeting an a-priori compressed size and trying to shove as much data as possible into the target budget. The high compression variantLZ4_compress_HC_destSize()would miss some important opportunities in highly compressible data, resulting in less than optimal compression (detected by @hsiangkao). This is fixed in this version. Even the "fast" variant receives some gains (albeit very small).Also, the corresponding decompression function,
LZ4_decompress_safe_partial(), officially supports a scenario where the input (compressed) size is unknown (but bounded), as long as the requested amount of data to regenerate is smaller or equal to the block's content. This function used to require the exact compressed size, and would sometimes support above scenario "by accident", but then could also break it by accident. This is now firmly controlled, documented and tested.Finally, replacing memory functions (
malloc(),calloc(),free()), typically for freestanding environments, is now a bit easier. It used to require a small direct modification oflz4.csource code, but can now be achieved by using the build macroLZ4_USER_MEMORY_FUNCTIONSat compilation time. In which case,liblz4no longer includes<stdlib.h>, and requires instead that functionsLZ4_malloc(),LZ4_calloc()andLZ4_free()are implemented somewhere in the project, and then available at link time.Changes list
Here is a more detailed list of updates introduced in
v1.9.3:LZ4_compress_HC_destSize()ratio, detected by @hsiangkaoLZ4_decompress_safe_partial()supports unknown compressed size, requested by @jfkthameLZ4F_compressBound()with automatic flushing, by Christopher HarvieLZ4_saveDictHC()in corner case scenario, detected by @IgorKorkin--fastwith large argument, detected by @picoHzLZ4_USER_MEMORY_FUNCTIONScontrib/cmake_unofficial/moved tobuild/cmake/visual/*moved tobuild/Known issues :
liblz4_static.libfile in the packagelz4_win64_v1_9_3.zip. This is probably amingw/msvccompatibility issue. If you have issues employing this file, the solution is to rebuild it locally from sources with your target compiler.Makefileinv1.9.3doesn't honorCFLAGSwhen passed through environment variable. This is fixed in more recent version ondevbranch. See #958 for details.Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Renovate Bot.