Add Native Multi-threaded Compression Support to liblz4
After reviewing both the documentation and the source code, it appears that while the lz4 CLI supports multi-threaded compression (-T#), the underlying liblz4 API itself does not currently provide native multi-threaded compression support.
At the moment, applications need to manually:
split input into chunks, manage thread pools, compress blocks independently, and handle framing/container logic externally.
It would be very useful to have an official library-level API for parallel compression, similar in spirit to how libzstd exposes multi-threaded compression through encoder parameters.
Potential benefits:
simpler integration for downstream applications, standardized framing/block handling, better interoperability, easier adoption in modern multi-core environments, reduced duplicated implementations across projects.
Possible API directions could include:
configurable worker thread count, automatic chunk partitioning, independent block mode, optional parallel frame compression APIs.
Example (conceptually):
LZ4F_preferences_t prefs;
prefs.nWorkers = 8;
...Source: lz4/lz4