#4779·zstd

ZSTD_compressSequencesAndLiterals() does not enforce its documented litCapacity >= litSize + 8 contract — the implemented check is 8 bytes weaker than the API documentation, its own error message, and the introducing commit

Author: bbingoCNCreated Sep 10, 2026Updated Sep 11, 2026

Describe the bug

ZSTD_compressSequencesAndLiterals() does not enforce its own documented litBufCapacity >= litSize + 8 contract: the implemented check in lib/compress/zstd_compress.c:8128 only tests litCapacity < litSize, i.e. it is 8 bytes weaker than the contract stated in three independent places:

  1. API documentationlib/zstd.h line 1700:

    @litBufCapacity must be at least 8 bytes larger than @litSize.

  2. The introducing commitb7a9e69d ("added parameter litCapacity to ZSTD_compressSequencesAndLiterals() to enforce the litCapacity >= litSize+8 condition.")
  3. The check's own error message

    "literals buffer is not large enough: must be at least 8 bytes larger than litSize (risk of read out-of-bound)"

c
/* lib/compress/zstd_compress.c:8126-8130 */
if (litCapacity < litSize) {   /* <-- should be: litCapacity < litSize + 8 */
    RETURN_ERROR(workSpace_tooSmall,
        "literals buffer is not large enough: "
        "must be at least 8 bytes larger than litSize (risk of read out-of-bound)");
}

Calls that violate the documented contract by up to 8 bytes (litSize <= litCapacity < litSize + 8) are silently accepted and produce valid frames, instead of being rejected with the documented ZSTD_error_workSpace_tooSmall.

To Reproduce

Steps to reproduce the behavior:

  1. Clone zstd and build the static library:
bash
git clone https://github.com/facebook/zstd && cd zstd
make -C lib libzstd.a    # or: make -C lib
  1. Save the following as reproducer.c in the repo root:
c
/* reproducer.c — demonstrates that ZSTD_compressSequencesAndLiterals()
 * accepts litCapacity values that violate its documented
 * "at least 8 bytes larger than litSize" contract. */
#define ZSTD_STATIC_LINKING_ONLY
#include "zstd.h"
#include <stdio.h>

int main(void)
{
    /* one explicit-delimiter block: 284 literals + a 16-byte match at
     * offset 8, then 16 trailing literals carried by the delimiter */
    ZSTD_Sequence seqs[2] = {
        { /*offset=*/8, /*litLength=*/284, /*matchLength=*/16, /*rep=*/0 },
        { /*offset=*/0, /*litLength=*/ 16, /*matchLength=*/ 0, /*rep=*/0 },
    };
    unsigned char literals[300 + 8];        /* +8 so the probe itself is memory-safe */
    unsigned char dst[1024];
    const size_t litSize = 300, srcSize = 316;
    size_t i, r;
    ZSTD_CCtx* cctx = ZSTD_createCCtx();

    for (i = 0; i < litSize; i++) literals[i] = (unsigned char)('a' + (i % 26));

    ZSTD_CCtx_setParameter(cctx, ZSTD_c_compressionLevel, 3);
    ZSTD_CCtx_setParameter(cctx, ZSTD_c_blockDelimiters, ZSTD_sf_explicitBlockDelimiters);

    /* documented minimum is litSize + 8; pass litSize + 7 on purpose */
    r = ZSTD_compressSequencesAndLiterals(cctx, dst, sizeof(dst),
                                          seqs, 2, literals, litSize,
                                          litSize + 7, srcSize);
    if (ZSTD_isError(r))
        printf("litCapacity = litSize+7 : rejected (%s) -> contract enforced\n",
               ZSTD_getErrorName(r));
    else
        printf("litCapacity = litSize+7 : ACCEPTED (%zu-byte frame) -> "
               "documented contract NOT enforced\n", r);

    ZSTD_freeCCtx(cctx);
    return 0;
}
  1. Compile and run:
bash
gcc -O2 -Ilib reproducer.c lib/libzstd.a -o reproducer -lpthread
./reproducer
  1. Scroll up on the log to the reproducer's output line:
litCapacity = litSize+7 : ACCEPTED (223-byte frame) -> documented contract NOT enforced
  1. See error: there is no error — the contract-violating call is accepted and produces a valid, round-tripping 223-byte frame. The same happens with litCapacity = litSize (violating by the full 8 bytes). With the one-line fix below, both calls are rejected with ZSTD_error_workSpace_tooSmall while a compliant litCapacity = litSize + 8 call keeps working.

Expected behavior

ZSTD_compressSequencesAndLiterals() should reject litCapacity values below litSize + 8 with ZSTD_error_workSpace_tooSmall, exactly as its documentation (lib/zstd.h:1700), its introducing commit (b7a9e69d) and its own error message all promise:

diff
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -8125,7 +8125,7 @@ ZSTD_compressSequencesAndLiterals(ZSTD_CCtx* cctx,
     /* Transparent initialization stage, same as compressStream2() */
     DEBUGLOG(4, "ZSTD_compressSequencesAndLiterals (dstCapacity=%zu)", dstCapacity);
     assert(cctx != NULL);
-    if (litCapacity < litSize) {
+    if (litCapacity < litSize + 8) {
         RETURN_ERROR(workSpace_tooSmall, "literals buffer is not large enough: must be at least 8 bytes larger than litSize (risk of read out-of-bound)");
     }
     FORWARD_IF_ERROR(ZSTD_CCtx_init_compressStream2(cctx, ZSTD_e_end, decompressedSize), "CCtx initialization failed");

Screenshots and charts

Not applicable — the reproducer output above shows the behavior (ACCEPTED ... contract NOT enforced on current dev; after the one-line fix the same call prints rejected (workSpace buffer is not large enough) -> contract enforced).

Desktop (please complete the following information):

  • OS: Linux (x86-64)
  • Version: Ubuntu 20.04
  • Compiler: gcc 9.4.0
  • Flags: -O2 for the reproducer; the fuller PoC (see Additional context) was additionally run with -g -O1 -fsanitize=address,undefined -fno-sanitize-recover=all
  • Other relevant hardware specs: n/a (no hardware dependency)
  • Build system: make -C lib (static library lib/libzstd.a); library sources compiled directly in the PoC verification as well

Additional context

  • Affected version: current dev (verified byte-identical at lib/compress/zstd_compress.c:8128 on dev @ d9c0c7e2, 2026-09-09). I could not find an existing issue or PR covering this mismatch (closest: open PR #4741 fixes a different validation gap in the same function).
  • Full verification PoC (attached): a three-phase test that (1) confirms a compliant litCapacity = litSize + 8 call succeeds, (2) demonstrates both litSize + 7 and litSize violations are accepted on the unpatched build and rejected with ZSTD_error_workSpace_tooSmall after the one-line fix, and (3) runs an ASan sweep of 360 configurations (30 literal sizes x 3 data patterns x 4 levels, literals buffer of exactly litSize bytes) probing for reads inside the documented-but- unenforced margin [litSize, litSize+8).
  • Honest impact statement: the sweep produced zero ASan reports — literal reads are currently bounded by litSize in the existing encoder paths, so this is a contract-enforcement / defense-in-depth gap rather than a demonstrated out-of-bounds read. The +8 slack exists because the literal encoders historically used 8-byte-wide loads near the tail; the check is the only guard between the caller and ZSTD_entropyCompressSeqStore_internal(), and today it enforces none of the slack that the documentation, the error message and commit b7a9e69d all promise.
  • Suggested title for the issue: "ZSTD_compressSequencesAndLiterals() does not enforce the documented litBufCapacity >= litSize + 8 contract (check is missing the + 8)".