#1779·lz4

Wrong or misleading piece of docs about the "synchronized mode" of streaming decompression

Author: michoechoCreated Aug 1, 2026Updated Aug 1, 2026

Describe the bug

There seems to be an error in the documented conditions for the "synchronized mode" of the streaming API.

The comment for LZ4_decompress_safe_continue in lz4.h says:

 *  Special : if decompression side sets a ring buffer, it must respect one of the following conditions :
[...]
 *  - Synchronized mode :
 *    Decompression buffer size is _exactly_ the same as compression buffer size,
 *    and follows exactly same update rule (block boundaries at same positions),
 *    and decoding function is provided with exact decompressed size of each block (exception for last block of the stream),
 *    _then_ decoding & encoding ring buffer can have any size, including small ones ( < 64 KB).

My issue is with the "exception for last block of the stream" part. My interpretation of this comment was: "for the last block, it's okay to pass a dstCapacity greater than the actual decompressed size". But apparently that's wrong. If dstCapacity isn't exact, then the decoding of a sequence within a block might overrun (with the optimized copy operations) some bytes (from the previous ring rotation) that are referenced by the next sequence in the same block, which corrupts the result. It doesn't matter if this is the last block or not. But the comment suggests otherwise.

Expected behavior

The documentation should make it clear that, when using "synchronized mode", LZ4_decompress_safe_continue needs to receive the exact decompressed size of each block, even if it's the last block.

To Reproduce

Here's an example of what I mean:

c
/*
 * lz4.h documents synchronized ring buffer mode as:
 *
 *  - Synchronized mode :
 *    Decompression buffer size is _exactly_ the same as compression buffer size,
 *    and follows exactly same update rule (block boundaries at same positions),
 *    and decoding function is provided with exact decompressed size of each block (exception for last block of the stream),
 *    _then_ decoding & encoding ring buffer can have any size, including small ones ( < 64 KB).
 *
 * But actually there is no "exception for last block".
 * This program shows what I mean.
 *
 * cc -std=c99 -O2 lz4_last_block_ring_reproducer.c -llz4 -o repro && ./repro
 *
 * Tested against lz4 1.10.0.
 */

#include <lz4.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define RING      80
#define MAX_BLOCK (RING / 2)            /* 40 */
#define NBLOCKS   3

/* One 105 byte message, compressed from a 80-byte ring buffer, in 3 blocks:
 *
 *   block 1, 40 bytes -> ring[ 0..40)
 *   block 2, 40 bytes -> ring[40..80)
 *   block 3, 25 bytes -> ring[ 0..25)
 */
static const char message[] =
        "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!?#$"
        "jklmnopqrstuvwxyz0987654321@#$%^&*()_+<>"
        "abcdefghabcdeZ0123wxyz-+*";
static const int block_sizes[NBLOCKS] = { 40, 40, 25 };

/* The compressor emits three sequences for block 3:
 *
 *     81 'a'..'h' 08 00     8 literals, match at op=8  len=5 offset=8
 *                           (in-block, "abcde")
 *     01 44 00              0 literals, match at op=13 len=5 offset=68
 *                           (reaches back past ring[0] into ring[25..30),
 *                           "Z0123")
 *     70 'w'..'*'           7 literals
 *
 * If we decompress this block with an output size sufficiently bigger than the actual output size,
 * then ring[25] gets overwritten during the decoding of the first sequence,
 * which causes a wrong decompression result for the second sequence. */

int main(void) {
    char enc_ring[RING], dec_ring[RING];
    char cbuf[NBLOCKS][LZ4_COMPRESSBOUND(MAX_BLOCK)];
    int csize[NBLOCKS];

    LZ4_stream_t cstream;
    LZ4_initStream(&cstream, sizeof(cstream));
    int in_off = 0, ring_off = 0;
    for (int b = 0; b < NBLOCKS; b++) {
        if (ring_off + MAX_BLOCK > RING) {
            ring_off = 0;                       /* the ring update rule */
        }
        memcpy(enc_ring + ring_off, message + in_off, block_sizes[b]);
        csize[b] = LZ4_compress_fast_continue(&cstream, enc_ring + ring_off,
                cbuf[b], block_sizes[b], (int)sizeof(cbuf[b]), 1);
        if (csize[b] <= 0) {
            printf("compression failed\n");
            return 2;
        }
        in_off += block_sizes[b];
        ring_off += block_sizes[b];
    }

    LZ4_streamDecode_t dstream;
    LZ4_setStreamDecode(&dstream, NULL, 0);
    int n = 0;
    ring_off = 0;
    for (int b = 0; b < NBLOCKS; b++) {
        if (ring_off + MAX_BLOCK > RING) {
            ring_off = 0;
        }
#if 0
        // Works fine.
        int block_size = block_sizes[b];
#else
        // The "Synchronized mode" comment suggests we can do this, but this is wrong and gives a corrupted result.
        int block_size = MAX_BLOCK;
#endif
        n = LZ4_decompress_safe_continue(&dstream, cbuf[b], dec_ring + ring_off,
                csize[b], block_size);
        ring_off += block_size;
    }

    const char *last_block = message + block_sizes[0] + block_sizes[1];
    printf("last block decoded to %d bytes: \"%.*s\"\n", n, n, dec_ring);
    printf("                      expected: \"%s\"\n", last_block);
    return memcmp(dec_ring, last_block, block_sizes[NBLOCKS - 1]) == 0 ? EXIT_SUCCESS : EXIT_FAILURE;
}

System (please complete the following information):

  • OS: x86-64 Linux
  • Compiler: GCC 15

Additional context

I had an application using "synchronized mode" where all blocks except the last one had decompressed size K, and the last block had decompressed size TOTAL % K. Because of that comment, I figured I only need to store post-compression sizes, and I don't need to store any decompressed sizes, because I can just decompress all blocks, including the last one, with dstCapacity = K. And that eventually turned out to give wrong results sometimes.