db.pl sync-files does not write compression metadata (compression, uncompressedBits, packetPosEncoding) for compressed PCAPs
Author: rht-jbittnerCreated Sep 3, 2026Updated Sep 7, 2026
When using db.pl sync-files to restore missing PCAP records in the arkime_files index, the command creates Elasticsearch documents that omit all compression metadata (compression, uncompressedBits, and packetPosEncoding).
In environments using simpleCompression (such as zstd or gzip), Arkime Viewer relies on these fields to calculate bit-shifted packet offsets. Because db.pl sync-files writes documents without these fields, Viewer treats compressed files as uncompressed raw PCAP files, resulting in:
- Header Errors: ERROR - Couldn't open file Error: Corrupt PCAP header (Viewer reads compressed magic bytes like 0x28B52FFD instead of PCAP magic numbers).
- Packet Offset Failures: ERROR - couldn't read packet at 64333189260922 (Viewer attempts to interpret a bit-shifted block address as a raw 64-terabyte byte offset in an uncompressed file).
https://github.com/arkime/arkime/blob/e05bd95de562d8ab9526cf602b68afe4d1867e61/db/db.pl#L9402
Source: arkime/arkime