#680·twemproxy

Murmur implementation is not endian/alignment-neutral

Author: paravoidCreated Feb 11, 2023Updated Feb 11, 2023

Describe the bug src/hashkit/nc_murmur.c contains an implementation of MurmurHash2, which is broken on big-endian and/or non-4-byte aligned architectures. I observed this issue on s390x.

This is well-known issue with this implementation, and several others exist and shipped by dozens of projects, including the lightly modified "MurmurHashNeutral2" one shipped by several projects (search for it on GitHub or codesearch.debian.net). It's slower, so typically it's guarded by an #ifdef to select between the original copy on x86_64/i386.

To Reproduce Steps to reproduce the behavior:

  1. Build on s390x
  2. Run "make check"
  3. Observe the failure:
[2023-02-11 17:16:34.307] nc_memcache.c:388 parsed bad req 171 of type 6 with key prefix 'xxxxxxxxxxxxxxxx...' and length 251 that exceeds maximum key length
[2023-02-11 17:16:34.307] nc_memcache.c:394 parsed bad req 174 of type 1 with an empty key
[2023-02-11 17:16:34.307] nc_memcache.c:394 parsed bad req 180 of type 3 with an empty key
FAIL Expected 4142305122, got 3703547805 (should have expected murmur hash for key "apple")

Expected behavior Test suite to pass.

Additional context

MurmurHash2 has been replaced by MurmurHash3. While exploring this issue it may be worthwhile to look into an even more complete replacement.