mbedtls_sha256_update produces incorrect result if ilen >= 0x100000000

Summary

mbedtls_sha256_update declares its ilen argument as size_t, so it seems reasonable to expect the function to work correctly on a 64-bit system even if ilen >= 0x100000000. But instead it computes the wrong SHA value in that case. The bug is that the 32-bit arithmetic done on ctx->total[] ignores the high 32 bits of ilen, so although all the bytes of the input are processed, the total byte count has the wrong value when folded in at the end of the computation.

System information

Mbed TLS version (number or commit id): Seen in all branches I've looked at, including development at 7ad26d3b Operating system and version: Rocky Linux 9.8 Configuration (if not default, please attach mbedtls_config.h): default Compiler and options (if you used a pre-built binary, please indicate how you obtained it): gcc (GCC) 11.5.0 20240719 (Red Hat 11.5.0-14) Additional environment information:

Expected behavior

Correct SHA

Actual behavior

Incorrect SHA

Steps to reproduce

Call mbedtls_sha256_update with ilen >= 0x100000000

Additional information

Here's a suggested point fix. It's designed to work on systems where size_t is either 64 or 32 bits, avoiding undefined behavior on 32-bit systems, but I only tested on a 64-bit system.

diff --git a/drivers/builtin/src/sha256.c b/drivers/builtin/src/sha256.c
index 3eb38dd1a..008f15176 100644
--- a/drivers/builtin/src/sha256.c
+++ b/drivers/builtin/src/sha256.c
@@ -642,6 +642,10 @@ int mbedtls_sha256_update(mbedtls_sha256_context *ctx,
         ctx->total[1]++;
     }
 
+#if SIZE_MAX > 0xFFFFFFFF
+    ctx->total[1] += ilen >> 32;
+#endif
+
     if (left && ilen >= fill) {
         memcpy((void *) (ctx->buffer + left), input, fill);

Trivia: I happened to hit this when working towards updating our mbedTLS version to bring in additional functions and eventually redoing FIPS certification. The current FIPS demo server provides a couple of test cases of 4GB and larger (by giving instructions on how to generate them). The acvpparser-based test harness we're using happens to always generate the test cases in place in memory and make one huge call into mbedTLS, rather than streaming the data in through multiple smaller calls as it's generated.