Compiler Optimization Inappropriately Removes Sensitive Memory Clearing

1. Summary

We identified two security-sensitive memory-clearing operations in MTProxy that are inappropriately eliminated by compiler optimization.

Both operations are intended to erase cryptographic secrets after use. However, because the cleared memory is not subsequently accessed, the compiler treats these writes as dead stores and removes them during optimization. Although this preserves the program's functional behavior, it violates the security intent of the original code and leaves sensitive data in process memory.

In our runtime test, 248 of the 256 bytes of the DH private exponent can still be recovered after the corresponding heap block is freed.

2. Sites

Site 1 — free_crypto_temp() (net/net-crypto-aes.c:329-333)

void free_crypto_temp (void *crypto, int len) {
  memset (crypto, 0, len);          /* removed by the compiler */
  free (crypto);
  MODULE_STAT->allocated_aes_crypto_temp --;
}

crypto is a struct crypto_temp_dh_params: a[256], the DH private exponent, plus the shared secret computed into the same block.

Site 2 — aes_create_keys() (net/net-crypto-aes.c:257-307)

unsigned char str[16+16+4+4+2+6+4+2+MAX_PWD_LEN+16+16+4+16*2 + 256];
...
memcpy (str + 54, key->secret, key->secret_len);   /* line 257 */
...
md5 (str + 1, str_len - 1, R->write_key);          /* keys and IVs, for both */
sha1 (str, str_len, R->write_key + 12);            /* directions, are derived */
md5 (str + 2, str_len - 2, R->write_iv);           /* from str */
...
memset (str, 0, str_len);                          /* line 307, removed */

str is 634 bytes on the stack and holds a copy of key->secret — the credential the client authenticated with — plus both nonces and everything derived from them.

Why the compiler removes them. After free, or once the frame is gone, no defined use can observe the store, so dead-store elimination drops it.

3. Evidence and how to reproduce

We build this project with -O0, -O1, -O2 and -O3 , and compare the generated assembly code . The "memset" is removed at -O1 and upwards, in both functions.

git clone https://github.com/TelegramMessenger/MTProxy.git ~/mtproxy
export SOURCE=~/mtproxy

# clearing instructions left in each function: a call to memset, or an
# inlined rep stos
bash whole-build/asm-diff.sh
#   net-crypto-aes.c  free_crypto_temp   -O0: 1   -O1/-O2/-O3: 0
#   net-crypto-aes.c  aes_create_keys    -O0: 1   -O1/-O2/-O3: 0
#   net-crypto-dh.c   dh_second_round    -O0: 1   -O1/-O2/-O3: 1   (control)

# builds both trees, then runs the probe against each
bash exploit-proof/run.sh
#   expected: pristine -> DH private exponent bytes still readable: 248 / 256
#             patched  -> the block comes back zeroed

free_crypto_temp() is the clearest case, the assembly code generated with different optimization is shown below:

MTProxy-repro.zip

MTProxy-repro.zip

MTProxy-repro.zip

       -O0                        -O2
    call  memset@PLT           call  free@PLT
    call  free@PLT             subl  $1, 4(%rax)
    subl  $1, 4(%rax)          ret
    ret

4. Fix and validation

Replace the ordinary memset with a clear the compiler may not remove, kept in a small header next to the file:

/* common/secure-zero.h */
static inline void
secure_zero (void *p, size_t n) {
  memset (p, 0, n);
  __asm__ __volatile__ ("" : : "r" (p) : "memory");
}

-  memset (str, 0, str_len);       /* net-crypto-aes.c:307 */
-  memset (crypto, 0, len);        /* net-crypto-aes.c:329 */
+  secure_zero (str, str_len);
+  secure_zero (crypto, len);

After a rebuild the clearing is present in the object and in the linked binary, and the same probe no longer recovers the exponent: the block is cleared before it is released. Where the platform provides explicit_bzero or memset_s those are preferable; the inline version above is the portable fallback and is what we verified. Since the unfixed code has no barrier-based clear at all, the same helper is worth using for every sensitive clear.

5. Attachments

MTProxy-repro-minimal.zip — only the code needed to reproduce the two sites, and the fix.

Rating. Deposit into the heap is remotely triggerable — a client only has to start a handshake; reading it back requires access to the process's memory, and we found no path that hands the freed block to a client. We therefore rate it CVSS 3.1 5.9 Medium (AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:N), High if such a path exists. The credential copy in str is long-lived: where one secret is shared by many users, it is effectively the proxy's credential.

MTProxy-repro.zip