Investigate unnecessary message-layer resets during TLS 1.3 HelloRetryRequest

Summary

TLS 1.3 HRR processing calls mbedtls_ssl_session_reset_msg_layer(ssl, 0); in both:

• ssl_tls13_prepare_hello_retry_request() on the server
• ssl_tls13_postprocess_hrr() on the client

This function performs a message-layer reset, including clearing buffers, transforms, counters, timers, and alert state. It is unclear why this is necessary during an ongoing handshake.

System information

Mbed TLS version (number or commit id): dev, 3.6, 4.1 Operating system and version: N/A Configuration (if not default, please attach mbedtls_config.h): default impacted Compiler and options (if you used a pre-built binary, please indicate how you obtained it): Additional environment information: N/A

Expected behavior

HRR processing should reset only HRR-specific state, such as:

• Transcript hash
• Key share
• Early-data state where applicable

It should preserve unrelated message-layer state.

Actual behavior

Both peers reset the complete message layer while processing HRR. This may discard buffered records or reset state unrelated to HRR.

Steps to reproduce

Run a TLS 1.3 handshake test that involves an HRR

Additional information

Determine whether either call is required. If not, remove both calls and add regression coverage for:

• HRR with compatibility mode enabled and disabled
• Coalesced or already-buffered records following HRR
• HRR after attempted early data
• Mbed TLS and external-peer interoperability

Initial results. Removing both calls passes:

• Mbed TLS client/server HRR
• Mbed TLS client → OpenSSL HRR
• The SSL test suite, including early-data HRR cases
• A build with fatal warnings enabled

This suggests the resets are unnecessary, but more targeted buffered-record testing would be useful before finalizing the change.