Aurora monitor can remain permanently blind after a failover when use_ssl=1

Summary

monitor_AWS_Aurora_thread_HG() does not clear OpenSSL's per-thread error queue after a failed SSL operation. Unlike the newer asynchronous monitor handlers, it runs a hand-written asynchronous loop in a long-lived dedicated thread. Consequently, an SSL error during an Aurora check can poison that thread and cause all later Aurora checks to fail with:

Lost connection to server during query

The failure can persist after the Aurora cluster, network, credentials, and the monitor connection itself have recovered. Since a failed check produces no topology data, the affected ProxySQL instance stops evaluating the Aurora writer/reader topology and can retain a former, read-only writer in the writer hostgroup.

We observed this with eight identically configured ProxySQL pods monitoring the same Aurora MySQL cluster: after failover, some pods converged while others remained blind for more than two hours.

Environment

• ProxySQL 3.0.9

Root cause in source

lib/MySQL_Monitor.cpp defines the following helper:

#define MYSQL_OPENSSL_ERROR_CLEAR(_mysql) if (_mysql->options.use_ssl == 1) {\
	ERR_clear_error();\
}

The adjacent call-site comment explains why it exists:

In the case of SSL-based connection to the backend server, any connection-related errors will cause all subsequent calls to the backend servers to fail. This is because OpenSSL maintains a thread-based error queue that must be cleared after an error occurs to ensure the next call executes successfully.

In this revision the helper is used only by the state-machine handlers:

monitor_AWS_Aurora_thread_HG() starts at line 5963 and does not use those handlers. It implements its own mysql_query_start() / mysql_store_result_start() / wait_for_mysql() loop and has no ERR_clear_error() call:

• query failure (if (mmsd->interr), line 6268);
• either check_timeout_ms exit;
• failed create_new_connection(), including a failed TLS connection where mmsd->mysql is already NULL.

Thus adding the existing macro only to the interr branch would be insufficient: it dereferences MYSQL * and cannot protect failed connection creation or timeout exits.