[WARN]: tcp: tls: first record does not look like a TLS handshake when terminating TLS

When I configure Fabio to terminate TLS for a Mosquitto MQTT server, I get the following error:

Dec 14 16:55:59 SERVER_NAME fabio[29480]: 2021/12/14 16:55:59 [WARN]: tcp:  tls: first record does not look like a TLS handshake

My setup is such:

• Fabio runs a the reverse proxy, and accepts MQTTS connections on port 8883 and terminates TLS.
• Fabio then makes unencrypted MQTT requests to the backend MQTT server listening on port 1883 on behalf of the client.
• Consul is obviously in use to keep track of services.
• Nomad + Docker are being used to run services.

The following instructions were derived from the official documentation at https://fabiolb.net/feature/tcp-proxy/.

Steps to reproduce

1. Setup a Mosquitto MQTT server (sudo apt install mosquitto, more in depth tutorial, part 2)
2. Register 2 services in Consul. mqtts with the tags urlprefix-:8883 and proto=tcp, and mqtt with the tags urlprefix-:1883 and proto=tcp
3. Configure Fabio with this:

proxy.addr = :1883;proto=tcp,:8883;proto=tcp;cs=your_valid_certstore;

1. Start Fabio and the mosquitto MQTT server
2. Attempt to connect with mosquitto_sub (sudo apt install mosquitto-clients):

mosquitto_sub -h mqtt.yourdomain.com -p 8883 -t '#' -v
# Or, with authentication setup:
mosquitto_sub -h mqtt.yourdomain.com -p 8883 -t '#' -v -u youruser -P yourpassword

1. Notice how the logs of Fabio contain many rows of this:

Dec 14 17:00:22 SERVER_NAME fabio[29578]: 2021/12/14 17:00:22 [WARN]: tcp:  tls: first record does not look like a TLS handshake

1. Notice how the mosquitto logs look like this:

1639501352: New connection from FABIO_IP_ADDR on port 1883.
1639501352: Socket error on client <unknown>, disconnecting.

1. Now attempt to connect with unencrypted MQTT:

mosquitto_sub -h mqtt.yourdomain.com -p 1883 -t '#' -v
# Or, with authentication setup:
mosquitto_sub -h mqtt.yourdomain.com -p 1883 -t '#' -v -u youruser -P yourpassword

1. Notice how it connects successfully. Try publishing a message:

mosquitto_pub -h mqtt.yourdomain.com -p 8883 -t 'some/topic' -m "a test message" -v
# Or, with authentication setup:
mosquitto_pub -h mqtt.yourdomain.com -p 8883 -t 'some/topic' -m "a test message" -v -u youruser -P yourpassword

Additional system info:

• uname -a: Linux SERVER_NAME 5.10.52-v7l+ #1441 SMP Tue Aug 3 18:11:56 BST 2021 armv7l GNU/Linux
• fabio -v: 1.5.15
• Mosquitto version: 1.5.7