mosquitto bug causes mqtt session persistence to be non-functional

[reubenmiller]

Describe the bug

Due to this bug in mosquitto, persistent MQTT sessions don't work when using mosquitto > 2.0.11 <= 2.0.18 and the per_listener_settings true setting (which is the default in thin-edge.io).

For thin-edge.io, it means that publishing messages to the local MQTT broker whilst the tedge-mapper-c8y is down will be lost, as the mapper does not receive these messages when it connects to the broker.

Related mosquitto bugs:

• "per_listener_settings true" causes messages for clients to no longer be stored eclipse-mosquitto/mosquitto#2618
• clean session false do not work with per_listener_settings true eclipse-mosquitto/mosquitto#2526

To Reproduce

The behaviour is easy to reproduce

1. Install any mosquitto version from 2.0.12 - 2.0.18 (inclusive)

2. Configure thin-edge.io and connect it to Cumulocity (as this will add the problematic per_listener_settings true mosquitto setting)

3. On the device execute the following script to

   systemctl stop tedge-mapper-c8y
   tedge mqtt pub -q 1 te/device/main///e/test "{\"text\":\"hello $(date +%s)\"}"
   systemctl start tedge-mapper-c8y

   Alternatively you can reproduce the exact mosquitto MQTT broker issue by using mosquitto_sub and mosquitto_pub directly. The following one-liner will print "PASSED" or "FAILED" at the end to show if the test was successful or not.

   mosquitto_sub -h 127.0.0.1 -c -i subscriber1 -t topic1 --qos 1 -W 2; mosquitto_pub -h 127.0.0.1 -t topic1 -m test --qos 1; sleep 1; mosquitto_sub -h 127.0.0.1 -c -i subscriber1 -t topic1 --qos 1 -C 1 -W 2 && echo PASSED || echo FAILED;

Expected behavior

thin-edge.io MQTT messages published with QoS 1 or 2 published when the mapper is not active should be published to the cloud once it starts (and the bridge is online).

Note: this is an upstream bug in mosquitto, so we still have to see how to address this from thin-edge.io.

Screenshots

Environment (please complete the following information):

Property	Value
OS [incl. version]	Debian GNU/Linux 12 (bookworm)
Hardware [incl. revision]	unknown
System-Architecture	Linux tedge 6.8.0-39-generic #39-Ubuntu SMP PREEMPT_DYNAMIC Sat Jul 6 02:50:39 UTC 2024 aarch64 GNU/Linux
thin-edge.io version	tedge 1.3.1
mosquitto version	2.0.18

Additional context