Data Service Connection Monitors

The DataService offers a connection monitor that automatically retries connecting to the remote broker whenever it is not connected and, when enabled, recovers from persistent connection failures by rebooting the gateway through the Watchdog Service.

Automatic reconnection

When Connect Auto-on-startup is enabled (see Data Service Configuration), the DataService tries to connect on startup and, after any disconnection, retries every Connect Retry-interval seconds until the connection succeeds. The first retry after a disconnection is delayed by a short random interval, to avoid many devices reconnecting to the same broker at the same time.

Manually disconnecting the device (for example using the Connect/Disconnect control in the Gateway Administration Console) suspends automatic reconnection: the DataService will not try to reconnect again until it is manually reconnected or the framework restarts.

📘

Note

If a connection schedule is configured, the connection monitor described in this page only applies while the DataService is within a scheduled "connect" window.

Recovery on connection failure

In addition to retrying indefinitely, the DataService can request a gateway reboot if the connection cannot be restored after a configurable number of attempts. This is useful to recover from transient network or driver problems that a simple retry cannot fix.

This feature relies on the Watchdog Service: the DataService only requests a reboot by no longer checking in with the Watchdog Service, so the Watchdog Service must also be enabled for the reboot to actually happen.

To enable it, configure the following two parameters (also shown in the image below):

• Enable Recovery On Connection Failure (enable.recovery.on.connection.failure) - when set to true, the DataService registers itself with the Watchdog Service while trying to reconnect. Every failed connection attempt counts toward the threshold below.
• Connection Recovery Max Failure (connection.recovery.max.failures) - number of consecutive failed connection attempts allowed before the DataService stops checking in with the Watchdog Service, which in turn causes the gateway to be rebooted.

Failures that do not count towards the threshold

Not every failed connection attempt is treated the same way:

• Failures caused by an authentication error, an invalid client ID, or an authorization error reported by the remote broker never count toward Connection Recovery Max Failure and never trigger a reboot on their own. A reboot cannot fix a broker-rejected identity or set of credentials, so in this case the DataService keeps retrying, and keeps checking in with the Watchdog Service, without ever triggering the recovery mechanism.
• Failures caused by the local message store being temporarily unavailable also do not count toward the threshold.

Only connectivity-related failures (for example, the broker being unreachable, or the connection attempt timing out) increment the failure counter that can lead to a reboot.

🚧

To be fully working, this feature needs the Watchdog Service to be enabled. See Watchdog Service.