ESF 7.1 Upgrade Changes

This section will list the changes and upgrades made in ESF 7.1.0 and that need to be considered when updating from a previous release. It is possible to perform an upgrade from ESF 7.0.x to ESF 7.1.0 and the upgrader will preserve existing configurations.

Rest API configuration

Configuration v1 REST APIs are deprecated. See Configuration Service for more details.

## Upgrade timeout

The RPM install/upgrade process has configurable guard timeout, the timeout value is set to 15 minutes by default. The purpose of the timeout is to ensure that the RPM installation eventually terminates. In fact, if the timeout triggers, the installation/upgrade will be interrupted and the gateway may be left in a broken state.

The timeout value can be configured by setting the following variable from the shell that will execute the dnf command:

export ESF_INSTALL_TIMEOUT_SECONDS="<value in seconds>"

<value in seconds> must be an integer greater or equal than 900 (15 minutes), if it is not, the default 15 minutes timeout will be used.

It is highly recommended to increase the default value if H2 Database migration needs to be performed.

H2 Database

ESF 7.0.x uses H2 v.1.4.199 whereas ESF 7.1.0 uses H2 v.2.1.210. The upgrader will migrate persisted DB instances to the new H2 version. There are some important changes in this version upgrade that can break custom bundles/applications. See H2Db Service for more details.

❗️

Migration of persisted H2 instances

The ESF upgrade process tries to migrate the persisted H2 instances to the new version. It is possible to disable this behaviour by setting the environment variable from the shell that will execute the dnf upgrade command:

export H2_UPGRADE_SKIP_MIGRATION="true"

It is recommened to set the ESF_INSTALL_TIMEOUT_SECONDS variable to a large value, since the migration process can take hours in case of large database instances.

By default, the upgrade process will try to defragment the persisted database instances to try save some disk space. This process can fail and, as result, the database file can become corrupted. The defragmentation can be disabled by setting:

export H2_UPGRADE_DEFRAG="false"

Depending on the database structure and the available free space on the device, the migration process can fail or produce a corrupted database. In such case, on the next restart, ESF will remove and create an empty database with the same name.

By default, the upgrade process creates a backup of the migrated databases. This can be disabled by setting:

export H2_UPGRADE_KEEP_OLD_DB="false"

To have the migration be successful there must be enough space for the database backup (if option H2_UPGRADE_KEEP_OLD_DB is not false) and for intermediate files, that usually are smaller than the database itself depending on the obtained compression level.

The intermediate files are created in the same location of the database and removed after usage. If there is not enough space in such partition, the path for the temp files can be changed using:

export H2_UPGRADE_TMP_DIR="/mnt/data/"

The use of temporary filesystem paths like /tmp/ requires high attention. Reasons are clearly explained in this Unix StackExchange post.

If H2_UPGRADE_KEEP_OLD_DB is not false the backup is guaranteed given that there is enough free space and the gateway is not shut down/powered off.

We suggest H2 persisted instances that contain critical data to be backuped manually before the upgrade.

Some useful Q&A

Q: Are the databases processed one at a time or in batch?

      A: The upgrade will process the persisted database instances one at a time. So at each iteration there needs to be enough space for the database that is being processed.

Q: How much space will the defragmentation save?

      A: It highly depends on the contents of the database. Note that the migrated database is not defragged, so it may be bigger than the original.

Network configuration changes

In previous ESF versions, the current network configuration is computed from system level configuration files and the /opt/eurotech/esf/user/kuranet.conf configuration file. For this reason it was not possible to configure networking changing only the snapshot.

Even if the network configuration properties are stored in the snapshot in previous versions as well, they are read by the implementation only in case of a configuration update and not on ESF startup.

Starting from ESF 7.1.0, the kuranet.conf file is no longer used, and the network configuration is computed basing on the snapshot only, also on ESF startup.

This allows to seed the initial network configuration by changing only the snapshot_0.xml and perform network configuration changes as regular configuration updates, for example by applying a new snapshot/performing a rollback from ESF Gateway Administration Console and/or using the CONF-V2 request handler and REST APIs.

The properties that describe the network configuration are documented in the Network Configuration section.
The previous versions included more properties that the ones defined in Network Configuration for internal implementation details. These additional properties have been removed in ESF 7.1.

New production security policy (ESF 7.1.1)

Starting from ESF 7.1.1, the security policy for production environment has been updated, as presented in the Security Policy section.
During the upgrade, if ESF is in production mode, the security policy is upgraded as well, forcing the filtering of bundles signed with revoked or not longer valid certificates. The filtering elements are added on top of the security policy, so they are activated firstly.

🚧

If the security policy has been modified by the customer with custom rules, it is strongly suggested to review the policy file after the upgrade to avoid unwanted behaviors.

Backwards incompatible changes

Network configuration

ESF 7.1.0 introduces the following backwards incompatible changes in network configuration:

• The management of modem devices that are not USB devices (e.g. serial modems) is no longer supported.
• Configuring the ppp interface number for a cellular interface is no longer supported; the ppp number is now automatically assigned.
• The properties that describe the modem configuration are now identified by the modem USB port and not by the ppp interface number.
  • For example the previous versions use net.interface.pppX.* properties for describing a cellular interface configuration, ESF 7.1 will use the net.interface.<usb port>.* scheme, <usb port> is the identifier of the USB port on which the modem is connected (e.g. 1-1.3).

In order to allow an upgrade from previous versions, the ESF 7.1.0 NetworkConfigurationService will migrate the configuration properties in the old format to the new one on startup or when its configuration is updated, if the following conditions are met:

• Modem network interface name must be included in net.interfaces property value
• Modem network interface name must be in the ppp[0-9]+ form.
• The following properties must be specified for the network interface:
  • net.interface.pppX.usb.busNumber: modem USB bus number
  • net.interface.pppX.usb.devicePath: modem USB device path

If the conditions are met, the properties for the interface will be migrated by replacing the pppX identifier with the USB port in the format busNumber-devicePath.
The conditions above should hold in case of upgrade from ESF 7.0. The properties will be saved in the new format the first time that a new configuration snapshot is created.

🚧

The new scheme must be used for cellular network configuration updates on ESF 7.1 (for example using ESF Gateway Administration Console and/or the CONF-V2 request handler and REST APIs).

Diagnostics Service

The changes in cellular interface naming induced the following changes in diagnostics service configuration:

• The properties related to the cellular interface monitoring now contain the interface path in their id instead of the PPP interface name
  • The upgrader will migrate existing configuration properties in the new format
• The cellular network interface diagnostic metrics name changes in the same way:
  • cell_sig_level_<ppp_interface_number> is now cell_sig_level_<modem_usb_port>
  • diag_net_rx_<ppp_interface_number> is now diag_net_rx_<modem_usb_port>
  • diag_net_tx_<ppp_interface_number> is now diag_net_tx_<modem_usb_port>
• The cellular signal strength related alerts will now reference the cellular interface by usb port number instead of ppp interface name.