ChangelogDownloads
HomeGuidesChangelog
ChangelogDownloadsSupportMulti-service IoT Edge GatewaysEUROTECH
Guides
SupportMulti-service IoT Edge GatewaysEUROTECH

Usage Examples

Suggest Edits

This section provides a list of possible use cases and some notes on suggested configurations for the Log Analytics components.

## Log Manager configuration

The Log Manager component can be used to filter a specific set of messages generated by a LogReader and publish them using a CloudPublisher instance.
Log Manager instances can be created and configured using ESF web console or, as an alternative, it is possible to manually add the component configurations in an XML snapshot file.
The following snippet shows an example configuration for a Log Manager that can be included in an XML snapshot file:

<?xml version="1.0" encoding="UTF-8"?><esf:configurations xmlns:esf="http://eurotech.com/esf/2.0" xmlns:ocd="http://www.osgi.org/xmlns/metatype/v1.2.0">
    <esf:configuration pid="ExampleLogManager">
        <esf:properties>
            <esf:property array="false" encrypted="false" name="rate.limit.enabled" type="Boolean">
                <esf:value>true</esf:value>
            </esf:property>
            <esf:property array="false" encrypted="false" name="CloudPublisher.target" type="String">
                <esf:value>(kura.service.pid=default.log.publisher)</esf:value>
            </esf:property>
            <esf:property array="false" encrypted="false" name="service.factoryPid" type="String">
                <esf:value>com.eurotech.framework.log.manager.LogManager</esf:value>
            </esf:property>
            <esf:property array="true" encrypted="false" name="key.regex" type="String">
                <esf:value>KEY_REGEX_1</esf:value>
                <esf:value>KEY_REGEX_2</esf:value>
                <esf:value/>
                <esf:value/>
                <esf:value/>
            </esf:property>
            <esf:property array="false" encrypted="false" name="kura.service.pid" type="String">
                <esf:value>ExampleLogManager</esf:value>
            </esf:property>
            <esf:property array="false" encrypted="false" name="rate.limit.time.unit" type="String">
                <esf:value>MINUTES</esf:value>
            </esf:property>
            <esf:property array="false" encrypted="false" name="rate.limit.average" type="Long">
                <esf:value>10</esf:value>
            </esf:property>
            <esf:property array="false" encrypted="false" name="LogReader.service.pid" type="String">
                <esf:value>LogReaderJournald</esf:value>
            </esf:property>
            <esf:property array="false" encrypted="false" name="context.cache.size" type="Integer">
                <esf:value>0</esf:value>
            </esf:property>
            <esf:property array="false" encrypted="false" name="rate.limit.burst.size" type="Integer">
                <esf:value>200</esf:value>
            </esf:property>
            <esf:property array="true" encrypted="false" name="value.regex" type="String">
                <esf:value>VALUE_REGEX_1</esf:value>
                <esf:value>VALUE_REGEX_2</esf:value>
                <esf:value/>
                <esf:value/>
                <esf:value/>
            </esf:property>
            <esf:property array="false" encrypted="false" name="duplicate.filter.enabled" type="Boolean">
                <esf:value>true</esf:value>
            </esf:property>
        </esf:properties>
    </esf:configuration>
</esf:configurations>

The snippet above will generate a Log Manager instance named ExampleLogManager. It is important that all Log Manager instances have different names.

The key.regex and value.regex properties allow to define multiple regex based filters. A log entry is selected by the log manager if it matches all of the configured filters.

In the snippet above the following two regexes are defined:

  • Key Regex expressions:
    • KEY_REGEX_1
    • KEY_REGEX_2
  • Value Regex expressions:
    • VALUE_REGEX_1
    • VALUE_REGEX_2

Audit Messages

Audit Messages can be used to detect many different interactions of an external entity with the framework, for example:

  • Web Console logins
  • Configuration updates
  • Request handler calls from the cloud
  • REST api calls

Audit Messages are logged using a slf4j logger with the AuditLogger id. A Log Manager can be configured to collect Audit Messages with the following configuration:

  • Key Regex expressions:
    • ESF_LOGGER
  • Value Regex expressions:
    • AuditLogger

Framework Configuration Change

A change in configuration is usually associated to a set of messages printed by the framework in the logs. An example for a configuration change triggered by a web ui request:

May 14 10:54:44 reliagate-10-14 ESF[21760]: {entrypoint=WebConsole, identity=admin, ip=192.168.0.104, web.path=/admin/denali/component, rpc.method=GwtComponentServiceImpl.updateComponentConfiguration, session.id=node0whhp4w8467df15gdc05u11wok0} ConfigurationService - Success - Get component configuration: org.eclipse.kura.clock.ClockService
May 14 10:54:44 reliagate-10-14 ESF[21760]: The 2-1 interface is not supported
May 14 10:54:44 reliagate-10-14 ESF[21760]: WPA in client mode is not configured
May 14 10:54:44 reliagate-10-14 ESF[21760]: getWifiHostConfig() :: hostapd-p2p0.conf file doesn't exist, will generate default wifiConfig
May 14 10:54:45 reliagate-10-14 ESF[21760]: Loading init configurations from: 1620986533592...
May 14 10:54:45 reliagate-10-14 ESF[21760]: Merging configuration for pid: org.eclipse.kura.clock.ClockService
May 14 10:54:45 reliagate-10-14 ESF[21760]: Updating Configuration of ConfigurableComponent org.eclipse.kura.clock.ClockService ... Done.
May 14 10:54:45 reliagate-10-14 ESF[21760]: Updated...
May 14 10:54:45 reliagate-10-14 ESF[21760]: Writing snapshot - Saving /opt/eurotech/esf/user/snapshots/snapshot_1620989685321.xml...
May 14 10:54:45 reliagate-10-14 ESF[21760]: Writing snapshot - Saving /opt/eurotech/esf/user/snapshots/snapshot_1620989685321.xml... Done.
May 14 10:54:45 reliagate-10-14 ESF[21760]: {entrypoint=WebConsole, identity=admin, ip=192.168.0.104, web.path=/admin/denali/component, rpc.method=GwtComponentServiceImpl.updateComponentConfiguration, session.id=node0whhp4w8467df15gdc05u11wok0} ConfigurationService - Success - Update configurations:  org.eclipse.kura.clock.ClockService
May 14 10:54:45 reliagate-10-14 ESF[21760]: {entrypoint=WebConsole, identity=admin, ip=192.168.0.104, web.path=/admin/denali/component, rpc.method=GwtComponentServiceImpl.updateComponentConfiguration, session.id=node0whhp4w8467df15gdc05u11wok0} ConfigurationService - Success - Update configuration: org.eclipse.kura.clock.ClockService
May 14 10:54:45 reliagate-10-14 ESF[21760]: {entrypoint=WebConsole, identity=admin, ip=192.168.0.104, web.path=/admin/denali/component, rpc.method=GwtComponentServiceImpl.updateComponentConfiguration, session.id=node0whhp4w8467df15gdc05u11wok0} ConfigurationService - Success - Update configuration: org.eclipse.kura.clock.ClockService
May 14 10:54:45 reliagate-10-14 ESF[21760]: {entrypoint=WebConsole, identity=admin, ip=192.168.0.104, web.path=/admin/denali/component, rpc.method=GwtComponentServiceImpl.updateComponentConfiguration, session.id=node0whhp4w8467df15gdc05u11wok0} UI Component - Success - Update component configuration
May 14 10:54:47 reliagate-10-14 ESF[21760]: {entrypoint=WebConsole, identity=admin, ip=192.168.0.104, web.path=/admin/denali/component, rpc.method=GwtComponentServiceImpl.findComponentConfigurations, session.id=node0whhp4w8467df15gdc05u11wok0} ConfigurationService - Success - Get component configurations: (!(|(service.pid=*SystemPropertiesService)(service.pid=*NetworkAdminService)(service.pid=*NetworkConfigurationService)(service.pid=*SslManagerService)(service.pid=*FirewallConfigurationService)(service.pid=*WireGraphService)(objectClass=org.eclipse.kura.wire.WireComponent)(objectClass=org.eclipse.kura.driver.Driver)(kura.ui.service.hide=true)))

Similar messages are reported by the framework when the configuration change for a service is requested from the cloud or from a snapshot update (local or remote) triggered by the Upload and Apply feature.

The ESF ConfigurationService will produce Audit Messages to track most of its operations, including configuration updates. If all Audit Messages are captured it is not necessary to add specific Log Managers for configuration updates.

Otherwise, it is possible to define a specific log manager that captures all ConfigurationService Audit messages with the following configuration:

  • Key Regex expressions:
    • ESF_LOGGER
    • MESSAGE
  • Value Regex expressions:
    • AuditLogger
    • [{].*[}] ConfigurationService -.*

In order to restrict the filter to configuration updates only, the following filter can be used:

  • Key Regex expressions:
    • ESF_LOGGER
    • MESSAGE
  • Value Regex expressions:
    • AuditLogger
    • [{].*[}] ConfigurationService - \w+ - Update configuration:.*

Otherwise, it is possible to add specific log managers that collect the logs generated by the specific component whose configuration has been modified.

The entrypoint propery in log messages can be used to identify the source of the request (e.g. a Web Console interaction, a REST API call). If entrypoint is reported as internal, the configuration change originate from a request of a bundle running inside ESF.

ESF UI User Access

ESF 7 web prints Audit Messages on login, both for certificate and password authentication.
These messages can be captured using the following Log Manager configuration:

  • Key Regex expressions:
    • ESF_LOGGER
    • MESSAGE
  • Value Regex expressions:
    • AuditLogger
    • [{].*[}] UI Login -.*

Fail2ban events

Fail2ban logs on Systemd journal when it detects failed on SSH or ESF Web Console login, or when it bans a specific IP address. It should be possible to collect these log events with the following filter:

  • Key Regex expressions:
    • SYSLOG_IDENTIFIER
  • Value Regex expressions:
    • fail2ban-.*

Linux User Access

In order to identify and report a user access to the system via serial port, SSH via local interface or SSH via VPN, the following configuration can be applied to a LogManager configuration:

  • Key Regex expressions:
    • SYSLOG_FACILITY
    • _AUDIT_TYPE_NAME
  • Value Regex expressions:
    • ^4
    • USER_LOGIN

The proposed configuration allows to identify successful and failed accesses to the system.

## Tamper detection related events

The Diagnostics Service can be configured to log alerts, this includes the alerts generated by AIDE Intrusion Detection and ReliaGATE 10-14 hardware tamper detection component.

The following Log Manager configuration can be used to collect these log events:

  • Key Regex expressions:
    • ESF_LOGGER
    • MESSAGE
  • Value Regex expressions:
    • com[.]eurotech[.]framework[.]diagnostics[.]impl[.]DiagnosticsServiceImpl
    • .*"category":"tamper".*

Update Requests

This section will provide information on how to configure a LogManager to identify (and publish) LogEntries related to Update Requests.

DEPLOY-V2

DEPLOY-V2 updates involve bundle updates or shell script updates. In both cases the interesting log messages should fall in the following two categories:

  • messages logged by the framework (by the DEPLOY-V2 cloudlet implementation).
    • messages logged during the download phase
    • messages logged during deployment package installation
    • messages logged when the verifier script is executed
  • messages logged by the update script or by the updated bundles

To capture messages logged by the DEPLOY-V2 cloudlet implementation, a LogManager instance should be configured with:

  • Key Regex expressions: ESF_LOGGER
  • Value Regex expressions: org[.]eclipse[.]kura[.]core[.]deployment.*

Detecting a DEPLOY-V2 update

In order to publish a single log entry that reports that an update related DEPLOY-V2 request has been received the following filter can be used:

  • Key Regex expressions: MESSAGE
  • Value Regex expressions: Message arrived on topic: \$EDC/[^/]*/[^/]*/DEPLOY-V2/EXEC/(?:download)|(?:install)|(?:uninstall)

CMD-V1

It is possible to capture messages logged by the CMD-V1 cloudlet with the following LogManager configuration:

  • Key Regex expressions: ESF_LOGGER
  • Value Regex expressions: org[.]eclipse[.]kura[.]cloud[.]app[.]command.*

This will publish a log entry when the request is received and another one in case the command exits with a non-zero status. In case the script completes successfully, only the first message will be logged.

Detecting a CMD-V1 update

In order to publish a single log entry that reports that a CMD-V1 request has been received the following filter can be used:

  • Key Regex expressions: MESSAGE
  • Value Regex expressions: Message arrived on topic: \$EDC/[^/]*/[^/]*/CMD-V1/EXEC/command

or

  • Key Regex expressions: MESSAGE
  • Value Regex expressions: EXECuting resource: command

Installing/Upgrading a dp through web UI

Performing this operation produces the same logs described in the Messages logged by a dp installation/upgrade section below plus the messages below by FileServlet

Wed 2020-08-26 14:15:50.746548 UTC [s=ce1453b2d05040ceb41274751f9c2682;i=1140;b=4e95e25d7e8043e6bd15a474cc1f7c5a;m=569663f07;t=5adc874f3b7ef;x=d17bfd85edda7231]
    PRIORITY=6
    SYSLOG_IDENTIFIER=ESF
    MESSAGE=Copying uploaded package file to file: /tmp/com.eurotech.framework.mbus.driver.armv7hf.feature_1.0.0.dp
    ESF_LOGGER=org.eclipse.kura.web.server.servlet.FileServlet
Wed 2020-08-26 14:15:50.769711 UTC [s=ce1453b2d05040ceb41274751f9c2682;i=1141;b=4e95e25d7e8043e6bd15a474cc1f7c5a;m=5696697fb;t=5adc874f410d4;x=6eb7977b89d614fb]
    PRIORITY=6
    SYSLOG_IDENTIFIER=ESF
    ESF_LOGGER=org.eclipse.kura.web.server.servlet.FileServlet
    MESSAGE=Installing package...

Detecting a dp install through the Web UI

Publishing all messages originated by FileServlet might include spurious messages. In order to publish a single log entry that reports that a CMD-V1 request has been received the following filter can be used:

  • Key Regex expressions:
    • MESSAGE
    • ESF_LOGGER
  • Value Regex expressions:
    • Installing package...
    • org[.]eclipse[.]kura[.]web[.]server[.]servlet[.]FileServlet

or

  • Key Regex expressions:
    • MESSAGE
    • ESF_LOGGER
  • Value Regex expressions:
    • Copying uploaded package file to file: .*
    • org[.]eclipse[.]kura[.]web[.]server[.]servlet[.]FileServlet

Detecting a dp install from Eclipse Marketplace

In order to publish a single log entry that reports that an install operation from Eclipse Marketplace has started, the following filter can be used:

  • Key Regex expressions: ESF_LOGGER
  • Value Regex expressions: Installing deployment package from Eclipse Marketplace, URL .*[.][.][.]$

Executing a command through the Web UI

Executing commands from the Web UI from the Device -> Command section do not seem to produce log entries generated by the framework.

Update Script

This section describes how to forward to journald messages produced by an update script triggered by CMD-V1 or DEPLOY-V2 and how to configure a LogManager instance to publish such messages to the cloud.

A way for simplifying this process can be executing the update script as a systemd unit with a well known name. The well known name can be used to configure a LogManager, for example if the update unit is called my-update.service, the following LogManager configuration should work:

  • Key Regex expressions: _SYSTEMD_UNIT
  • Value Regex expressions: my-update.service

Executing the update as a systemd unit has the following advantages regarding log collection:

  • allows to collect the log published by all child processes of the update script
  • allows to collect messages delivered to journald with different transports:
    • stdout
    • sd-journal APIs
      -syslog (assuming that syslog messages reach journald)

The systemd-run command line tool can be used as a simple way to run an update as systemd unit. This system has the advantage that it allows to spawn a process as a transient systemd unit, without the need of creating configuration files.

If the same unit name is used for all update script executions, a single LogManager instance is enough for collecting the generated log entries.

🚧

Note that systemd allows at most a single unit with a given name to exist at a time. This implies that using systemd-run will fail if an execution is started when a previous execution with the same unit name is still running.

🚧

If the --unit argument is not passed to `systemd-run, it will start a unit with an automatically generated name.

As an alternative, instead of relying a well known unit name it is possible to run the update scripts inside a well-known systemd slice.
This can be done by passing the --slice= argument to systemd-run, without specifying the `--unit argument.

If this is done the LogManager filter can be changed as follows, assuming that the `system-ethupdate.slice slice name is used:

  • Key Regex expressions: _SYSTEMD_SLICE
  • Value Regex expressions: system-ethupdate[.]slice

🚧

Note that dashes `- in slice name express parent-child relationships.

This change would allow to have multiple update scripts executing concurrently, the log entries generated by them can be reported by the single LogManager instance shown above

DP installation/upgrade

The messages generated by the framework in case of dp installation/upgrade can be captured with the following LogManager configuration:

  • Key Regex expressions: ESF_LOGGER
  • Value Regex expressions: org[.]eclipse[.]kura[.]deployment[.]agent.*

🚧

This will likely cause the publishing of log entries during system startup, when the dps are installed

Updated about 1 year ago