Configuration

Agent

The Agent configuration section offers configuration options for each Agent object, with one or several groups pertaining to each Agent.

These options are used to configure the settings for file uploads, as well as to define the parameters affecting the groups in a given Agent. The following screenshot shows the different available options for Agent configuration:

Parameters:

• The Output Path is the path where the module will save exported files. This path can either be absolute or relative to the base N3uron folder. If the module does not contain file uploads, the final path will be “outputPath/AgentName/” or alternatively, “/outputPath/AgentName/Pending or Sent or Failed/” will be used for modules containing file uploads. The default value is <null>, which corresponds to /n3uron/Data/DataExporterInstance/. 
• The Include Agent Name parameter specifies whether the Agent Name will be appended to the output path or not. For example, if the Output path is Files (a relative path to /data/DataExporterInstance/) and this setting is set to “True”, the path will be: <program_root>/data/DataExporterInstance/
  Files/AgentName. If this setting is set to “False”, the path will be: <program_root>/data/
  DataExporterInstance/Files/. 
• The File Compression setting specifies how files should be exported. If this setting is set to “ZIP”, files are packed together in a .zip file. If set to “None”, files are exported as loose files without compression. All exported files are tagged with the .tmp extension until they have been fully loaded and taken the desired extension (XML or CSV).
• The Filename string will be used as the filename, appended to the timestamp. For example, if the filename is 1minAggregation and calculations trigger at 13:17:00 on the 14th of June 2018, the resulting filename will be 20180614131700-1minAggregation.zip. If this field is left blank, the filename will simply be the timestamp. Therefore, in this example, the filename would be 20180614131700-no-file-upload-zip.zip. This setting only applies when the File compression setting is set to “ZIP”.
• The File Type parameter specifies the file type to be exported. If this setting is set to “XML”, data will be exported in XML file format as XML elements with attributes. If this setting is set to “CSV”, data is exported in CSV files with comments to denote the group, start and end times, and configurable column separator. 

The File Upload setting determines the protocol used to automatically upload files to a server. The available protocols are: 

• None: Files will be stored locally. 
• SFTP: Files will automatically be uploaded to an SFTP server, with the option of storing a local copy.
• FTP: Files will automatically be uploaded to an FTP server via plain FTP, with the option of storing a local copy.

Internal Scan

The Internal scan menu offers options for controlling how each Agent is executed. Valid default parameters will usually be as follows:

• The Execution rate parameter specifies the module’s check rate, a process used to verify whether groups are in need of calculating aggregations or not, displayed in milliseconds. A lower rate means that checks are done more often (which implies more CPU usage). A higher rate means that checks are carried out less often (which implies higher latency when creating the aggregation file, but less CPU usage). The execution rate does not affect the result of any calculations. The minimum value is 1000 milliseconds.
• The Delay parameter specifies the time period the module will wait for after each interval has finished, in case there are any events that have not yet reached this node due to network inconsistencies. For example, for a delay of 5000 milliseconds with 1 minute aggregation periods, aggregations will start at 00:01:05, 00:02:05, etc., although the aggregation interval will remain as 00:00:00 to 00:01:00. The minimum value is 0 milliseconds (which means there is no delay).

Files Format

The Files format option is used to configure the format of CSV or XML files. Two of these options are available for both file types, although the CSV file type has an additional option that allows users to specify the separator type.

• The Tag naming parameter specifies how the path of each tag will be displayed. If this setting is set to “Relative”, the input tag group will be omitted from the name of the tag, leaving only the relative path from this input tag group. If this setting is set to “Absolute”, the full tag path will be displayed for each tag
• The Path separator setting is used to specify which character should be used to separate the tag path, whether the full path (when using Extended structure) or relative to the input group (when using Compact). By default, the separator is set to “/”. As an example, if the Input Tag Group is “/Plant/BLK001”, the tag in the file would be as follows:
  • Tag in tag model: /Plant/BLK001/Inv001/POWER
  • Compact (Tag path separator is “/”): Inv001/POWER
  • Compact (Tag path separator is “.”): Inv001.POWER
  • Extended (Tag path separator is “/”): Plant/BLK001/Inv001/POWER
  • Extended (Tag path separator is “.”): Plant.BLK001.Inv001.POWER 
• The Prefix option specifies the prefix that will be prepended to the tag path.
• The File Structure (for XML files) setting specifies the file structure for XML files. If this setting is set to “Compact”, events are added as an array of “Event” nodes for each tag. If this setting is set to “Extended”, events are added as an array of “Event” nodes, each containing the tag as an attribute. Examples of both options for the following tags (DCVoltage, DCCurrent, Frequency) can be found in the Appendix. 
• The CSV Separator (for CSV files) setting is used to choose either a comma or semicolon for use as column separators in CSV files.
• The Error Value setting specifies the value that is set to aggregations when they cannot be calculated.

Date Configuration

The Date Configuration menu is used to configure the date in all resulting files.

• The Date Format setting specifies the date format used in the output file. If this setting is set to “ISO8601”, the date will be formatted according to the ISO8601 format (Example: 2019-01-22T10:30 :01Z). If this setting is set to “Unix Epoch”, the date will be saved as the number of milliseconds elapsed since the Epoch (Jan 1 1970, 00:00:00 UTC). When set to “Custom”, the date format will be a custom format string.
• When Date Format is set to “Custom”, Format string specifies the date format that will be used in the resulting file.
• Timezone specifies the timezone used when formatting dates in the resulting file.

Files Management

Each protocol has its own specific options for connections to the server, in addition to common general options for determining how files will be uploaded to servers.

• If File Upload is set to “None”, the Max number of files will be shown in the Files Management menu. This setting specifies the maximum number of files to store in the hard drive. Older files will be deleted whenever this number is exceeded.

SFTP 

When selecting SFTP file upload, the following options will be displayed:

Connection:

• Host: Specifies the host address of the SFTP server.
• Port: Specifies the port used to connect to the host. The default value for SFTP servers is 22. The valid range for this setting is 1 to 65536.
• Username: Specifies the username used for authentication with the server.
• Password: Specifies the password used for authentication with the server. For SFTP servers, this field is not required if authentication is carried out using a private key. However, it can still be used as an alternative authentication method if the private key is rejected.
• Private key: Specifies the private key used for authentication. The key must be in PEM format.
• Passphrase: Specifies the passphrase used to decrypt the private key (if using an encrypted password). It can be left blank if the private key is not password encrypted or if there is no private key.
• Keep-Alive Interval: It is used to determine when a keep-alive packet needs to be sent in order to ensure that the connection does not drop due to inactivity. It is also used to check whether the connection is still active or not. By setting this value to 0, the keep-alive mechanism is automatically disabled, however, this is not recommended.
• Retry Interval: Specifies the wait time after a connection drops before attempting a reconnection.

SFTP File Upload Options 

The File Upload Options section is included within the Files Management menu. This section is used to configure connections to FTP or SFTP servers. Most of these options are available in both the SFTP and FTP server settings. They are used to configure various aspects relating to how files are sent, or whether or not sent files should be kept.

File Upload Options: 

• Remote path: is used to specify the path where files will be uploaded to. This path can either be relative to the home directory of the SFTP server, or absolute. Users must include write and rename permissions in this directory (which must be created before this step).
• Pending Files Local Buffer: specifies the maximum number of files to keep if file uploads to the SFTP server have failed. Once the maximum number of pending files has been reached, older files will be deleted. The minimum value is 1.
• Send Order: Specifies the order for file uploads to the server. If this setting is set to “Oldest first”, the module will send the oldest files first. If this setting is set to “Newest first”, the module will send the newest files first.
• File rename delay: Specifies a delay after uploading the temporary file (with .tmp extension) and before renaming it with the target extension. Set to 0 to disable the delay.
• Keep Files After Upload: Specifies whether or not to store files locally once uploaded to the server.
• Sent Files Local Buffer: Specifies the maximum number of successfully sent files to be kept locally before older files are deleted. The minimum value is 1.
• Delay Between File Uploads: This option specifies the time (displayed in milliseconds) to wait before starting the next file upload after a successful transfer has been completed. For example, this can be used if the server is only able to handle one file per second (in this case, the delay would be 1000 ms). The minimum value is 0, which means that the next file will be sent as soon as possible after the previous upload has been completed.
• Keep Files After Failed Upload: This setting specifies whether or not to store files locally after failing to upload them to the server.
• Failed Files Local Buffer: This option specifies the maximum number of failed files to keep locally before older files are deleted. The minimum value is 1.

FTP

When selecting FTP file upload, the following options will be displayed:

Connection:

• Host: Specifies the host address of the FTP server
• Port: Specifies the port used to connect to the host. The default value for FTP servers is 21. The valid range for this setting is 1 to 65536.
• Username: Specifies the username used for authentication with the server.
• Password: Specifies the password used for authentication with the server.
• Keep-Alive interval: It is used to determine when a keep-alive packet needs to be sent in order to ensure that the connection does not drop due to inactivity. It is also used to check whether the connection is still active or not. By setting this value to 0, the keep-alive mechanism will be disabled. However, this is not recommended.
• Retry Interval: This setting specifies the wait time before attempting to reconnect after a connection drops.
• Command Timeout: This setting defines how long the client will wait for a reply from the FTP server. If no response is received within this period, the client will close the connection and attempt to reconnect.

FTP File Upload Options

The File Upload Options section can be accessed from the Files Management menu. This section is used to configure connections to FTP or SFTP servers. Most of these options are shared across both SFTP and FTP server settings. They are used to configure various aspects relating to how files will be sent or whether or not files should be stored locally.

File Upload Options:

• Remote path: Specifies the destination on the server where files will be sent. This can be a path relative to the server's home directory or an absolute path.
• Pending Files Local Buffer: Defines the maximum number of files to retain if uploads to the SFTP server fail. When this limit is reached, older files are automatically deleted. The minimum value is 1.
• Send Order: Determines the order of file uploads to the server. If this setting is set to “Oldest first”, the module will send the oldest files first. If set to “Newest first”, the module will send the newest files first.
• File rename delay: Specifies a delay after uploading the temporary file (with .tmp extension) and before renaming it with the target extension. Set to 0 to disable the delay.
• Keep Files After Upload: This setting specifies whether or not to store files locally once uploaded to the server.
• Sent Files Local Buffer: Specifies the maximum number of successfully sent files to keep locally before older files are deleted. The minimum value is 1.
• Delay Between File Uploads: Sets the wait time (in milliseconds) after a file is successfully sent. For example, this can be used if the server is only able to handle one file per second (in this case, the delay would be 1000ms). The minimum value is 0, which means that the next file will be sent as soon as possible after the current file finishes.
• Upload Retries: Determines the number of attempts to upload a file to the FTP server before moving it to the failed folder or deleting it.
• Keep Files After Failed Upload: Specifies whether or not to store files locally after failing to be uploaded to the server.
• Failed Files Local Buffer: Specifies the maximum number of failed files to store locally before older files are deleted. The minimum value is 1.

Group Configuration

Each Agent can have one or several groups. Each group corresponds to an XML or CSV file in the .zip. 

Available Group options can be seen in the following screenshot:

Parameters:

• Filename: It is the name of the XML or CSV file created by this group. In this example, assuming this is a CSV file, the file would be named File1.csv.
• Format: Specifies the format for tag values in the resulting file. Available formats include Integer, Decimal (with precision options), String, and Custom value format expressions:
  • Boolean as string (true-false): %s
  • Boolean as number (1-0): %d
  • Integer: %i
  • Binary: %b. Example: number 6 is presented as 110
  • Floating-point: %f. With 2 decimal places: %.2f
  • String: %s
    Note:

    This feature is available from N3uron version 1.21.10.
• Period: Specifies the group execution period, displayed in milliseconds. The minimum value is 10000 milliseconds .
• Mode: Specifies whether or not the data will be aggregated when uploaded to the server. When this setting is set to “Aggregated”, the Mode options menu will appear.
• Input Tag Group: It is the group from the tag model that will be used as a base path for all tags forming part of this group. Tag filters only apply to tags contained in the group that has been defined in this setting.

Whenever Mode is set to “Aggregated”, the Mode Options menu will become available. This is used to select which of the following values should be calculated for this group: 

• Average: Time-weighted average of the values received in the specified interval.
• Minimum: Lowest value received in the interval.
• Maximum: Highest value received in the interval.
• First: The first value that occurs in the interval. This is not the first value received, but the value present at the start of the interval (typically, this would be the last value from the last interval).
• Last: Last value received in the interval.
• Count: Total number of events received in the interval. This includes both good and bad-quality events, as well as good-quality events with a null value.
• Good: Ratio of good quality events to total received events, expressed as a decimal. 1 means all events had good quality, 0 means all events had bad quality.

The Tag Filters setting allows users to define rules based on Regular Expressions, which is used to select which tags will be included or excluded from the result. Tag filters options include:

• Mode: Specifies whether the filter will include or exclude tags from the result.
• RegExp Pattern: Specifies the regular expression used to select which tags should be included or excluded in the exported file. If set to default, all tags contained within the input tag group will be exported.