--- title: "Data Importer | Configuration | N3uron KB V.1.20" slug: "data-importer-configuration" description: "Data Importer is a N3uron module designed to import data from CSV files containing aggregated data. This module allows data to be automatically imported…" updated: 2021-12-16T09:43:05Z published: 2021-12-16T09:43:06Z canonical: "docs.n3uron.com/data-importer-configuration" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.n3uron.com/llms.txt > Use this file to discover all available pages before exploring further. # Configuration ## The configuration section is divided into configuration settings for the API, Logger, Agent objects and Tags. ## API Configuration API parameters define the communication settings between the module and **N3uron**'s bootstrap. ![](https://cdn.document360.io/54093ab5-6b22-4542-a265-04377931f11a/Images/Documentation/03-API-parameters.png) Figure 3: API parameters The API configuration settings include the following parameters: - **Event rate:** Determines the Event exchange rate between bootstrap and the module, displayed in milliseconds. The default value is 1.000ms with a minimum value of 100ms. - **Timeout:** Maximum wait time for a response from bootstrap, displayed in milliseconds. The minimum value is 100ms and the default value is 5.000ms. For nodes dealing with a large number of tags, this time can be increased to avoid timeouts when communicating with bootstrap. - **Keep-alive period:** Time between keep-alive checks with bootstrap, displayed in milliseconds. Set to 0 to disable keep-alive checks for this module. The default value is 60.000ms. When the module fails to respond to keep-alive requests, bootstrap will automatically restart the module process. Once the API parameters have been set, the configuration settings must be saved and the module needs to be restarted in order to apply the new values. ## Logger configuration These parameters define how the module logs activity. ![](https://cdn.document360.io/54093ab5-6b22-4542-a265-04377931f11a/Images/Documentation/04-Logger-parameters.png) Figure 4: Logger parameters Parameters: - **Enabled:** When enabled, the module sends logs to a text file located at /n3uron/log/. The default value is set to enabled. - **Level:** Level of detail provided in the information sent to the log file. - **Error:** Only logs errors - **Warning:** Logs errors and warnings. - **Info (default):** Logs errors, warnings, and general information messages. - **Debug:** Logs most module activity for debugging and troubleshooting purposes. - **Trace:** Logs full module activity. This mode may log large amounts of data and for this reason, is not recommended for production environments. - **Days to keep:** Log files are automatically deleted when they become older than the number of days specified in this section. The default value is 7 days. ## Agent Agent configuration defines the settings for importing files. The following screenshot shows the different available options for Agent configuration: ![](https://cdn.document360.io/54093ab5-6b22-4542-a265-04377931f11a/Images/Documentation/05-Creating-new-agent.png) Figure 5: Creating a new Agent ![](https://cdn.document360.io/54093ab5-6b22-4542-a265-04377931f11a/Images/Documentation/06-Agent-settings.png) Figure 6: Agent settings Parameters: - The **Enable** parameter enables or disables the **DataImporter** agent. - The **Scan Rate** specifies the check rate for the input folder, displayed in milliseconds. The minimum value is 100ms. - The **Path** parameter specifies the input directory where files are located. This path can either be a relative path, in which case it will be relative to the data folder of this **DataImporter** instance, or an absolute path. - The **Parser** specifies the parser used to parse the contents of the imported files. ## File Management The **File Management** menu contains the following settings: - The **Loaded Path** setting specifies the destination directory for files that have been imported correctly. This path can either be a relative path, in which case it will be relative to the Loaded folder (existing within the data folder of this **DataImporter** instance), or an absolute path. - The **Maximum Loaded Files** setting specifies the maximum number of loaded files to store in the loaded folder (existing within the data folder of this **DataImporter** instance). Set to 0 to immediately delete loaded files. - The **Failed Path** parameter specifies the destination directory for files that could not be imported correctly. This path can either be a relative path, in which case it will be relative to the Failed folder (existing within the data folder of this **DataImporter** instance), or an absolute path. - The **Maximum Failed Files** setting specifies the maximum number of failed files to store in the loaded directory (existing within the data folder of this **DataImporter** instance). Set to 0 to immediately delete loaded files. ## Parser The **Parser** is used to configure the parser in order to properly parse files for import. This menu contains the following settings: - The **Separator** setting specifies the column separator used in the CSV file. - The **Comment Character** setting specifies the character to be used for comments in the CSV file. Any lines beginning with this character are ignored by the parser. - The **Row Skip** setting specifies how many lines the parser will skip from the beginning of the file, not counting blank lines. This can be used to skip the first line (or lines) containing metadata or the header. Set to 0 to disable it and parse the CSV from the first line. - The **Mode** setting specifies how data is interpreted from the CSV file. **Time Stamp** settings: - The **Timezone** setting specifies the time zone used to parse the data timestamp in the CSV file. The available options are: - **UTC:** Timestamp will use the UTC timezone or GMT. - **Server:** Timestamp will use the timezone of the server where the module is located. - **Custom:** Timestamp will use the timezone specified in “Custom Timezone”. - The **Custom Timezone** parameter specifies the desired timezone format when the Timezone setting is set to “custom”. - The **Format** setting specifies how the timestamp will be parsed. The following options are available: - **Auto-detect:** attempts to parse the timestamp using several standard formats, such as ISO 8601 or RFC 2822. If the format can’t be parsed, the corresponding tag will be set to bad quality. - **Custom Format:** uses a format string to parse the timestamp. If the parser fails, the corresponding tag will be set to bad quality. - The **Format String** setting specifies the format of the timestamp string used when the format setting is set to Custom. **Columns:** - The **Tag** parameter specifies the index of the column containing the tag path. The minimum value is 1, which refers to the first column of the CSV. - The **Value** setting specifies the index of the column containing the tag value. The minimum value is 1, which refers to the first column of the CSV. - The **Quality** setting specifies the index of the column containing the tag quality. This value can be set to 0 (or Disabled), which means that the CSV file does not contain a value for the tag quality and all values will be imported with a quality of 192 (Good). Otherwise, it can be set to any value higher than 0. - The **Timestamp** setting specifies the index of the column containing the timestamp. This value can be set to 0 (or Disabled), which means that the CSV file does not contain a timestamp for the different values. This will prompt **DataImporter** to import all values with the current date and time as their timestamp. Otherwise, it can be set to any value higher than 0. ## Files Filter The **Files Filter** contains the following settings: - The **Pattern** setting specifies the regular expression used to filter which files from the directory are parsed with the **DataImporter** agent. If left empty, any filename will match the filter. - The **Ignore Flag** setting specifies the ignore case flag for the regular expression. ## Tag configuration **Agent** configuration defines the settings used for importing files. After configuring the **Agent**, users will be able to create and configure any **Tags** associated to the data received from imported files, as seen in the following example: ![](https://cdn.document360.io/54093ab5-6b22-4542-a265-04377931f11a/Images/Documentation/07-Tag-options.png) Figure 7: Tag options Source contains the following settings: - The **Enabled** setting enables reading of tags from their source. - The **Module Type** parameter specifies the source module type that will be used to read the tag. - The **Module Name** parameter specifies the source module name that will be used to read the tag. Config contains the following settings: - The **Agent** setting specifies the agent used to provide values to this tag. - The **Data Address** parameter specifies the address of the data for this tag. For CSV parsers, this option is used to specify the name of this tag within the CSV file.