Introduction

What is SparkPipe

SparkPipe is a cutting-edge connectivity solution designed for IIoT and DataOps that seamlessly bridges operational data at the edge with advanced data processing in the cloud. This service integrates operational data into various data-driven applications, such as stream processing, advanced analytics, machine learning, and predictive maintenance. By leveraging open protocols and standardized data models, SparkPipe ensures the efficient and effective use of data from industrial assets and systems.

SparkPipe is a cloud-native service that utilizes the MQTT Sparkplug v3 specification to securely connect to any MQTT 3.1 compatible broker, capturing all the events published by any Sparkplug-enabled edge node, and routing this data to cloud-based services and applications.

Designed to be highly reliable, secure by default, easy to configure, and simple to operate, SparkPipe is a no-code solution that integrates natively with AWS deployment systems, security features, and access mechanisms, making it accessible to a broad range of users.

SparkPipe transforms raw OT data into actionable insights, driving operational efficiency and unlocking the full potential of your Industrial IoT and DataOps initiatives.

Connectors:

• Kafka: Simplifies the use of Kafka for IIoT by integrating MQTT Sparkplug v3 messages into the Kafka messaging flow. It publishes all events to Kafka clusters, such as Amazon MSK or Confluent Cloud, using an easy-to-parse JSON representation of the Sparkplug payload. It supports various authentication methods including SASL/PLAIN, SASL/SCRAM, mTLS, and AWS IAM. Additionally, it offers data compression to reduce bandwidth and disk usage, and topic partitioning to scale throughput.

• Stdout: Provided for testing purposes, it logs the Sparkplug metrics to the console’s standard output using an easy-to-read JSON format that represents the Sparkplug payload.