IMPORTANT NOTE
THIS PLUGIN WAS DEPRECATED ON MARCH 31 2021.
PLEASE REFER TO THE NEW VERSION "Plugins: Connect AWS IoT Core to Ubidots using HTTPS"
To connect AWS and Ubidots, you can use the AWS Plugin offered by Ubidots which automatically links AWS resources (AWS IoT Core, AWS IoT Rules, and AWS Kinesis Data Streams) for you to stream data directly to your Ubidots powered Apps.
This user guide explains how to integrate AWS IoT with Ubidots using the AWS Plugin offered by Ubidots and an AWS Access Key. This integration will automatically create the integration between AWS and Ubidots using the below data-flow diagram.
Table of Contents:
About AWS IoT Core
Retrieve an AWS User Access Key
Adding the AWS Plugin
Approve Automatic Resource Setup
Test the Plugin integration
How to send data to Ubidots, through AWS IoT Core
NOTE: For an advanced setup using AWS IAM Role to integrate AWS IoT with Ubidots, please reference to this guide.
NOTE: THIS IS A BETA FEATURE. PLEASE REACH OUT TO YOUR UBIDOTS ACCOUNT REPRESENTATIVE TO ENABLE THIS FEATURE FOR YOU
1. About AWS IoT Core
AWS IoT Core allows you to connect devices to the AWS cloud. You can think of AWS as a managed MQTT broker that accepts any type of message, then routes it to other AWS services and resources.
As an MQTT service, AWS IoT Core is similar to Ubidots MQTT broker, with 3 notable differences:
AWS IoT Core does not store your sensor data. It is only an intermediate broker to route messages between your devices and other AWS services.
AWS IoT Core doesn't have a predetermined JSON structure. As a platform that targets software developers, AWS IoT Core accepts arbitrary JSON payloads, which are then interpreted by self-built, native web applications. Ubidots is different because we've already built that web application for you, and that application needs a predetermined JSON payload structure so we can extract the Ubidots device label, variable label and value.
AWS IoT Core doesn't accept non-encrypted connections. AWS IoT Core provides authentication and end-to-end encryption throughout all points of connection, so that data is never exchanged between devices and AWS IoT Core without proven identity. This is a plus in terms of security, but it can also slow down your prototyping phase.
AWS IoT Core: Rules
As a broker, AWS IoT Core needs a way to know where to send the messages received from the devices. For this purpose, AWS IoT Core allows you create rules to filter, transform, and act upon device data on the fly. Here's a sample scenario of data routing using Rules:
The Rule's "Data destinations" can be any AWS service within AWS ecosystem, such as S3 or DynamoDB. When used in conjunction with Ubidots, your AWS IoT Core will use a Rule to route the traffic to an AWS services called AWS Kinesis and then on to Ubidots:
AWS Kinesis Data Stream
Kinesis is an AWS real-time data streaming service that allows developers to create applications that collect and process large streams of data in real time. To enable the AWS+Ubidots integration, Ubidots choses to ingrate with AWS Kinesis among other AWS services because of these reasons:
Persistence: With Kinesis you expect your data to be queued reliably until it reaches its destination.
Speed: Kinesis is designed for real-time data streaming, making it suitable for IoT use cases.
Cost: Kinesis uses a simple pay as you go model where the main cost is based on "shards". Each "shard" has a capacity of 1,000 records/second and starts at $10.8 / month. Forwarding your data through Kinesis is more cost-effective than other approaches such as AWS Lambda.
Endorsement: AWS Technical Team and Ubidots Engineers went to the drawing board together and after analyzing efficient pricing, speed, and security services, the solution designed proved the winner of all options. :)
With the above understanding, let's get started with the integration!
2. Retrieve an AWS Access Key
Ubidots uses AWS Access Keys to securely create the required AWS resources to stream data to Ubidots. Using the Access Key Ubidots will create an AWS IAM Role to automatically execute the integration for you.
To retrieve your AWS Access Keys, go to your AWS IAM Management console, then click on the “Users” option at the left panel, and follow the below steps:
Step 1: Click on the "Add user" button.
Step 2: Enter a name for the user, Select the option “Programmatic Access” and move to next page.
Step 3: Click on the option "Attach existing policies directly". Look for "IAMFullAccess" and select it. Move to next page
Step 4: Skip the tags page and move to next page.
Step 5: Click on the "Create user" button.
Step 6: Copy the “Access key ID” and the “Secret access key” which will be used later to complete the AWS Plugin on Ubidots platform. You can also download the keys with the "Download .csv" button.
Once created,
IMPORTANT NOTE: As a security measure, Ubidots does NOT store any AWS Access Key IDs or Secret Keys. The permission with Access Keys is used only once at the beginning of a Plugin's setup to create a restricted role in AWS and securely setup AWS resources to stream data to Ubidots. If you’d rather not provide your AWS keys, an advanced integration can be setup using AWS IAM Role and this user guide.
3. Adding the AWS Plugin
Now with your Access key ID and Secrete access key, we'll move back to your Ubidots Admin account and execute the below 4 steps to add the AWS Plugin and start streaming data.
Step1: Create a new Plugin – from your Ubidots account, go to “Devices” → “Plugins” → “Create Data Plugin”:
Step 2: Select the "AWS IoT" and then click in the arrow button located at the bottom-right.
Step 3: Click “Connect” to link your AWS IoT account with Ubidots.
Step 4: Enter your AWS Access Key ID and Secret Access Key created earlier in this guide. By entering your AWS Access Key, Ubidots will automatically setup the required resources in AWS to complete the integration and begin streaming data to Ubidots that is flowing to AWS IoT.
IMPORTANT NOTE: As a security measure, Ubidots does NOT store any AWS Access Key IDs or Secret Keys. The permission with Access Keys is used only once at the beginning of a Plugin's setup to create a restricted role in AWS and securely setup AWS resources to stream data to Ubidots. If you’d rather not provide your AWS keys, an advanced integration can be setup using AWS IAM Role and this user guide.
4. Approve Automatic Resource Setup
After plugging Ubidots into your AWS account, Ubidots will automatically create the required resources to stream data from an AWS IoT device to Ubidots.
The resources to be created/used to stream data from AWS IoT Devices to Ubidots are:
AWS IoT Core: This is AWS’ managed cloud platform for connected devices to securely interact with cloud applications, AWS services and other devices. By default, all AWS accounts have this module enabled. Ubidots will not create an IoT core for you. Learn more.
AWS IoT Rules: A feature within AWS IoT Core, which gives your devices the ability to interact with AWS services. Ubidots will create one IoT Rule to route your device traffic to an AWS Kinesis Data Stream. Learn more.
AWS Kinesis Data Stream: This is AWS' real-time data streaming service, which allows developers to create applications that collect and process large streams of data in real time. Ubidots will create one Kinesis Data Stream. Learn more.
To create the above resources, simply select the "Deployment region" and click on “Create resources” in the Data Plugin creation process, then wait a few seconds while Ubidots knocks on Amazon’s doors to complete the integration:
5. Test the Plugin Integration
At this point, your Data Plugin is ready to be used. To test it, go to your AWS IoT Core console, then click on “Test”, then “Publish to a topic”:
Then fill the test fields with the following test / dummy data:
Subscription topic:
ubidots/v1/{DEVICE_LABEL}
Note: The IoT Rule that Ubidots creates on your behalf automatically routes all traffic going to the topic “ubidots/v1” through the AWS Kinesis Data Stream to Ubidots. This allows you to keep other devices private from Ubidots by simply creating a new topic.Dummy data:
{
"temperature": 4,
"humidity": 81
}
Please note the message structure below as a valid data format. You can add more variables and devices according to your application using this format if needed:
{
"variable1": value1,
"variable2": value2
}
Finally, click “Publish topic” to test the integration sending data from AWS to Ubidots; you should see your data reflected in your Ubidots account:
6. How to send data to Ubidots, through AWS IoT Core
Although AWS IoT Core is a generic broker that allows you to send arbitrary JSON messages, in order for Ubidots to understand the payload, please utilize the below JSON structure.
MQTT Topic:
/ubidots/v1/{DEVICE_LABEL}
Message format:
{
"variable1-API-label": "value1",
"variable2-API-label": "value2",
"variable3-API-label": "value3",
"variable4-API-label": "value4"
}
To learn more on how to connect your IoT devices to AWS IoT Core, check out these resources: