In this guide, we will go through the process of receiving Senseware Data Forwards in AWS. Specifically, we will put each data packet published by a Data Forward into a MySQL database in RDS. Listed below are the things we need:
• A MySQL database hosted in AWS RDS. For simplicity reasons, we will store the incoming data denormalized. It may be useful to normalize the data depending on your use case.
• A small application that is used to process the Senseware data forwards and put the data into the database. We will use 'AWS lamda' to avoid managing Servers and EC2 instances.
• An HTTP endpoint that we can point the Senseware data forward toward. We will use Amazon's API Gateway.
Step 1: Getting the code
You can download the code, and all other files needed for this, here. Download the entire Repository as a Zip file.
Step 2: Setting up the database
Log into your AWS Console and select “Amazon RDS”. Then Press "Launch a DB Instance"
Select MySQL as the Database engine
For this tutorial, we will use a development instance. For production usage, a database with production settings such as Multi-AZ Deployment is recommended.
Select a instance size and and allocated storage that fits your needs. Allocated storage can be increased through the AWS RDS console later. Make sure you pick a secure database username and password.
Configure your database name and pick a security group.
Note: Security Group setup is outside the scope of this tutorial because it depends on individual security requirements. For a start, it may be acceptable to make the database available to the public internet, but for general use, we recommend limiting who has access to the database through IP address filters or other Security Groups. For more info, see the RDS documentation.
Once the database has been created by Amazon, we have to access the MySQL database to create a table to store data in. Connect to the database via your favorite MySQL interface (e.g. MySQL Workbench) and run the SQL script to create the new table. The script can be found in the create-table.sql file.
Our database is now set up, and we're ready to configure AWS Lambda to receive data!
Step 3: Setting up AWS Lambda
Now select 'AWS Lamda' in the AWS Services menu, and click 'Get Started'.
Create a new function and select the 'microservice-http-endpoint' blueprint. Make sure the Security is set to 'Open'.
Next we will configure the function. Make sure the Name is set to index. Click upload to upload the zip file found in the code repository. Next, you have to configure your Environment Variables to pass in your database credentials that you set up in step 2. See the picture below. Make sure the variable name matches exactly and the variable value is the value you chose in step 2.
Lastly, set the Handler to index.myHandler and setup a new Lambda Role.
Note: For security reasons, it is advisable to assign this lambda function to a VPC, and assign a Security Group which has access to the MySQL database. For simplicity reasons, we will assume that your Database is open to the public internet in this guide and skip this step.
Next, verify that a Amazon API Gateway trigger was successfully set up. The API Gateway URL is used as a endpoint for Senseware's Data Forwards
Setup the Senseware Data Forward by pasting the API Gateway URL and enable channels for which you want to receive data
Your database should now begin receiving data!