How to Easily Integrate Your Data with Google Cloud IoT Core

11 min read

Jul 8, 2021 2:00:00 PM


IoT, IIoT, MQTT, Connectivity of Things, Big Data, The Cloud - by now, you've heard these terms in some way, shape or form.  Perhaps you're working on a project requiring you to push and/or pull important process or business data to/from a cloud solution. Big data analytics applications ranging from predictive maintenance to "smart" manufacturing, amongst many other use cases, are becoming more and more common.

And Google© is a household name both in the private and professional world. Chances are, you got here using Google – whether through a Google search, or by using Google Chrome (or both). But did you know that Google has their own cloud solution called Google Cloud IoT Core©? Amongst many other things, Google Cloud IoT Core acts as an MQTT broker for storing and sharing important data in a secure cloud along with artificial intelligence and analytics and many other IoT services.

But how do you go from your data sources to Google Cloud IoT Core? In this second post in a series about connecting DataHub to IoT solutions (if you missed our first post on connecting to Amazon AWS IOT Core©, click here), we'll step through how you can configure Google Cloud IoT Core and connect it to your process and business data sources using the Cogent DataHub IoT Gateway©.

So you likely clicked on the "Continue Reading" button because you have a use case to take some data source whether it's OPC data, data in a database, data in an old DDE server or Excel, or some other data source and get that data into the cloud. And you're considering Google Cloud IoT Core as the potential cloud service you use for that purpose.

You want to know "HOW", though, so let's dive right in to stepping through how you can setup Google Cloud IoT Core and how to setup the Cogent DataHub to exchange data (whether pushing or pulling).

The following steps cover the bare minimum, quick start steps you'll need to follow to push data to Google Cloud IoT Core from an OPC DA server specifically. However, it's just as easy for any other data interface supported by DataHub including OPC UA, OPC AE, databases (ODBC), Modbus TCP devices, DDE data sources and more. And it's easy to consume data from Google Cloud with DataHub, as well. (Don't get discouraged by the fact that you see 9 steps - they go quickly so it's not a measure of difficulty by any means!)

First, it's assumed that:

Now, let's get started!

Step 1:  Create a new Project in Google Cloud

First, ensure your account has the Google Cloud IoT API enabled. You can do this through the Marketplace. Search IoT API and select Google Cloud IoT API. If it is not enabled, you will be able to click ENABLE.

Screenshot - Enabling Google Cloud IoT API

While logged in to your Google Cloud account in the Google Cloud Console, you'll click on Select a Project at the top.

Screenshot - Select a Project in Google Cloud IoT Core

Next, you'll click the New Project button to begin the creation process.

Screenshot - Add a New Project in Google Cloud IoT Core

Step 2:  Create a Google Cloud IoT Core Registry

Now that you have created a project, open the IoT Core from the left-hand menu.

Screenshot - IoT Core menu in Google Cloud IoT Core

Then click on the CREATE REGISTRY button towards the top.

Screenshot - Creating a Registry in Google Cloud IoT Core

Next you'll set the Registry ID and Region for this registry. The Registry ID is the permanent identifier and the Region is the specific global region where the data for this registry will be stored (this is permanent, so make sure to select the correct region since choosing a region that is significantly distant geographically from MQTT clients could negatively impact performance).

Screenshot - Defining Registry details in Google Cloud IoT Core

Then just click the CREATE button at the bottom to finish creating your registry.

Screenshot - Create Button for Registry in Google Cloud IoT Core

Step 3:  Add a Device to your Google Cloud IoT Core Registry

The next step is adding a device to the registry you just created that will push and/or pull data for the registry. Still under IoT Core and your Registry, select Devices from the left-hand menu.

Screenshot - Devices menu in Google Cloud IoT Core

Then click the CREATE A DEVICE button at the top.

Screenshot - Creating a Device in Google Cloud IoT Core

Enter the desired Device ID to represent your device - something meaningful is recommended.

Screenshot - Device properties in Google Cloud IoT Core

Step 4:  Create an Encryption Key using OpenSSL

You'll need to run the following command in a Windows PowerShell prompt with Admin permissions on a machine with OpenSSL installed.

openssl req -x509 -newkey rsa:2048 -keyout rsa_private.pem -nodes -out rsa_cert.pem -subj "/CN=unused"

This will create the following two files:

  • rsa_cert.pem
  • rsa_private.pem

Again using Windows PowerShell, you'll need to convert the rsa_cert.pem and rsa_private.pem to PFX format in your local machine with the following command:

openssl pkcs12 -inkey rsa_private.pem -in rsa_cert.pem -export -out rsa_cert.pfx

Step 5: Add Authentication to the Device in Google Cloud IoT Core

Back in Google Cloud IoT Core, select the device you just added and select its Authentication tab, and click ADD PUBLIC KEY.

Screenshot_Google_IoT_Add_Authentication

Select RS256_X509 for the Public Key Format. Open the rsa_cert.pem file generated in Step 4 in a text editor and copy the entire text into the Public Key Value box. Then click Add.

Step 6:  Attach Your AWS Policy to Certificate for your AWS Thing

Now that your certificate and associated public and private keys (for secure encryption purposes) have been created, you need to download a root Certificate Authority for Google Cloud IoT for use with the MQTT Client in Cogent DataHub IoT Gateway.

Google houses available CA certificates in its certificate repository. You can download the root .pem certificate file here: https://pki.goog/roots.pem.

Step 7:  Define a Data Source in Cogent DataHub

Now that your Google Cloud registry and device are configured, you can proceed to your Cogent DataHub. DataHub is multi-functional and can aggregate data from a wide variety of data sources into "Data Domains" which are essentially a grouping of data points of interest from devices and other data sources such as databases.

As I mentioned earlier, DataHub supports OPC UA, OPC DA, OPC AE, DDE, ODBC databases, Modbus devices and so much more.  For a full list of supported interfaces, click here. For our purposes here, an OPC DA server data source already exists in the DataHub with data in a Data Domain named "OPC_DA".

Screenshot - Accessing data from OPC DA server in DataHub

For detailed information on creating OPC DA connections, OPC UA and other supported data sources, click here for our how-to video library.

Step 8:  Define a new MQTT Client configuration in Cogent DataHub

Next, you'll need to configure an MQTT Client connection in the DataHub with settings specific to the Google Cloud registry and device you just created. So click the Add button (making sure to "Enable MQTT client connections").

Screenshot - Add a new MQTT Client connection in DataHub

Then, go to the specialized section for Google IoT and configure the following:

Screenshot - Google IoT specific settings in DataHub MQTT Client

  • Project ID - the Project ID from the Project you created in Google
  • Registry ID - the Registry ID from the Registry you created in Google
  • Device ID - the Device ID from the Device you created in Google
  • Cloud Region - the Region you selected in the Registry you created in Google
  • Client Certificate - the path to rsa_cert.pfx that you created in Step 4 above

Once you've entered the correct values, click the Reconfigure button to populate some of the necessary information in the Connection, Authentication and Message Content tabs.

Go to the Authentication tab. In the CA Certificate field, browse to the Google CA root certificate file that you downloaded previously. If the certificate is in PEM format, you can use it without modification even if the DataHub requests a CRT format file (just make sure to set the type to "All Files" when browsing so that the file shows up).

Screenshot - Selecting Google CA Root certificate in DataHub

Then go to the Connection tab. Enter a meaningful Label to identify the connection. There should be no spaces in the name and it can be whatever you want as long as it is unique to other connection names in DataHub.

Screenshot - Google IoT connection properties in Cogent DataHub

Step 9:  Select Data to Push to Google Cloud from Cogent DataHub

Now that the connection to your Google Cloud IoT Core device is configured, you need to tell DataHub what specific data you'd like to push to Google from your configured data source or sources (this example will simply push a single OPC DA point).

Under the "Push data points to the MQTT broker (with Google IoT being the MQTT broker here), you'll find a tree view of available DataHub data domains.

As you saw earlier, there is already an OPC DA data source configured with a data domain of "OPC_DA". Expanding that data domain exposes the available data points and checking the box next to the desired data points in the domain adds them to the "Selected Points" list.

Screenshot - Selecting data to push to AWS via MQTT

Also in this section, you need to define the desired MQTT Topic to publish your data to in the AWS MQTT broker - check the box "Send all messages to this topic" and define the desired meaningful label for your Topic which will get created in your Google registry and will be the repository for your data from this connection.

Then, click OK in DataHub and make sure to click the Apply button at the bottom to ensure your settings are saved and your connection to Google Cloud IoT is initiated and established.

So, in closing, due to the specialized interface in the Cogent DataHub IoT Gateway with settings specific to Google Cloud IoT Core, it's straightforward to get setup integrating your own process and business data to the Google Cloud.

And though the above steps focused on pushing data from an OPC DA server to Google Cloud, it's just as easy to consume data from Google Cloud IoT Core - simply define the desired MQTT Pub/Sub topic for other registries you may have configured in your the Google Cloud under "Pull topics from the MQTT broker" and define the data domain in DataHub where you wish that data to be available. Once in the DataHub, you can use it for bridging to other data sources or in any other function supported by the DataHub using data.

As long as you have a free or paid Google Cloud IoT Core account, you can test things out with your own data sources with a free trial of the Cogent DataHub. And make sure you don’t miss future useful topics on Cogent DataHub and other helpful industrial automation topics by subscribing to our blog.Push Your Process Data to Google Cloud IoT Core with DataHub Free Trial

 

Zachary Parham
Written by Zachary Parham

Software ToolBox Technical Blog

We're engineers like you, so this blog focuses on "How to" appnotes, videos, tech team tips, product update announcements, user case studies, and other technical updates.  Subscribe to updates below. Your feedback and questions on posts are always welcomed - just use the area at the bottom of any post.

Subscribe to our Blog

Recent Posts

Posts by Topic

See all