IoT, IIoT, MQTT, Connectivity of Things, Big Data, The Cloud - by now, you've heard these terms in some way, shape or form. Perhaps you're working on a project requiring you to push and/or pull important process or business data to/from a cloud solution. Big data analytics applications ranging from predictive maintenance to "smart" manufacturing, amongst many other use cases, are becoming more and more common.
And Google© is a household name both in the private and professional world. Chances are, you got here using Google – whether through a Google search, or by using Google Chrome (or both). But did you know that Google has their own cloud solution called Google Cloud IoT Core©? Amongst many other things, Google Cloud IoT Core acts as an MQTT broker for storing and sharing important data in a secure cloud along with artificial intelligence and analytics and many other IoT services.
But how do you go from your data sources to Google Cloud IoT Core? In this second post in a series about connecting DataHub to IoT solutions (if you missed our first post on connecting to Amazon AWS IOT Core©, click here), we'll step through how you can configure Google Cloud IoT Core and connect it to your process and business data sources using the Cogent DataHub IoT Gateway©.
So you likely clicked on the "Continue Reading" button because you have a use case to take some data source whether it's OPC data, data in a database, data in an old DDE server or Excel, or some other data source and get that data into the cloud. And you're considering Google Cloud IoT Core as the potential cloud service you use for that purpose.
You want to know "HOW", though, so let's dive right in to stepping through how you can setup Google Cloud IoT Core and how to setup the Cogent DataHub to exchange data (whether pushing or pulling).
The following steps cover the bare minimum, quick start steps you'll need to follow to push data to Google Cloud IoT Core from an OPC DA server specifically. However, it's just as easy for any other data interface supported by DataHub including OPC UA, OPC AE, databases (ODBC), Modbus TCP devices, DDE data sources and more. And it's easy to consume data from Google Cloud with DataHub, as well. (Don't get discouraged by the fact that you see 9 steps - they go quickly so it's not a measure of difficulty by any means!)
First, it's assumed that:
- You have a Google Cloud Platform (free or paid) account
- You have a data source configured in Cogent DataHub for the data you'd like to push to Google Cloud.
- If not, we have a full list of detailed how-to's here on configuring various data sources in the Cogent DataHub!
Now, let's get started!
Step 1: Create a new Project in Google Cloud
First, ensure your account has the Google Cloud IoT API enabled. You can do this through the Marketplace. Search IoT API and select Google Cloud IoT API. If it is not enabled, you will be able to click ENABLE.
While logged in to your Google Cloud account in the Google Cloud Console, you'll click on Select a Project at the top.
Next, you'll click the New Project button to begin the creation process.
Step 2: Create a Google Cloud IoT Core Registry
Now that you have created a project, open the IoT Core from the left-hand menu.
Then click on the CREATE REGISTRY button towards the top.
Next you'll set the Registry ID and Region for this registry. The Registry ID is the permanent identifier and the Region is the specific global region where the data for this registry will be stored (this is permanent, so make sure to select the correct region since choosing a region that is significantly distant geographically from MQTT clients could negatively impact performance).
Then just click the CREATE button at the bottom to finish creating your registry.
Step 3: Add a Device to your Google Cloud IoT Core Registry
The next step is adding a device to the registry you just created that will push and/or pull data for the registry. Still under IoT Core and your Registry, select Devices from the left-hand menu.
Then click the CREATE A DEVICE button at the top.
Enter the desired Device ID to represent your device - something meaningful is recommended.
Step 4: Create an Encryption Key using OpenSSL
You'll need to run the following command in a Windows PowerShell prompt with Admin permissions on a machine with OpenSSL installed.
Note: If you must install OpenSSL before proceeding, please consult with your IT department to remedy any issues that are encountered during install. Installing OpenSSL successfully has been known to involve changing the Execution Policy on the Windows OS and using a Package Manager. It is important to know the risks that are involved.
openssl req -x509 -newkey rsa:2048 -keyout rsa_private.pem -nodes -out rsa_cert.pem -subj "/CN=unused"
This will create the following two files:
- rsa_cert.pem
- rsa_private.pem
Again using Windows PowerShell, you'll need to convert the rsa_cert.pem and rsa_private.pem to PFX format in your local machine with the following command:
openssl pkcs12 -inkey rsa_private.pem -in rsa_cert.pem -export -out rsa_cert.pfx
Step 5: Add Authentication to the Device in Google Cloud IoT Core
Back in Google Cloud IoT Core, select the device you just added and select its Authentication tab, and click ADD PUBLIC KEY.
Select RS256_X509 for the Public Key Format. Open the rsa_cert.pem file generated in Step 4 in a text editor and copy the entire text into the Public Key Value box. Then click Add.
Step 6: Download the Root Certificate Authority for Google Cloud IoT
Now that your certificate and associated public and private keys (for secure encryption purposes) have been created, you need to download a root Certificate Authority for Google Cloud IoT for use with the MQTT Client in Cogent DataHub IoT Gateway.
Google maintains available CA certificates in its certificate repository. You can download the root .pem certificate file here: https://pki.goog/roots.pem.
Step 7: Define a Data Source in Cogent DataHub
Now that your Google Cloud registry and device are configured, you can proceed to your Cogent DataHub. DataHub is multi-functional and can aggregate data from a wide variety of data sources into "Data Domains" which are essentially a grouping of data points of interest from devices and other data sources such as databases.
As I mentioned earlier, DataHub supports OPC UA, OPC DA, OPC AE, DDE, ODBC databases, Modbus devices and so much more. For a full list of supported interfaces, click here. For our purposes here, an OPC DA server data source already exists in the DataHub with data in a Data Domain named "OPC_DA".
For detailed information on creating OPC DA connections, OPC UA and other supported data sources, click here for our how-to video library.
Step 8: Define a new MQTT Client configuration in Cogent DataHub
Next, you'll need to configure an MQTT Client connection in the DataHub with settings specific to the Google Cloud registry and device you just created. So click the Add button (making sure to "Enable MQTT client connections").
Then, go to the specialized section for Google IoT and configure the following:
- Project ID - the Project ID from the Project you created in Google
- Registry ID - the Registry ID from the Registry you created in Google
- Device ID - the Device ID from the Device you created in Google
- Cloud Region - the Region you selected in the Registry you created in Google
- Client Certificate - the path to rsa_cert.pfx that you created in Step 4 above
Once you've entered the correct values, click the Reconfigure button to populate some of the necessary information in the Connection, Authentication and Message Content tabs.
Go to the Authentication tab. In the CA Certificate field, browse to the Google CA root certificate file that you downloaded previously. If the certificate is in PEM format, you can use it without modification even if the DataHub requests a CRT format file (just make sure to set the type to "All Files" when browsing so that the file shows up).
Then go to the Connection tab. Enter a meaningful Label to identify the connection. There should be no spaces in the name and it can be whatever you want as long as it is unique to other connection names in DataHub.
Step 9: Select Data to Push to Google Cloud from Cogent DataHub
Now that the connection to your Google Cloud IoT Core device is configured, you need to tell DataHub what specific data you'd like to push to Google from your configured data source or sources (this example will simply push a single OPC DA point).
Under the "Push data points to the MQTT broker (with Google IoT being the MQTT broker here), you'll find a tree view of available DataHub data domains.
As you saw earlier, there is already an OPC DA data source configured with a data domain of "OPC_DA". Expanding that data domain exposes the available data points and checking the box next to the desired data points in the domain adds them to the "Selected Points" list.
Also in this section, you need to define the desired MQTT Topic to publish your data to in the AWS MQTT broker - check the box "Send all messages to this topic" and define the desired meaningful label for your Topic which will get created in your Google registry and will be the repository for your data from this connection.
Then, click OK in DataHub and make sure to click the Apply button at the bottom to ensure your settings are saved and your connection to Google Cloud IoT is initiated and established.
So, in closing, due to the specialized interface in the Cogent DataHub IoT Gateway with settings specific to Google Cloud IoT Core, it's straightforward to get setup integrating your own process and business data to the Google Cloud.
And though the above steps focused on pushing data from an OPC DA server to Google Cloud, it's just as easy to consume data from Google Cloud IoT Core - simply define the desired MQTT Pub/Sub topic for other registries you may have configured in your the Google Cloud under "Pull topics from the MQTT broker" and define the data domain in DataHub where you wish that data to be available. Once in the DataHub, you can use it for bridging to other data sources or in any other function supported by the DataHub using data.
As long as you have a free or paid Google Cloud IoT Core account, you can test things out with your own data sources with a free trial of the Cogent DataHub. And make sure you don’t miss future useful topics on Cogent DataHub and other helpful industrial automation topics by subscribing to our blog.
