If you are familiar with the OmniServer, you probably already know that it can be used to retrieve information from devices that use ‘non-standard’ protocols to communicate – typically these are your weight scales, barcode readers, and printers.
However, did you know that the OmniServer can also retrieve data that are exposed via web services, or even websites? Keep reading to find out you can "scrape" web sites or services to integrate useful information into your systems using OmniServer.
With the push towards the edge in integration of all of the various devices and data sources out there, it's not a stretch at all that the web itself is a valuable source of data that you could use in conjunction with your other data sources to make informed decisions in your enterprises.
Why Would I Need To Do This?
Many of today’s instruments are also now displaying data on self-hosted web pages in addition to exposing the data through a communication port via the “normal” means most control engineers are familiar with. Doing this allows any user with a browser to access and retrieve the information, and allows OmniServer to do the same anywhere from the same network.
There are now also many web sites that may hold valuable information for your application that will be exposed via web APIs, such as current weather information and forecasts, commodities pricing, energy pricing, and so on. Instead of hard-coding the information into your application or having to manually enter it periodically, why not have OmniServer retrieve the values for you automatically?
How Can I Do This?
For this example we will be retrieving the current weather information for Charlotte, North Carolina (where our offices are located), using the free api that GeoNames.org exposes at http://api.geonames.org. For a comparison of various other weather information APIs out there, there is a useful article by ClimaCell that runs down eight of the most popular APIs here.
(For new users, if you'd like an introduction to using OmniServer, please click here.)
1. In the OmniServer Configuration, create a new "Ethernet" device, and set the address to be the base url we will be retrieving data from, and set the port to be 80:
2. Create a new Protocol named "WEATHER" with the following parameters:
- A new Item named “testString” of data type String
- A new Command/Response Message of Type Read:
- Request Message (you can copy/paste):
GET /weatherIcaoJSON?ICAO=KCLT&username=demo HTTP=1.1{$CR}{$LF}{$CR}{$LF} - Response Message (copy/paste):
{testString}{$CR}
- Request Message (you can copy/paste):
This protocol will request the current weather information and allow us to view the structure of the response in the OmniServer’s IO monitor.
3. Create a new Topic that links your "WEATHER" protocol to the Ethernet device created in Step 1.
4. At this point we can connect an OPC Client and request the testString item.
This will result in our request being sent to the web service, allowing us to see what the response will look like:
5. From here the rest of the response message can be written based on the response we received on our first poll.
Download the completed protocol to take a look in your OmniServer. Don't have OmniServer - Download the Free Trial Today!
6. Finally, update your client application to read the new protocol items, and see what values are returned:
Are there any limitations?
Unlike the vast majority of devices, web services do not contain static content. That is, it is in their very nature to change on a daily (sometimes real-time) basis.
The OmniServer protocol design must also take this into consideration. Creating a single Host message to process all data assumes that the format of the data returned will never change (i.e. fields will never be added or removed, and the order that fields are returned in will always be static, etc.).
An easy solution can be to configure any data fields as unsolicited messages, to allow some variance in the message structure, as was done with the example above.
Where can I get more information?
The web is full of information such as weather forecasts, commodities pricing, energy pricing and other useful data that, if it could be easily retrieved by your control systems, could help you make such decisions as when it is most efficient and cost-effective to operate, when it makes more sense to not operate, etc.
Email us at support@softwaretoolbox.com with your questions and subscribe to our blog for more quick and easy OmniServer tutorials and tips. And download the sample protocol from this blog and download your free trial of OmniServer to access weather in your own client.
