Edge Analytics and IoT – How Kx is Revolutionizing Manufacturing from the Edge

27 Mar 2017 | , , , , , , , ,
Share on:

By Michael Donaghy


Edge analytics is a method of data analysis and collection that allows an automated analytical computation to be performed on a tag or sensor rather than having all of the ingested data sent back to a centralized data warehouse before being acted upon. Manufacturing is an industry which requires real-time action on analytics at their source. This is a challenge which Kx excels at.

An average offshore oil rig has around 30,000 sensors. These sensors measure gas emissions, track pressure, monitor drills and yet only one percent of what they ingest and send to a data warehouse is of value to the decision-making process. Setting up sensors to route all of the data to a centralized system in a raw state is time-consuming and costly. It requires the data be cleansed and analyzed, eating heavily into available bandwidth. Therefore a solution is needed to carry these tasks out at the point of collection to take advantage of devices or gateways on the “edge.”

The “edge” aggregates the tags and sensors scattered across a network or embedded within devices, such as a sensor in a connected car. By embedding computational capability in the form of Complex Event Processing (CEP) at the furthest points of the network, Kx can take advantage of the ‘chatty’ or ‘noisy’ data which is generated by making sense of it at the point of ingestion and sending on securely only what it deems to be useful to the data historian for situational awareness.

However, this is not the only advantage of performing edge analytics with Kx. By running your data through an analytics algorithm at the point of ingestion, not only will you create a framework that only allows the data that passes a certain criteria to flow through to the cloud or data warehouse, but it will allow you to perform decisions on malfunctioning or faulty equipment in real-time, thus preventing unnecessary downtime.

As manufacturing is an incredibly time-sensitive industry where every moment of equipment downtime counts, the sub-millisecond CEP latencies of Kx have proven popular with manufacturing clients who benefit greatly from rapid automated decisions.

One needs only to look at the autonomous vehicle industry to see why rapid, real-time automated decisions are necessary. For example, an autonomous “smart” car produces around 25 GB/hour in data.  However, only a small portion of this data is relevant when carrying out hazard detection. The CEP performed by Kx at the edge allows abnormalities to be detected in real-time with an alert generated immediately. The same applies in the aviation industry where the average aircraft produces data volumes of 51 TB/hour, or in locomotives, where a “smart” train contains 250 tags that ingest data from 150,000 points per minute.

These planes, trains and automobiles cannot afford to rely on decisions coming from the centralized data warehouse when it comes to fault and hazard detection and therefore require technology capable of performing analytics and generating alerts in real time. Manufacturing is no different.

By building edge-based analytical systems into machinery, manufacturers can utilize Kx’s ability to work on the edge of the network by having it send alerts and load new scripts into appliances when conditions change.

For example, think of an automated drilling station. The drill may have sensors that measure vibration, rotation speed, temperature, velocity, etc.  Through CEP analysis, Kx may notice from the data it has ingested from the tag points that the drill is overheating. An alert is generated when the anomaly is discovered, a new command script loaded to halt the drill and the resulting data fed into a machine learning model.

The small footprint of Kx technology (500KB) and its ability to perform edge analytics with sub-millisecond latencies while compressing data to within a fragment of its original size, all make Kx a prime candidate for manufacturers to store and analyze huge volumes of IoT data in a scalable, efficient and secure way.

In its most simplistic form, edge analytics allows manufacturers to have data ingested from a sensor, filter out the noise surrounding it, identify a potential fault and act on that fault. It is imperative that manufacturers take advantage of edge analytics to reduce downtime and improve throughput as many are currently doing. The platform which will allow these analytics and computations to be carried out in the fastest time and the most secure way is Kx.

For more information on Kx for Manufacturing click here.

Michael Donaghy is a Global Business Development Executive at First Derivatives based in Newry, Northern Ireland.

© 2018 Kx Systems
Kx® and kdb+ are registered trademarks of Kx Systems, Inc., a subsidiary of First Derivatives plc.


Head of Products, Solutions and Innovation at Kx on Product Design and the Vision for the Future

16 Mar 2018 | , , ,

As the SVP of Products, Solutions and Innovation at Kx Systems, James Corcoran is part of a new chapter in software development at Kx. Since joining Kx parent First Derivatives as a financial engineer in 2009, James has worked around the world building enterprise systems at top global investment banks before moving to the Kx product team in London. James sat down with us recently to discuss his perspective on product design and our technology strategy for the future.

Kx featured on 2018 IoT Landscape Map

7 Mar 2018 | , ,

First Mark Capital’s Matt Turck has been talking about the growth of both the Big Data and the Internet of Things ecosystems for several years in his blog and elsewhere. An outgrowth of this interest has been graphic “maps” of both the Big Data landscape and the Internet of Things landscape that portray all of the significant players in those markets in a particular year. Subsequent versions of Matt’s maps, which are published annually, show the speed of change of IoT technologies and approaches, especially for managing and analyzing large datasets generated by IoT devices. 

Kdb+ Utilities: Essential utility for identifying performance problems

28 Feb 2018 | ,

If you are a kdb+/q developer, you will find the utilities created by Kx Managing Director and Senior Solution Architect Leslie Goldsmith to be a valuable resource. The “Kdb+ Utilities” series of blog posts gives a quick introduction to the utilities, available at Leslie Goldsmith’s GitHub. In this third part of the series we look at Leslie’s qprof, which allows a programmer to drill down into q functions or applications to inspect performance and CPU usage in a fine-grained fashion.