Kx powers Data Mart for Ontario’s Smart Meter Data

23 Aug 2017 | , , , , , , , ,
Share on:

By Przemek Tomczak

Ontario’s electricity system is overseen by a semi-governmental entity called the Independent Electricity System Operator (IESO). The IESO’s responsibilities include ensuring that there is enough power to meet the province’s energy needs and to plan for future needs. As part of that mission, a Smart Meter Entity within the IESO administers the data generated by Ontario’s four million smart meters.

The volume of data generated by the province exceeds 250 billion smart meter reads, and grows by 100 million meter reads per day. The administrator of this data is required to provide it to the utilities in the province in a timely manner and has set up a meter data management system (MDM/R) to do so.

However, the existing transaction and relational database systems within the MDM/R were not originally designed to handle the increasing requirements of the fast growing database. That’s where Kx came in with its high-performance database system kdb+, and the suite of powerful visualization tools built on top of it.

The IESO chose Kx after thoroughly testing a number of other technologies. They found that kdb+ delivered very high levels of performance at the lowest total cost of ownership. The IESO conducting over 25 tests on five years of data from five million meters, and proved that kdb+ outperformed any of its competitors.

To read more about the IESO case study please click here.

SUGGESTED ARTICLES

Signal processing in kdb+

Signal processing with kdb+

6 Sep 2018 | , ,

In the latest in our ongoing series of kdb+ technical white papers published on the Kx Developer’s site, Kx engineer Callum Biggs examines how kdb+/q can be used instead of popular software-based signal processing solutions. Signal processing is used for analyzing observable events, such as IoT sensor data, sounds, images and other types of pulses […]

Kx Insights: Machine learning and the value of historical data

2 Aug 2018 | , , ,

Data is being generated at a faster rate now than ever before. IDC has predicted that in 2025, there will be 163 zettabytes of data generated each year—a massive increase from the 16.1 zettabytes created in 2016. These high rates of data generation are partially an outcome of the multitude of sensors found on Internet of Things (IoT) devices, the majority of which are capable of recording data many times per second. IHS estimates that the number of IoT devices in use will increase from 15.4 billion devices in 2015 to 75.4 billion in 2025, indicating that these immense rates of data generation will continue to grow even higher in the years to come.