kdb+ at IESO

Kx powers Data Mart for Ontario’s Smart Meter Data

23 Aug 2017 | , , , , , , , ,
Share on:

By Przemek Tomczak

Ontario’s electricity system is overseen by a semi-governmental entity called the Independent Electricity System Operator (IESO). The IESO’s responsibilities include ensuring that there is enough power to meet the province’s energy needs and to plan for future needs. As part of that mission, a Smart Meter Entity within the IESO administers the data generated by Ontario’s four million smart meters.

The volume of data generated by the province exceeds 250 billion smart meter reads, and grows by 100 million meter reads per day. The administrator of this data is required to provide it to the utilities in the province in a timely manner and has set up a meter data management system (MDM/R) to do so.

However, the existing transaction and relational database systems within the MDM/R were not originally designed to handle the increasing requirements of the fast growing database. That’s where Kx came in with its high-performance database system kdb+, and the suite of powerful visualization tools built on top of it.

The IESO chose Kx after thoroughly testing a number of other technologies. They found that kdb+ delivered very high levels of performance at the lowest total cost of ownership. The IESO conducting over 25 tests on five years of data from five million meters, and proved that kdb+ outperformed any of its competitors.

To read more about the IESO case study please click here.

© 2017 Kx Systems
Kx® and kdb+ are registered trademarks of Kx Systems, Inc., a subsidiary of First Derivatives plc.

SUGGESTED ARTICLES

Kx collaborating with Fintech startup chartiq

Collaboration: The Dominant Trend in Finance

13 Dec 2017 | , , , ,

In December we are re-blogging some of our favorite content from Kx partners and affiliated companies, starting with this article on the ChartIQ blog. ChartIQ is an agile FinTech company that sells an advanced HTML5 charting library used in technical data analysis, trading configurations and for charting in the capital markets industry. Kx offers a ChartIQ integration as an addition to our Dashboards. In Collaboration: The Dominant Trend in Finance, ChartIQ’s Hanni Chehak writes about the rise of FinTech companies, and the role collaboration plays as FinTech companies are increasingly disrupting the traditional banking sector.

Water system workers with kdb+ historical database

Kdb+ Use Case: Machine Learning Water System Maintenance Application

6 Dec 2017 | , , , ,

Kdb+ is being used much more widely in machine learning applications today. Its ability to quickly ingest and process data, particularly large, fragmented datasets, is one way that developers are adding kdb+ to their technology stack of artificial intelligence and machine learning tools.
For Australian kdb+ developer Sherief Khorshid, who also develops machine learning systems, incorporating kdb+ into a predictive maintenance application gave him the edge in a hackathon win that landed him a cash prize and a contract with the Water Corporation of Western Australia.

kdb+ FFI

Kdb+ FFI: Access external libraries more easily from q

22 Nov 2017 | , , ,

Following on from the hugely popular Python library and interface embedPy and PyQ, Kx has released an FFI as part of the Fusion for kdb+ interfaces. As with embedPy and PyQ, this FFI is open-sourced under the Apache 2 license.
The kdb+ FFI is a foreign function interface library for loading and calling dynamic libraries from q code. It has been adapted and expanded upon from a library originally written by Alex Belopolsky of Enlightenment Research. With the kdb+ FFI you can now call your favorite C/C++ libraries directly from q without the overhead of having to compile shared objects and load into q using the 2: command.