utilities using kdb+ for Prosumers

Kx Insights: How Prosumers are driving Big Data Innovation at Utilities

13 Jun 2018 | , , , ,
Share on:

By Kate Jory and Przemek Tomczak


Prosumers in the utility sector, who both produce and consume electricity, are increasingly influencing how utilities are interacting with rate payers. These clients utilize distributed renewable energy resources to produce energy, both for themselves and for the grid, and actively monitor and manage their own energy use. Their unique ability to return power to the grid introduces a bi-directional flow of energy where there was formerly “one-way traffic.” This poses a coordination challenge, as the energy sector’s traditional supply-and-demand management approaches are forced into new and challenging territories.

Devices that are capable of not only producing energy, but also of storing it, have become readily available for prosumers, lowering the barriers of entry and the total cost, making renewable energy increasingly competitive with regulated and centralized grid electricity prices. This trend is set to continue, with Bloomberg New Energy Finance estimating that 72% of the worldwide investment in new power generation until 2040 will be focused on renewables.

Challenges to the connected grid

Although prosumers can generate their own power, they are still connected to the grid. This presents a challenge for utilities and grid operators: they must continue to provide reliable, on-demand energy—yet the prosumers’ ability to inject energy back into the grid results in a great deal of supply variability. In contrast, the traditional large-scale production of energy relied on long distance transmission and one-way supply, with little fluctuation in energy consumption. This power distribution structure is no longer sufficient for the bi-directional needs of the utilities sector.

In the days of the traditional grid, utilities typically engaged with customers only when there was an issue with their bill, when there was a power outage, or when they needed to take a manual meter reading. Now, customers are more engaged with the energy process; their needs are becoming more demanding and complex resulting in an increasing number of service options and service providers vying for their attention. New companies are offering lower prices, improved customer service, and a strong commitment to renewables, which has caused the average churn rate in the European utilities industry to have almost doubled since 2008.

Changing business models

The prosumer trend is pressuring utilities to modify their business models to include a greater number of engagement opportunities, information exchanges, pricing options, and non-regulated services. To support this change, players in the energy ecosystem need to have a more granular understanding of each individual consumer, incorporating information such as their energy consumption and generation, personal preferences, voltage, and the weather.

Investments in new technologies and approaches

Utilities are already making significant investments in advanced metering infrastructure that provide visible consumption and generation patterns for billing, outage management, planning, and operations. The foundation of this enhanced understanding is capturing and blending large amounts of different time-oriented data sets at highly frequent intervals. For example, utilities with advanced metering infrastructure are able to detect power outages and take corrective action before receiving any customer complaints. The information collected from advanced meters is also being used to implement energy conservation programs based on disaggregated energy consumption data that shows high energy consuming appliances. This allows utilities to accurately report and recommend energy savings technologies and practices to customers and alert utility operations to significant deviations from a customer’s energy consumption profile.

Benefits to utilities

In addition to predicting customer behavior, greater data capture can also help utilities predict the load on different system assets. For instance, a large transformer may not be required in regions where a significant proportion of customers are generating their own electricity. However, a transformer investment in an area with few prosumers and growing demand, like from electric vehicle charging, may prove to be quite valuable.

Today’s prosumer environment requires utility providers to simultaneously take advantage of large volumes of historical and real-time data from heterogeneous sources (like from smart meters and transmission equipment) and dynamically aggregate them into an overall view of both the grid and of each prosumer. It happens that most of these data sources are time-series data and are becoming a challenge to manage and analyze.

The benefit to utilities of having more granular and timely data about consumer energy use and generation is a high-quality view of their customers, assets, and operations in nearly real-time. As big data becomes a core element of digitized energy systems, utilities are being pushed to invest in new processes and systems for simultaneous ingestion, analysis, decision making, and access to information by authorized parties. The return on these investments is hoped to be greater customer loyalty and more efficient energy transmission and distribution.

Przemek Tomczak is Senior Vice-President of Internet of Things and Utilities at Kx Systems. Previously, Przemek held senior roles at the Independent Electricity System Operator in Ontario, Canada and top-tier consulting firms and systems integrators. Przemek also has a CPA and has a background in business, technology, and risk management.

Kate Jory is the Kx Business Development Director in South Korea. Kate’s background is in physics and international business. In her role at Kx she is working on expanding the use of Kx technology across a portfolio of global corporations in industries ranging from manufacturing to utilities to telecommunications. 


kdb+ for industrial internet of things 4.0

Kx Insights: IIoT for Predictive Maintenance and Big Data

9 Jan 2018 | , , , , ,

IIoT for predictive maintenance enables more extensive monitoring of equipment and processes at a much lower cost than traditional methods and delivers actionable warnings to prevent or minimize the consequences of an impending failure. Where IIoT for predictive maintenance is deployed in a well-designed program using Reliability Centered Maintenance (RCM) it will reduce surprise outages, lost production, extensive repairs, secondary damage and increase safety.

Kx powers Data Mart for Ontario’s Smart Meter Data

20 Mar 2017 | , , , ,

The IESO’s Data Mart uses Kx technology to support high volumes of daily data access requests and anticipated growth with 24x7x365 availability, as well as current and future demands for advanced analytics in Ontario’s energy sector. For example, with kdb+, the IESO is able to aggregate an entire year of data for all meters in a few minutes, which supports analysis of load profiles, consumption behavior and different rate models.

Utility industry trends, kdb+

Eight Utility Industry Trends in 2017

11 Jan 2017 | ,

2017 is shaping up to be a year of change for the electric utility industry as the adoption of smart grid and customer data management technologies accelerates. Based on dozens of conversations with utility market players, Przemek Tomczak explains what the industry’s current primary focus is and the role technologies like those powered by kdb+ can play.