How Prosumers are Driving Big Data Innovation at Utilities

13 June 2018 | 5 minutes

By Kate Jory and Przemek Tomczak


Prosumers in the utility sector, who both produce and consume electricity, are increasingly influencing how utilities are interacting with ratepayers. These clients utilize distributed renewable energy resources to produce energy, both for themselves and for the grid, and actively monitor and manage their own energy use. Their unique ability to return power to the grid introduces a bi-directional flow of energy where there was formerly “one-way traffic.” This poses a coordination challenge, as the energy sector’s traditional supply-and-demand management approaches are forced into new and challenging territories.

Devices that are capable of not only producing energy but also of storing it, have become readily available for prosumers, lowering the barriers of entry and the total cost, making renewable energy increasingly competitive with regulated and centralized grid electricity prices. This trend is set to continue, with Bloomberg New Energy Finance estimating that 72% of the worldwide investment in new power generation until 2040 will be focused on renewables.

Challenges to the connected grid

Although prosumers can generate their own power, they are still connected to the grid. This presents a challenge for utilities and grid operators: they must continue to provide reliable, on-demand energy—yet the prosumers’ ability to inject energy back into the grid results in a great deal of supply variability. In contrast, the traditional large-scale production of energy relied on long-distance transmission and one-way supply, with little fluctuation in energy consumption. This power distribution structure is no longer sufficient for the bi-directional needs of the utility sector.

In the days of the traditional grid, utilities typically engaged with customers only when there was an issue with their bill, when there was a power outage, or when they needed to take a manual meter reading. Now, customers are more engaged with the energy process; their needs are becoming more demanding and complex resulting in an increasing number of service options and service providers vying for their attention. New companies are offering lower prices, improved customer service, and a strong commitment to renewables, which has caused the average churn rate in the European utility industry to have almost doubled since 2008.

Changing business models

The prosumer trend is pressuring utilities to modify their business models to include a greater number of engagement opportunities, information exchanges, pricing options, and non-regulated services. To support this change, players in the energy ecosystem need to have a more granular understanding of each individual consumer, incorporating information such as their energy consumption and generation, personal preferences, voltage, and the weather.

Investments in new technologies and approaches

Utilities are already making significant investments in advanced metering infrastructure that provide visible consumption and generation patterns for billing, outage management, planning, and operations. The foundation of this enhanced understanding is capturing and blending large amounts of different time-oriented data sets at highly frequent intervals. For example, utilities with advanced metering infrastructure are able to detect power outages and take corrective action before receiving any customer complaints. The information collected from advanced meters is also being used to implement energy conservation programs based on disaggregated energy consumption data that shows high energy-consuming appliances. This allows utilities to accurately report and recommend energy savings technologies and practices to customers and alert utility operations to significant deviations from a customer’s energy consumption profile.

Benefits to utilities

In addition to predicting customer behavior, greater data capture can also help utilities predict the load on different system assets. For instance, a large transformer may not be required in regions where a significant proportion of customers are generating their own electricity. However, a transformer investment in an area with few prosumers and growing demand, like from electric vehicle charging, may prove to be quite valuable.

Today’s prosumer environment requires utility providers to simultaneously take advantage of large volumes of historical and real-time data from heterogeneous sources (like from smart meters and transmission equipment) and dynamically aggregate them into an overall view of both the grid and of each prosumer. It happens that most of these data sources are time-series data and are becoming a challenge to manage and analyze.

The benefit to utilities of having more granular and timely data about consumer energy use and generation is a high-quality view of their customers, assets, and operations in nearly real-time. As big data becomes a core element of digitized energy systems, utilities are being pushed to invest in new processes and systems for simultaneous ingestion, analysis, decision making, and access to information by authorized parties. The return on these investments is hoped to be greater customer loyalty and more efficient energy transmission and distribution.

Przemek Tomczak is Senior Vice-President of Internet of Things and Utilities at KX. Previously, Przemek held senior roles at the Independent Electricity System Operator in Ontario, Canada, and top-tier consulting firms and systems integrators. Przemek also has a CPA and has a background in business, technology, and risk management.

Kate Jory is the KX Business Development Director in South Korea. Kate’s background is in physics and international business. In her role at KX, she is working on expanding the use of KX technology across a portfolio of global corporations in industries ranging from manufacturing to utilities to telecommunications. 

Start your journey to becoming an AI-first Enterprise with a personal demo.

Our team can help you to:

    For information on how we collect and use your data, please see our privacy notice. By clicking “Download Now” you understand and accept the terms of the License Agreement and the Acceptable Use Policy.