Utility industry moving toward real-time data platforms

6 Oct 2016 | , , ,
Share on:

By Przemek Tomczak


Utilities are faced with growing pressure to reduce carbon emissions, help their customers optimize and reduce energy use, and update their aging infrastructure. To tackle these challenges, utilities will need to provide platforms with data and analytics to bring buyers and sellers of energy together, instead of just acting as sellers and providers of energy. These tools will enable the industry and its customers to implement distributed energy resources and adopt innovative measures to address the challenges and goals of the energy system.

I was recently at the GTM’s New York Rev Future 2016 conference where utility industry leaders talked a lot about engaging customers and the private sector to help reduce greenhouse gas emissions and generate more energy from renewable resources. Some of the approaches presented for achieving these goals included:

Advanced Metering and Data: Advanced Metering Infrastructure (AMI) is foundational for innovation in the energy system – including optimizing voltage, shifting energy use and conservation, and de-risking capital investments. Utilities need to understand the distribution system better and with much greater granularity with regards to time and space. To that end, one of the largest utilities in New York, ConEdison (ConEd), is moving to near real-time metering and data collection, with five and 15-minute reporting intervals for electricity and hourly reporting intervals for gas.

Communications Networks: Utilities require a resilient and low-latency network to support AMI; advanced distribution automation; distributed energy resources; diverse sensor traffic; rich data processing and analytics. For example, ConEd is deploying a network with sub-10 millisecond device-to-device communications; battery backup; Internet security protocols; dual-band mesh and > 2Mbps bandwidth.

Access to Data and Analytics: AMI data and access to data are important to enable the private sector, enabling customers and utilities ability to deliver energy resources through the distribution system at the right time and at the right price. Analytics are essential to engage with customers to help them adjust their behavior and manage their energy use. For example, ConEd is planning to launch the Green Button initiative by the middle of 2017 to facilitate access to energy data while complying with security and privacy requirements. The Green Button initiative provides utility customers with their energy usage information in a consumer-friendly, secure access format.

De-risking investments: Granular consumption and generation data is essential to reduce the risk of investments and to help determine the optimum placement and management of distributed energy resources. Recognizing that all risk can’t be eliminated, utilities are encouraged to look for partnerships and controlled experimentation to help evaluate and implement technologies at scale.

Powering the new electricity system with analytics

In New York, and around the world, there is excitement about the potential for the transformation of the energy system. As part of this future system, we see that utilities, customers and service providers will be interacting in real-time with their data. As a result, they will become more reliant on data analytics to:

(1) make risk-based investments

(2) optimize placement and operation distributed resources

(3) extend the useful life of assets

(4) engage with customers to help them better manage their energy use and generation.

This transition will require utilities to update their distribution management systems, data processing and analytics platforms to be able to work with many more devices and sensors on their grids and at greater speeds than ever before.

Kx delivers the highest-performance and most cost-effective sensor data management platform available for utilities, telecommunications, manufacturing and Internet of Things applications.

Kx for Sensors is an integrated platform for ingesting, processing, validating, estimating and analyzing real-time, streaming and historical data from IoT and sensors. It is fully extensible for organizations’ enterprise systems to power their current and future IoT and sensor analytics.


Signal processing in kdb+

Signal processing with kdb+

6 Sep 2018 | , ,

In the latest in our ongoing series of kdb+ technical white papers published on the Kx Developer’s site, Kx engineer Callum Biggs examines how kdb+/q can be used instead of popular software-based signal processing solutions. Signal processing is used for analyzing observable events, such as IoT sensor data, sounds, images and other types of pulses […]

Kx Insights: Machine learning subject matter experts in semiconductor manufacturing

9 Jul 2018 | , ,

Subject matter experts are needed for ML projects since generalist data scientists cannot be expected to be fully conversant with the context, details, and specifics of problems across all industries. The challenges are often domain-specific and require considerable industry background to fully contextualize and address. For that reason, successful projects are typically those that adopt a teamwork approach bringing together the strengths of data scientists and subject matter experts. Where data scientists bring generic analytics and coding capabilities, Subject matter experts provide specialized insights in three crucial areas: identifying the right problem, using the right data, and getting the right answers.