By Przemek Tomczak
2017 is shaping up to be a year of change for the electric utility industry as the adoption of smart grid and customer data management technologies accelerates.
Judging by the latest world gathering of the utility industry at European Utility Week in Barcelona last November, the industry’s conversations around Big Data, that started a decade ago, have evolved from the theoretical to the practical. There are more projects and real-life customer stories, as well as wide recognition that Big Data technologies are already available to support the industry’s digital transformation. Going forward in 2017, the focus will be on removing barriers to data-intensive best practices.
The reality for the utility industry worldwide is that the demand for energy, water, and transportation resources is outpacing available supply, while energy consumers are facing increasing prices and fewer choices of energy sources from their utilities.
To help meet these demands, the electricity industry is moving from centralized to decentralized energy production, conventional to renewable energy, fixed to flexible resources, the electrification of the transportation system, and from passive to active consumers.
There are also new entrants to the industry offering renewable generation, battery storage, electric vehicle charging, and energy management services to customers for a share of the customer’s savings. However, these new providers are faced with significant barriers with the lack of access to granular and timely energy data, and markets that still prefer traditional technologies.
Based on dozens of conversations with utilities, vendors and other market players, I see the following eight trends for the industry for the upcoming year:
- Renewed focus on engaging with the customer — providing customer choice and tailored solutions. This will involve customers being provided with more relevant and timely information and with tools for active participation in the market and to better manage their energy use and consumption. Communications with customers will include mobile, web, and social media. Customers will be increasingly empowered through data and data access platforms to choose different suppliers, technologies and services to support their unique needs.
- Market reforms and regulations to drive increased interactions among electricity stakeholders based on metering data and new innovative products being offered to customers. Regulators will remove barriers to adoption of new technologies. Data hubs and data access platforms will be in various stages of implementation in Europe to facilitate more competitive markets, consumer choice and new participants, such as aggregators.
- Next generation smart meters and advanced metering infrastructure will provide greater granularity and frequency of information to utilities, consumers and authorized solution providers. For example, the first generation of smart meters that were deployed in Europe in the early to mid-2000s are in the process of being replaced to support distributed generation, new pricing models, and operational requirements.
- Building energy management services will gain more attention based the results of studies and pilots that show savings of 15% to 40% on water, gas, electricity use by using analytics, control systems and new technologies.
- New suppliers and service providers will offer customers simple integrated solutions for generation and consumption together with power savings agreements, time-of-use and spot market pricing.
- Real-time information from transformers and distributed resources will be used to optimize the distribution and transmission systems through improved congestion management, voltage control, balancing, flexibility, supervision, control and planning. Utilities and service providers will revisit their data platforms to support much larger data sets and real-time applications. For example, for some demand response applications data sets involve 450 times more data than smart meter data used for billing purposes.
- Software-as-a-service and cloud-based applications will gain more acceptance as reliability, availability and performance is demonstrated by early adopters.
- Security and privacy of consumers’ data concerns will require utilities and providers to demonstrate adequacy of their systems and procedures.
This transformation is going to pose new challenges for the utility industry. It will need better governance to ensure the timeliness, completeness, accuracy, and validity of the increasing volumes of data they will be collecting. This will require new proficiencies, Big Data specialists, data scientists, and business analysts, to define and deliver actionable insights and information security controls to protect sensitive information and deliver business results.
In my experience designing Big Data solutions in the utility industry, I found that widely-used legacy software was not sufficient for the increase in volume and velocity of data that utilities are facing. That’s what spurred me to investigate technologies used in other industries with similar problems, which led me to Kx Systems’ kdb+. Kdb+ is a time-series database platform with a powerful array-oriented language called q, that is well known in the financial services industry and is increasingly being used by utilities for its scalability and performance.
Time-series data analytics with kdb+ eases reporting for utilities to governments and regulators, as well as enabling faster research, billing and resource allocation decisions. These are the types of problems that today’s utilities are facing as they upgrade their systems.
I’m reminded of how similar this transformation is to what took place in the communications industry with deregulation. It resulted in rapid adoption of wireless technology, the introduction of smartphones and mobile applications, and a new range of pricing and billing options. We now live in a society where wireless service and our mobile phones are essential to almost every part of our daily lives.
There are also similarities to how the capital markets industry has transformed in the past two decades as voice trading has been overtaken by algorithmic trading, and many more trading venues have been created. Technological advances were required to manage the resulting massive market data flows. This transformation resulted in new technologies for analytics to keep up with trading, pricing, fraud and risk management requirements.
It was in this marketplace that kdb+ first found its home. The financial industry invested in kdb+-powered trading systems for large-scale streaming market data. Now this same technology is being adopted by the electricity, manufacturing and other industries, to manage their own Big Data.
There is an opportunity today for the electricity industry to reinvent itself around the consumer. By building more data-driven businesses and becoming early adopters of innovative services and practices, utilities can gain a competitive edge. This transformation will depend on having high-quality data and rich analytics to deliver actionable insights.
Przemek Tomczak is Senior Vice-President Internet of Things and Utilities at Kx Systems. For over twenty years, Kx has been providing the world’s fastest database technology and business intelligence solutions for high velocity and large data sets. Previously, Przemek held senior roles at the Independent Electricity System Operator in Ontario, Canada and top-tier consulting firms and systems integrators. Przemek also has a CPA and has a background in business, technology and risk management.