By 2020 Gartner analysts estimate an astronomical number of 25 billion devices will be connected to the internet. Dubbed as the internet of things (IoT), a sizeable portion will be sensors. In the energy and utilities industry, these sensor devices will comprise of meters, gauges, and thermostats attached to energy turbines, oil drills, smart home appliances, solar panels, power stations, and windmills—all as part of a smart grid. They will act like incessant, inaudible chattering sentinels spewing electrical impulses as data streams.
In this savannah of raw data streams are invaluable insights. By using machine learning techniques to implement predictive statistical models, data scientists can discover valuable insights and illuminative patterns from these data streams. Utility companies can use these results to deliver efficiency energy usage, leading to smart decisions and cost savings.
Without using machine learning predictive models and real-time detection of defective devices, you cannot discover trends. Without discerning trends you cannot deliver energy savings.
The Future is here for Smart Grid
Yet you don’t have to wait until 2020. Today, utility companies are employing Apache Spark and MLlib algorithms to tap into sensor data to capitalize on savings. As well building predictive models, managing and optimizing assets, and monitoring the general health of the sensors and grid, data scientists at energy companies can predict potential faults and prevent failures in their electrical grids by garnering, extracting, and examining data from various sources, as depicted in the diagram.
Source: http://smartgridcenter.tamu.edu/sgc/web/wp-content/uploads/2016/03/grey-border-BigData_illustration_v5.jpg
Harnessing the power of machine learning
Thomas W. Dinsmore explains what machine learning (ML) can do for your business. By citing various use cases, he examines how ML algorithms can detect cancer pathology better than trained human reviewers; classify paintings into historical genres quicker than art curators; detect fraudulent credit card transactions faster than humans; and recognize voice and image rapidly than we do, across vast array of images and audio files.
However, these ML techniques can be extended to other sectors, argues Bruce Harpham in “How data science is changing the energy industry.” Harpham suggests that innovation and use of ML techniques borrowed from other sectors that Dinsmore above refers to can be transferred to the energy sector. He elaborates on how energy sector is already building smart saving grids.
For example, Francisco Sanchez, president of Houston Energy Data Science, says that “The energy industry has recently started to adopt the survival analysis concept from the medical field.” Used in medicine, the survival analysis is a statistical method to predict survival rates for patients based on their existing conditions and treatment program.
This model is now being applied to predict field equipment health in the oil and gas industry, Harpham writes, enabling early detection of defective equipment, preventing failures and outages, and improving reliability of energy consumption.
Another case in which machine learning techniques that defeated Chess and Go masters are being used to conduct, coordinate and conserve excessive energy for flexibility and demand-site capacity. Michael Bironneau, head of technology development at Open Energi, discusses how these ML techniques are employed to create smart energy grids.
In particular, Bironneau explains that one of the benefits of ML techniques and algorithms comes from “unlocking and utilizing flexibility” during high and low energy demands. Through data analytics from the sensors, such algorithms engender ways to redirect or reschedule power consumption, hence balancing the grid and minimizing waste of energy.
But the key here, Bironneau emphasizes, is that with “sufficient [harnessed] data, a ML model can look at a sequence of actions leading to rescheduling of power consumption and making grid-scale predictions.”
All this indicates that ML techniques, used elsewhere in predictive analytics for medicine, customer churn in the retail sector or defeating strategic games, can be applied toward building smart energy grids. Using sensor data from drills, energy turbines, smart house appliances, and windmills, data science and ML can be used to construct smart energy grids, and through data discovery, it can deliver smart energy.
Two Uses Cases of Machine Learning in Smart Energy Grids
The unstoppable rise in connected devices, and the volume of data generated, has engendered a demand for advanced analytics using machine learning techniques to mine data. Whether for predicting, monitoring, or identifying patterns and trends of energy usage, some utility companies have embraced this imperative to capitalize on three aspects of advanced analytics to gain real advantage: Disruptive energy insights; the digital customer; and the energy integrator.
Let’s look at two energy utility companies that incorporate data analytics in their smart grid.
Consider DNV GL, a consulting firm, guides energy, maritime, and oil and gas clients to build safer smarter, and greener grids. Using Apache Spark MLlib, the firm has developed machine learning models and analytics services to analyse massive amounts of sensor data, combined with weather data, to predict, gauge, and temper energy demands, both for natural gas and electric usage.
At Spark Summit East 2017, Jonathan Farland, senior data scientist at DNV GL, shared how they built two machine learning models with Spectral Clustering and K-means and Semiparametric Regression algorithms. These models analyzed real-time and historical sensor data combined with weather data, taking into account humidity, cloud cover and solar radiance, to study consumer power consumption behavior and predict demand-capacity.
The advent of smart meters [and sensors] has allowed for hourly and sub-hourly observations of electricity demand data,” said Farland. “Before this, energy consumption records were more or less limited to monthly or bi-monthly billing records.”
As well as detecting faulty devices and studying usage behavior, these predictive models measure impacts that prevailing weather conditions and other factors bear on energy demand. Such an ability to predict the ebb and flow of energy capacity throughout a smart grid can shed significant cost savings for consumers.
British Petroleum (BP) is another company that has borrowed innovative machine learning ideas from another sector and applied them to oil and gas sector. Driven and motivated not to repeat the company’s 2010 Deepwater Horizon disaster, BP has invested heavily in data science and data analytics. Its investment starts with high quality data and monitoring capabilities. Over 100 offshore wells feed continuous sensor data to a high-performance computing center in Houston, TX, providing field technicians live data to react should data reveal any predictive insights.
Drawing from talent of data scientists from local universities, BP has built machine learning models used in the oil and gas sector to predict equipment failures and explore drill conditions, saving time and costs for the end consumers. For example, “Survival analysis is used to predict the maintenance requirements for field equipment such as compressors through monitoring and modeling,” Francisco Sanchez says, president of Houston Energy Data Science. Through a proactive action enabled by machine learning and data science, BP engineers can repair a damaged oil well in a day, instead of three days. This translates to energy savings, costs, and efficiency.
But at the heart of smart oil wells is a smart grid of devices and sensors incessantly collecting data and feeding it back to the mother-ship for data analytics. “As an organisation, as an industry, we are increasingly putting more sensors into our facilities, on rigs, wells and pipelines, for example, to measure temperature, pressure, chemicals and equipment vibrations,” says Paul Stone, decision analytics network leader. “The biggest benefit of analytics is that it provides the opportunity to predict what will happen, instead of recording what has happened or is happening.”
Without data and analytics, and without data science and machine learning techniques, BP can neither predict production failures nor proactively repair damaged oil rigs. Nor can it deliver oil or gas at reduced prices to its consumers if the cost of production is high.
What’s the future of energy and data science
Don’t wait until 2020, start today, for the future is here. We saw two cases in which Apache Spark, data science and machine learning algorithms borrowed from other sectors successfully applied to energy sectors to drive down energy production costs and deliver energy smartly. In both cases, connected sensors comprised the smart grid, generating big data used for analytics.
Not unique to the above two cases, the adoption of innovative technologies is widespread in the energy sector. Both companies above aren’t so much jumping on the bandwagon of internet of things, big data and machine learning hype as indicating true and tangible transformation and the momentum in building smart energy grids.
“Momentum in the development of data-led [analytics] oil and gas technologies and adoption of these systems will undoubtedly continue to increase,” says Nav Dhunay, President & CEO at ambyint, in “Big Data’s Big Impact Coming to Oil and Gas,” seen as more evidence of this adoption.
In short, big data is a critical element in discovering insights and delivering smart energy. And data science, using Apache Spark and MLlib for advanced analytics, enables smart consumption and conservation of electrical or renewable energy grid.
Contributed by: Jules S. Damji is an Apache Spark Community Evangelist with Databricks. He is a hands-on developer with over 15 years of experience and has worked at leading companies building large-scale distributed systems.
Sign up for the free insideAI News newsletter.
Speak Your Mind