The Importance of Quality Data in Modeling

Modeling offers informative insight into the future of energy - with data as a foundation

The future poses a vast quantity of questions, and challenges to the energy industry.  Increasingly extreme weather events alongside stringent clean energy requirements bring resource adequacy and reliability into question.  New technologies are introduced on a regular basis, geopolitical events can alter things in an instant, and change is the one constant in the industry.  Modeling and simulation offer the ability to make informed decisions about the future of energy – both short and long term.  However, the foundation of that modeling and simulation is data. Quality data can make or break the accuracy of modeling and simulation.  

The purpose of modeling is to provide accurate and meaningful representations of system behavior in specific scenarios.  But, the outputs of these modeling exercises are only accurate and meaningful when fed by quality data.  When it comes to quality data – what does that mean, and what does it take to produce? 

Data Engineering Process

First, the data necessary must be scraped from all the sources required.  To run an energy model, the data required could include things like price forecasts, generator characteristics, nodes, demand forecasts, and resource expansion plans.  The challenge is that scraping the data is not all that’s required.  The data must then be transformed and engineered into standard and usable formats.  Data engineering is a laborious and time intensive process.  If you’re trying to create a dataset for an area that spans multiple regions, data is likely to come in a variety of formats.  Items such as generator names and information may not always match across sources. If the dataset spans multiple countries, data can also be published in different languages.  Additionally, the data may be missing information, or contain errors that must be rectified.  

Model Development and Calibration

Once data is transformed into a usable and standard format, the model must be developed within the specifications and timeframes required.  Building the model is hardly the last step though.  Following the model build, the dataset and model must be validated and calibrated.  During this step, it must be checked that the model and data produce results that accurately represent the behavior and characteristics of the systems being simulated.  In this phase of the process, past scenarios will be modeled to compare simulated results with real-world results.

Data Upkeep   

Armed with a calibrated model and dataset, you’re ready to run and analyze scenarios to fuel decision making.  The issue is, if you want to continue to use your model and dataset for an extended period (as is usually the case), data will need to be continuously updated so that it remains current and accurate.  So, the process outlined above is ongoing.  While never as time intensive as the initial model and dataset build, model and dataset upkeep require a considerable amount of work.  

Energy Exemplar's Data Team - Enabling More Strategic Use of Time and Resource

Quality data is absolutely essential to a good model.  Because of this, Energy Exemplar has a global data team, staffed by approximately 40 staff worldwide.  The Energy Exemplar data team works year-round to create and maintain datasets for our customers around the world.  By providing customers with dataset solutions, we allow them to focus their time on the work that really matters – modeling, analyzing, and preparing for the future.  Developing a dataset takes months' worth of work - and then it must be maintained.   

When organizations can outsource this time and labor-intensive work, time and resources can instead be used to focus on organization-specific challenges and initiatives rather than regularly gathering, cleaning, and updating the basic data essential to run a base model.  Organizations are then able to instead focus their time and resources on complimenting their models and datasets with proprietary portfolio data, and deeper analysis to enhance decision making.  

Every Energy Exemplar dataset goes through an intensive development process and is tested and calibrated before it is offered to customers.  Dataset solutions come with documentation on data source locations and cleaning methods, analysis on model assumptions, and proof of model calibration and validation – which is ongoing.  

Energy Exemplar's Data Offerings

Quality data is the foundation of meaningful modeling and simulation.  Without it, simulation results are meaningless and error prone.  To learn more about the datasets offered by Energy Exemplar, and how they can support accelerated decision making, click here.