In our last post, we were pleased to announce the release of the California Independent System Operator (CAISO) sponsored “Demand Response Advanced Measurement Methodology” statewide study resulting in the open-source FLEXmeter becoming eligible to clear demand response transactions in the CAISO market. 

Read the full FLEXmeter study here.

In this post, we explain a little more about FLEXmeter, why it is needed, and how an open source approach can give ISOs the confidence to deploy and integrate demand response and other behind-the-meter resources in order to solve evolving grid challenges.  

Background

California is facing a 4.4GW forecasted electric generation shortfall by the summer of 2022.  To avoid disruptions like those in the summer of 2020, the state must find ways to manage demand and use supply more efficiently, especially during peak periods. Demand response can help, but current methods for measuring the impact of this vital resource have failed to accurately capture its value. 

The problems with current approaches to measurement became clear in August of 2020, when CAISO was forced to institute rotating electricity outages in California in the midst of an extreme heatwave in the western United States. In order to help mitigate the crisis, the ISO called on demand response resources to help reduce demand and stabilize the grid.

A subsequent root cause analysis, however, indicated that there some demand response capacity during the critical event was unused, in part because resources were not able to respond to real-time conditions, but mostly because of bidding practices that “reduced the likelihood of demand response resources being selected in the day-ahead market, even on days with extremely high day-ahead demand forecasts.”

In order to address concerns about the accuracy of its methodology for measuring supply side demand response resource performance during the event, CAISO asked Recurve to expand an analysis it had done on demand response provider OhmConnect’s impact in MCE’s territory on the day of the event. After reviewing the results of the study, CAISO engaged Recurve to conduct a similar analysis statewide, refine the FLEXmeter methods, and standardize an approach that can be deployed at scale to enable its use as a performance measurement for the participation of supply-side demand response resources in the CAISO market.

The FLEXmeter methodology that Recurve developed is designed to fix endemic problems and provide a unified weights and measures to eliminate silos and integrate and harmonize demand response and energy efficiency as reliable and consistent resources to manage both short- and long-term grid challenges. The FLEXmeter takes advantage of smart meters, verifiable, open source methods and code, and Energy Differential Privacy tools to provide access to necessary data while rigorously protecting customer privacy.

The broader study included a wide range of demand response providers, covering both residential and commercial customers. The goal was to demonstrate that the open-source FLEXmeter methods (developed by Recurve through a stakeholder process in partnership with DOE, NREL, and MCE) could reliably assess demand response impacts on grid operations, particularly during times of extreme stress on the power system, and to provide more confidence in the ability of demand-side virtual power plants to rapidly balance loads and reduce potentially detrimental system peaks.

The purpose of this analysis was to:

• Understand and operationalize the baseline and comparison group methods in relation to existing guidance and practice;
• Understand barriers to data access and identify a viable path to overcome them;
• Understand the 2020 heat storm events based on the baseline and comparison group methods implemented by Recurve to inform and support decision-making.

The study showed that the FLEXmeter methods can provide a robust assessment of the performance of demand response resources in relation to similar actors on the grid and demonstrates the advantages of an open-source data-driven approach that combines hourly site-level models with automated non-participant GRIDmeter comparison groups.  

The study confirmed that resulting in a revenue-grade measurement of net impact to the grid is tariff compliant for calculating energy measurement of supply-side demand response and can therefore be used for settlement of transactions. 

"As the independent market operator, ISO seeks to settle demand response transactions transparently, appropriately, and fairly for their supply-side performance value. The FLEXmeter methodologies described in this report would allow the ISO to fulfill that objective consistent with the ISO Tariff."

Development of the Methods

To conduct this study, Recurve analyzed 38 demand response events spanning Investor-Owned Utility (IOU) and Community Choice Aggregator (CCA) territories across 11 climate zones, multiple demand response providers (DRPs), and more than 12,000 participating residential and commercial customers. 

The FLEXmeter produced high-quality site-based baselines by combining the CalTRACK methods anchored in the LBNL Time of Week Temperature Model, with comparison GRIDmeter groups. This approach successfully measured impacts even during extreme events with accuracy and predictability for providers and grid planners. The methods demonstrated consistent results across market sectors and resource types.

Transparent and standardized baseline and comparison group methods offer a path to improved accuracy, consistency, and reliability of the demand response measurement. The FLEXmeter methods will not only provide an accurate and reliable basis for measuring demand response impacts in the short term but can open the door for a new generation of programs focused on more continual resource deployment and integration. 

In its report to the California Independent System Operator (CAISO), the Baseline Accuracy Working Group (BAWG) found that “control groups consistently outperformed day and weather matching baselines.” Several of the BAWG’s recommendations for control group methodologies were subsequently adopted in the CAISO tariff governing demand response. 

Until now, however, the adopted control group methods have remained largely untested and to date have not been used for settlement. Without refining methods through implementation, the ISO tariff’s approved control group framework still leaves room for interpretation — and, therefore, uncertainty. Adding to these risks, data access constraints, particularly for third parties, have limited the opportunity to leverage comparison groups, which require non-participant data. 

Unlocking Data and Protecting Privacy

In order for comparison group approaches to work, parties need access to non-participant energy consumption data. At the same time, utilities must rigorously protect the privacy of this data using modern, mathematically rigorous methods.

In the past, non-participant data availability and data privacy concerns have proven barriers to the implementation of comparison groups in demand response measurement. However, data privacy methods and security practices have advanced in recent years. Privacy protections and rigorous data security practices can be put in place going forward to minimize the risk of customer re-identification.

Differential privacy, an emerging best practice in consumer protection that has been used recently by the US Census and Google among others, is a core component of the FLEXmeter. To learn more about how differential privacy is applied to protecting energy consumption data, see our blog series or check out the energy differential privacy website. 

FLEXmeter Study Findings

The FLEXmeter methods provide a robust assessment of the performance of demand response resources in relation to similar actors on the grid. 

This report demonstrates the advantages of open-source data-driven measurement that combines CalTRACK hourly site-level models with open-source automated non-participant GRIDmeter comparison groups, resulting in a revenue-grade measurement of net impact to the grid.

The California Independent System Operator commissioned this study to address four objectives. 

1. Understand and operationalize the baseline and comparison group methods in relation to existing guidance and practice

Recurve assessed the FLEXmeter comparison group methods in relation to the FERC-approved control group methodology outlined in the CAISO tariff. The FLEXmeter methods are specified to a greater degree yet are compliant with the tariff. The full summary of the methods comparison is provided in Appendix A of the report.  

2. Understand barriers to data access and identify a viable path to overcome them

Recurve demonstrated that with the proper data organization, frequency, and security protections in place, a centralized settlement model can both handle data securely and present results while mitigating the risk of re-identification. Standardized data specifications can streamline the execution of FLEXmeter methods going forward. Recommended specifications for both consumption and categorical data are detailed in Appendix B. 

Relying on data donors for non-participant data to enable this study meant that the full cross-section of the state was not available. Recurve recommends that state agencies coordinate to authorize the development of a centralized non-participant data pool to enable comparison group settlement methods. In addition, Energy Differential Privacy enhances customer data privacy while still allowing the extraction of the necessary information to support demand flexibility as a reliable CAISO market resource. 

3. Understand the 2020 heat storm events based on the baseline and comparison group methods implemented by Recurve to inform and support decision making; 

The 2020 extreme heat events are the subject of the measurements detailed in this study. Since the FLEXmeter methods are compliant with the existing tariff, distributed energy resource providers can opt to utilize them immediately for settlement with access to the necessary data. The methods are observed to work well across geographies and for both residential and commercial demand response, illustrating their viability for wider adoption.

This report can inform and support contemporary and future decision-making across state agencies. Since FLEXmeter methods are available as open-source frameworks, they could serve as a transparent foundation for assessing performance across market actors and agencies. Some near term considerations for each agency include: 

• The California Independent System Operator can recognize the CAISO tariff for performance settlement using comparison groups as a tariff compliant methodology for calculating energy measurement of supply-side demand response and refine its business practices in its application. The CAISO may additionally utilize the analysis to further explore forecasting or settlement of distributed energy resource demand reduction impacts.
• The California Energy Commission is currently hosting a staff workshop on resource adequacy qualifying capacity of supply-side demand response (CEC Docket 21-DR-01) that will inform the next resource adequacy proceeding. This analysis could serve to support inputs to the Effective Load Carrying Capabilities model.
• The California Public Utilities Commission is in the midst of considering how to address Summer Reliability for 2022 and 2023. This analysis may support a review of solutions adopted by the Commission to provide visibility to all agencies in understanding the impacts of demand flexibility on grid operations or in considering alternatives to the Load Impact Protocol process. 
  

4. Understand impacts of demand response events in 2021 and operationalize methods at scale.

While an analysis of 2021 has not been implemented at this point, Recurve’s 2020 analysis shows that the comparison group methods provide a reliable means for the assessment of demand response impacts. It is functional at scale and can be implemented across climate zones and sectors.  

CAISO can set the expectation that consistent, transparent, revenue-grade measurement for demand response and other flexible resources is a requirement for understanding grid impacts in the state. 

Report Recommendations

As a result of the outcomes of this report, the following recommendations were presented for the CAISO, CEC, and CPUC to operationalize comparison groups for demand response:

• CAISO should update their existing control group method tariff to add important detail and address minor recommendations in Appendix A of the report.  
• Cooperating state agencies should adopt the comparison group methods, approved for settlement in CAISO, and operationalized with the full-stack open-source codebase as the preferred performance measurement approach across demand response resources. This would ensure consistent, robust, and transparent measurement to underpin this important and growing resource statewide.
• The California Public Utilities Commission, in collaboration with the California Energy Commission and CAISO, should authorize secure data access to a non-participant pool for qualified vendors to allow this method to be used.

Learn more about how the methods work on the FLEXmeter website.

‍

‍