How it works

Comparison Group Identification

The Need for Comparison Groups

Meter-based approaches for determining energy savings have the potential to re-invent energy efficiency by allowing it to scale and making it more responsive to a rapidly changing grid. However, to function as a grid resource, the procurement of energy efficiency has to account for factors that impact energy usage outside the influence of a program, such as economic cycles, natural adoption of new technology, or other population-level changes.

The net impact of energy efficiency for meter-based performance procurement must reflect the incremental effect of the known intervention above and beyond population trends of energy consumption. It is also essential that this analysis can run longitudinally and transparently as efficiency is deployed, to deliver the greatest value to the system by creating a feedback loop.

The Two-Stage Approach

Estimates of energy consumption changes using the core CalTRACK methods do not include the effect of unmeasured factors on energy consumption. Quasi-experimental approaches offer one way to control the effect of exogenous trends in energy use on savings estimates. For example, using a two-stage approach, site-level models may be fit to treatment and comparison group consumption followed by a difference-of-differences calculation to estimate the savings net of exogenous trends and other market effects, including savings from codes and standards, midstream and upstream programs and natural adoption of energy efficiency. Using a difference of differences with a comparison group allows for the longitudinal tracking of both site-level savings and the impact on load, net of population trends. These kinds of automated approaches are appropriate for meter-based efficiency because they can be deployed up front and are replicable and accessible to all players. Combining them with auditable custom and non-routine adjustments maintains the consistency of meter-based calculations and is a necessary prerequisite for creating the confidence needed to attract capital, manage risk, and establish pay-for-performance markets and ultimately scale energy efficiency investments.

The Recurve Platform uses automated site-level comparison group matching to select non-participant groups to account for natural population-level consumption changes. The default method used by the platform is monthly consumption matching, although other methods and even custom comparison groups can be accommodated, if needed. In monthly consumption matching, a comparison group is constructed by selecting n matches (e.g. n=5) from the comparison group pool with the shortest distance d to the treatment group customer under consideration. The distance d is, in essence, a way to reduce 12 monthly consumption differences between any two customers to one metric. By default, the Recurve Platform uses Euclidean distance, for its simplicity and intuitiveness. When feasible (e.g. for retrospective analysis), it is also recommended to simultaneously use multiple comparison group identification methods (e.g. future or past participants), which provides more stable results.

Support Center
Contact Us