No More Black Boxes: Results from PG&E’s Commercial Whole Building NMEC Pilot

September 9, 2019

Normalized Metered Energy Consumption (NMEC) and open-source approaches are less expensive and performed better than modeled baselines and proprietary tools for evaluating energy efficiency in commercial buildings, according to a PG&E and SBW Consulting study.

The study’s goal was to test the reliability of site-level NMEC and to recommend the best methodology for estimating savings. To do so, it compared different approaches on twelve commercial building retrofits that were selected for model fit and level of savings.

Among other important results, the study’s key findings across both the SBW Consulting report and the final joint commission summary found that NMEC models consistently produce accurate estimates and that program administrators should use transparent, open-source, Option C based algorithms.

The SBW Consulting report “observed that the proprietary algorithms did not offer advantages over the open-source, public-domain (free) algorithms for verifiable savings estimation, while the open-source algorithms offer the key evaluability advantage of transparency.” 

It was also found that neither implementers nor vendors adequately monitored behind the meter non-routine events.

When comparing IPMVP Option C (metered) vs. IPMVP Option D (modeled) baselines, it was found that cost was 2-3x greater to follow Option D and also resulted in significant uncertainties in savings, and particularly poor estimation of daily energy consumption profile (meaning not well suited for hourly savings).

Images above were from public presentations given at the California Demand Management Council (CEDMC) EM&C Conference on January 31st, 2019 by Alison Erlenbach of PG&E, the SBW Consulting report and the joint study report authored by PG&E and the CPUC. This study was a single study looking at specific software tools, vendors and buildings; results should be interpreted in that context.

Recurve’s Take:

These results are consistent with our team’s experience. The cost and complexity from traditional and often bespoke M&V can result in significant bias and less accuracy while increasing cost and uncertainty.

These findings are critically important for any site-based NMEC program design, particularly those that incorporate business models in which payments based on M&V results are made directly to building owners or to their tenants through energy service agreements. In all of these cases, the definition of performance must be clear and transparent so that cash flows can be forecast and managed and all parties have the confidence to participate. 

In other words, there is too much risk for market actors and stakeholders if the method to determine savings (and its implementation) is not known upfront and fully transparent and verifiable. Savings can vary significantly based on an evaluator’s choice of normalization procedure, which variables to include in a model, how to implement that model, how to identify and handle events, and even the process for cleaning and organizing data.

Recurve believes that the path to scale in commercial energy efficiency begins with the standardization and automation of routine M&V. Transparency and open models should replace black boxes and complex custom calculations, and real consumption data should be used wherever possible over models and assumptions.

When non-routine events do occur there should be a pre-approved contract that defines the parameters of non-routine adjustments so they can be managed and enforced consistently through an enforceable contract (for more details on how this can work, see Bankable M&V for Commercial Buildings).

If it were your business on the line, would you leave your cash flow up to a black box? If you are a utility, are you ready to hand over the keys to the cash register?

With the CalTRACK Methods and open-source OpenEEmeter, everyone can measure with confidence.

Let’s stop arguing over bespoke models and grow the DSM pie together. 


The Recurve Platform implements the open-source CalTRACK Methods and OpenEEmeter and allows utilities, regulators, and aggregators to have confidence in savings outputs through built-in verification to focus on the customer and improving outcomes rather than arguing over M&V.

We would love to discuss how we can help. Schedule a discussion with our expert team.

Learn how to enable transparency in your savings results

It's time to end black box M&V. Contact us to learn how to enable transparent results for your program.

Let's start a conversationLet's start a conversation

Get the latest Recurve articles, updates and webinars directly to your inbox.

Subscribe Now
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Support Center
Contact Us