Energy consumption patterns at the building level are generally difficult to model, due to the impact of human behavior. While statistical models can control for observable factors such as weather and occupancy patterns, human behavior is more stochastic and difficult to measure. However, energy consumption over portfolios of large numbers of buildings tend to be much more stable and manageable. Portfolios are generally composed of buildings that share common characteristics (e.g. building type, location, installed measures etc.)
The uncertainty in model predictions can be expressed through the savings uncertainty at a specified confidence level. When this is expressed as a percentage of the average metered savings, it is referred to as Fractional Savings Uncertainty (FSU). Smaller FSUs indicate higher confidence in the savings estimates. For example, a building or group of buildings that have saved 1000 kWh on average with a 10% FSU, will have a confidence interval between 900 and 1100 kWh, while a different group of buildings with the same average savings and 50% FSU will have a confidence interval between 500 and 1500 kWh. ASHRAE Guideline 14 specifies that the maximum acceptable level of fractional savings uncertainty at the building-level is 50 percent (at the 68 percent confidence level). CalTRACK tests have shown that aggregation of individual projects, can, in most cases, deliver less than 25 percent portfolio-level savings uncertainty (at the 90 percent confidence level) with very reasonably sized portfolios. It must be noted that, by its nature, FSU decreases with larger portfolios and deeper savings.
CalTRACK empirical tests found that NMEC models can achieve promising uncertainty thresholds at a portfolio level for both residential portfolios and most types of small and medium commercial buildings, without the need for extensive site-level non-routine adjustments on most buildings. The utility and ratepayer will almost always have sufficient portfolio assets to ensure that mutual benefits are being delivered with enough confidence to assess portfolio level impacts, without requiring that adjustments on each building be tracked, counted, and regulated and without having to micromanage the details of each project. Moreover, site-level thresholds on model fit metrics (e.g. CVRMSE) tend to exclude a large proportion of potential candidates for energy efficiency retrofits, while offering little to no increased confidence in portfolio-level savings.
While the aggregated approach offers a pathway for scaling energy efficiency in certain types of buildings, it is not a panacea and is not suitable in sectors with fewer buildings or where the buildings have a relatively high rate of energy consumption. CalTRACK includes the following set of recommendations for applying program rules using an aggregated approach:
BPA report on uncertainty
ASHRAE Guideline 14