Does disaggregated electricity feedback reduce domestic electricity
A systematic review of the literature
(Swipe or press right-arrow on your keyboard to change slides)
Andrey / shutterstock
Evidence that NILM can help save energy...
1) People want disaggregated energy data
2) Behaviour affects energy consumption
modifying behaviour → reduce energy consumption
3) People are bad at estimating
the energy consumption of their appliances
→ Fix the ‘information deficit’ then users can
operate as rational ‘resource
(I’m now sceptical of this idea)
4) Multiple studies report that disaggregated
feedback reduces energy consumption
5) Smart meters
- Common in
medicine, social sciences etc.
- Distinct from ‘narrative’ reviews
- Aim to collect all papers matching a defined search criteria
- Quantitative summary of each paper and biases
- Quantitative synthesis of all results
Background image from UCSF
- Three search engines: Google Scholar,
the ACM Digital Library and IEEE Xplore
- Search terms:
- ‘disaggregated AND
[energy|electricity] AND feedback’
- ‘N[I|A|IA]LM AND
- Searched papers’ bibliographies
- Sent draft literature review to
authors for comments
12 groups of studies identified
Q1. Can disaggregated electricity feedback enable ‘energy enthusiasts’ to save energy?
- Very likely...
- Weighted-mean energy reduction = 4.5%
- A lot of uncertainty...
The Hawthorne Effect
- Hawthorne effect is illustrated by
Schwartz et al. 2013:
- Randomised controlled trial
- 6,350 participants split into 2
groups: control & treatment
- Treatment received weekly
postcard saying: ‘You have been selected to be
part of a one-month study of how much electricity you
use in your home... No action is needed on your
part. We will send you a weekly reminder postcard
about the study...’
- Treatment group reduced energy consumption by 2.7%!
- Failure to control for Hawthorne very likely to be
strong positive bias
- 8 studies did not control for Hawthorne
- 6 studies used attention-grabbing
- 10 studies were short (4 months or
- Cherry-picking statistical analyses
and comparison periods?
- 8 studies used sub-metered data,
hence avoiding mistrust from participants
- Publication bias?
Q2. How much energy would the whole population save?
- All 12 studies suffer from ‘opt-in’ bias
- Subjects self-selected
hence are probably more interested in energy than the average person
- Very likely to be a strong positive bias
Q3. Aggregate versus disaggregated feedback
- 4 of the 12 studies directly
compared disaggregated against aggregate feedback
- 3 studies found aggregate to
be more effective
- 1 study found aggregate to
be equally effective
- 2 field trials & 2 lab experiments
- IHD: 8.1% (statistically significant)
- Disaggregation: 0.5%
- Control: -2.5%
Findings from surveys:
- Follow-up survey revealed that the
disag group were not significantly more likely
to be willing to replace large, inefficient appliances
compared to controls or IHD group.
- Neither controls nor the disag group
significantly increased their perception of control
(initial survey versus follow-up).
- IHD group did increase
their perception of control.
Findings from surveys:
- Users viewed their devices:
- 0.86 times per day for disag users
- 8.16 times per day for IHD users
PG&E 2014 trial results
- IHD users significantly more likely
to report taking actions to reduce electricity usage
and to use their device to deduce power demand of
- Several users did not
trust the disag data.
- IHD more successful in communicating
power demand now
Bidgely have redesigned their website since these studies
- NILM has many uses! This talk just considered one use!
- Available evidence suggests that
aggregate feedback is more effective than
- But these results
confounded by effect of IHD versus website
- Disag feedback might drive savings
of 0.7% - 4.5% in general population
- Disag feedback might drive larger savings
in ‘energy enthusiast’ populations
- Fine-grained disag may not be
- But! Lots of gaps in our knowledge.
Cannot robustly falsify any hypotheses yet.
Suggestions for future studies
- Compare aggregate versus disagg
(both on an IHD)
- Compare 2 groups:
- Aggregate on an IHD
- Aggregate (on an IHD) + disagg (on a website)
- Compare fine-grained disag versus
- If you have data then please consider
releasing it; or writing a paper; or collaborating with
someone who will write a paper with you!
Users might become more interested in disag feedback if:
- Energy prices increase
- Concern about climate change
- Disag accuracy increases or if
designers communicate uncertain estimates
- Lots of ideas in the literature
about how to improve disag feedback. e.g. disag
by behaviour; or display feedback near
appliances; or provide better recommendations etc.