Domestic power consumption data on github

I've just put all my existing smart meter data on github.

This dataset isn't especially useful for NILM work yet because I don't have a "ground truth" record of each appliance's state change.  This will change when I install my 24 individual appliance monitors.

Monitoring individual appliances

For some time I've been monitoring my home's aggregate power consumption using a CurrentCost EnviR.  I'm now planning to upgrade my monitoring hardware.  Firstly, I want to install CurrentCost Individual Appliance Monitor plugs on my appliances (£13.33 each).  Secondly, I want to measure aggregate real, reactive power and voltage using an Open Energy Montitor.

List of appliances to monitor

(each Current Cost ENVI display can only cope with 9 IAMs)


A (livingroom)
  1. TV
  2. Amp
  3. Subwoofer
  4. HTPC
  5. Washing machine
  6. ADSL modem
  7. Livingroom lamp1
  8. Livingroom lamp2
  9. Livingroom lamp3
B (livingroom)
  1. Bedroom1 lamp1
  2. Bedroom1 lamp2
  3. Bedroom2 lamp
  4. Bedroom DAB radio etc
  5. Hair dryer
  6. Hair straighteners
  7. Iron
  8. Hoover
C (in study)
  1. Toaster
  2. Kettle
  3. Coffee Maker / Bread Maker
  4. Microwave
  5. Fridge
  6. Kitchen Radio
  7. Dish washer
  8. Kitchen lamp
D (in study)
  1. Laptop
  2. Desktop
  3. 24" LCD
  4. Office HiFi
  5. Study lamp1 & lamp2 (sharing a plug)
  6. Printer
  7. GigE switch
  8. Fan
  9. Battery charger


Update 21/6/2012

I bought 3 CurrentCost Individual Appliance Monitors to test.  They seeem to work well.  My main concern was that the wireless range would be too short to allow me to monitor every appliance in my house but the wireless range seems fine.  Sure, the system drops a few more samples from the wireless monitor that's furthest from the CurrentCost EnviR but the data is entirely usable.  I've modified my Python logging script to handle multiple sensors.

Python notes

Documenting code

Python libraries

    Paper accepted into Imperial College Energy and Performance Colloquium 2012

    My submission to the Imperial College Energy and Performance Colloquium 2012 has been accepted. It's just an extended abstract which briefly outlines some ideas for my PhD research.

    The paper is:

    • Kelly J, Knottenbelt WDisaggregating Multi-State Appliances from Smart Meter Data. Imperial College Energy and Performance Colloquium. 29 May - 1 June 2012.  PDF


    Smart electricity meters record the aggregate consumption of an entire building.  However, appliance-level information is more useful than aggregate data for a variety of purposes including energy management and load forecasting. Disaggregation aims to decompose an aggregate signal into appliance-by-appliance information.

    Existing disaggregation systems tend to perform well for single-state appliances like toasters but perform less well for multi-state appliances like dish washers and tumble driers.

    In this paper, we propose an expressive probabilistic graphical modelling framework with two main design aims: 1) to represent and disaggregate multi-state appliances and 2) to use as many features from the smart meter signal as possible to maximise disaggregation performance.

    A new language for mathematical computing: Julia

    Julia is a high-level, high-performance dynamic programming language for technical computing, with syntax that is familiar to users of other technical computing environments. It provides a sophisticated compiler, distributed parallel execution, numerical accuracy, and an extensive mathematical function library. The library, mostly written in Julia itself, also integrates mature, best-of-breed C and Fortran libraries for linear algebra, random number generation, FFTs, and string processing. 

    More info: The Julia Language and Why We Created Julia and A Matlab Programmer's Take on Julia.  Sounds pretty awesome.

    Incidentally, the third link includes a quote which pretty much exactly captures my current feelings about Matlab:

    The Matlab language is slow, it is crufty, and has many idiosyncracies... I strongly disagree, however, with the opinion, common among some circles, that Matlab is to be dismissed just because it is crufty or "not well designed". It is actually a very productive language that is very well suited to numerical computing and algorithm exploration. Cruftiness and slowness are the price we pay for its convenience and flexibility.

    I fundamentally disagree with the last statement though.  Cruftiness and slowness should not be the price we pay for convenience and flexibility.  Matlab could've been designed to be both high-performance and productive.  For example: one source of slowness and cruftiness is that objects are usually passed by value, not by reference (yes, I know MATLAB does copy-on-write... which is great... until you want to write to an object).  I think that defaulting to pass-by-value is simply a design mistake.  Pass by reference wouldn't prevent MATLAB from doing the things it does, and would make it faster.

    Awesome stats, machine learning & information theory videos on YouTube

    I'm still very much enjoying the Coursera / Stanford Probabilistic Graphical Models course but occassionally I need to turn to another source to help explain the concepts.  I've just re-descovered MathematicalMonk on YouTube.  He has over 200 videos on machine learning, information theory and stats.  The videos I've sampled so far have been excellent.  Very lucid. 

    Added list of academic writing

    Just a very quick note to say I've started a list of my academic "publications".  It's pretty anemic at present.  But hopefully that'll change soon!

    Summary of "green" features we've added to our house

    Our house is a solid-walled house built around 1905.  Being end-of-terrace, it used to be very cold in winter.  We've gradually insulated over the past three years.  In terms of thermal performance, the house should now perform roughly on a par with a new build.  The majority of the work has been insulating the walls.  I did the bedrooms, living room and dining room and we used builders to do the bathroom.  In total, the energy-saving measures now installed include:

    • 65-80mm of rigid-foam insulation on all external walls (mostly DIY; some done by builders during other work)
    • at least 270mm of glass-wool insulation in the loft (DIY)
    • insulated the suspended timber floors in the living room and dining room (DIY)
    • we worked with a local sash window maker to put high performance double glazing units into wooden frames for the front of the house
    • lots of draught proofing and a focus on airtightness during the DIY refurbishment
    • mechanical ventilation with heat recovery in the bathroom (it works very well)
    • fitted wet underfloor heating in the living room (DIY).  UFH is wonderful!
    • solar thermal (evacuated tube) fitted professionally (would have done it DIY if it weren't for the new regs)
    • light pipes to bring natural light into the kitchen and corridor (installed by builders)
    • home-made 450 litre rain water tank in back garden, with piping running under living room floor to bring rain water to front garden
    • thermostatic radiator valves on all radiators; new condensing boiler with walk-about thermostat (which is great)... plan to install room-by-room digital radiator controls

    Overall it has been a lot of work and at times it's felt overwhelming.  But we're pretty much finished with the insulation and there's absolutely no question that the house is considerably easier to heat and more comfortable than it was.

    Getting LaTeX and Lyx to use ACM SIG class file

    Installing the ACM SIG LaTeX class file on Ubuntu using tex-live2011 and using it in Lyx.

    First, download the ACM class file and let LaTeX know about it (modified from Ubuntu wiki):

    MATLAB notes

    Just some random notes about MATLAB.


    Subscribe to RSS - blogs