Jack's blog

PhD thesis submitted!

Last night I submitted my PhD thesis on "Disaggregation of Domestic Smart Meter Energy Data"! Hurray!

May 2016 release of UK-DALE

I've just updated my UK-DALE dataset with the latest data. House 1 now has 3.5 years of data!

Please help design a competition for energy disaggregation algorithms!

Has disaggregation accuracy improved since the 1980s? Which algorithms are most accurate for a given use-case? Which (if any) use-cases are well served by NILM already?

It's pretty much impossible to answer any of these questions with confidence (unless you only consider the tiny number of algorithms for which you have access to executable code). We can't directly compare published results across papers because, when testing the disaggregation accuracy of NILM algorithms, each paper uses different datasets, different metrics, different pre-processing, etc.

This means that we can't measure progress over time. Nor can we decide which NILM algorithms are most promising and which might be dead-ends.

These are bad problems. Let's work towards fixing them.

Some other machine learning communities have had great success running yearly competitions. For example, the ImageNet "Large Scale Visual Recognition Challenge" has been running yearly since 2010. Some regard this competition as having played a crucial role in the recent dramatic increase in the accuracy of image classification algorithms.

The idea of running a NILM competition has been rumbling around for several years. But designing and implementing a NILM competition is hard. The community uses sample rates ranging from monthly to MHz. No single metric is informative for all use-cases. Collecting ground truth data (the power demand of individual appliances) is expensive and time-consuming.

Maybe we can pull this off. The first step is to decide on a design which will work for everyone.

To give us something concrete to debate, we'll outline one way this could work. This is not meant to be definitive! Think of this as the DNA for a clumsy, inefficient animal 500 million years ago. Together, we need to evolve this design into an elegant, efficient beast, well adapted to its environment.

Please shoot holes in this proposal! What won't work for you? What's impractical? What's unfair? What opens the competition up to cheating? How can we make the competition more attractive to researchers? How can we make the competition more informative for the community? How can we simplify the process?

The draft proposal is available on Google Docs. I've linked to a Google Doc rather than copying-and-pasting the proposal into this post so that we can update the proposal as the discussion develops. Please add your comments either to the mailing list discussion; or to the Google Doc (please sign your comment with your name; unless you deliberately want to be anonymous); or if you want to keep your comment private then email me.

Thanks, (in no particular order) Jack, Mario, Oli, Stephen, Grant, Marco, Peter

Books with strong female characters suitable for 4 to 5 yearolds

A couple of weeks ago I wrote a rant on facebook where I asked for recommendations for "books with strong female characters which are suitable for 4 to 5 yearolds. Characters who can think for themselves. And get shit done. Without a prince. Or without trying to impress a prince."

I got loads of great suggestions from my friends. And these suggestions came in several different ways (chatting over coffee, facebook comments etc). So this blog post an attempt to maintain a list of links on the topic of "books with strong female characters".

Portable air quality monitors for cyclists

I cycle 14 miles each day in London (7 miles each way). I'm getting increasingly worried that I'm breathing in lots of bad stuff. I love my cycle ride into work: It's the best way to get my brain working in the morning. So I'm not planning to stop cycling. But I would like to figure out which route into work is least polluted.

Simplifying NILMTK

NILMTK is now over two years old. Having had a chance to use it as a user (rather than a developer), and also having had a chance to take a step back from NILMTK development, it feels like there are quite a few opportunities to simplify the NILMTK code base, without modifying the public API much (although the public API probably could also do with some tidying up - but if we were to do that then we'd be careful to slowly make functions deprecated rather than just strip stuff out straight away).

The NILM issue queue now has a 'simplify' label to indicate which issues are to do with, well, making NILMTK more simple! The two main ideas are:

  1. Replace NILMTK's out-of-core code with Blaze
  2. NILMTK should interact with other Python tools more smoothly

Please do let me know your thoughts (ideally on the issue queue rather than on this blog post).

The core NILMTK developers are really busy at the moment so these ideas definitely won't be implemented any time soon - and may never be implemented. But it would be great to hear your opinions. And, of course, we'd try very hard not to modify the public API unless it really needs to be modified.

Current Cost and EDF EcoManager RF protocol docs are back online :)

Back in 2012, a few of us hacker folks chipped away at reverse-engineering the radio protocols for Current Cost and EDF EcoManager electricity meters. We kept our notes on a github wiki. Then Current Cost asked us to take those docs down. But, today, Current Cost very kindly allowed us to put the docs back online! Specifically, these two docs are now back online:

Have fun :)

3rd International Workshop on NILM -- SAVE THE DATE!

Dear NILM researchers,

The 3rd International Workshop on Non-Intrusive Load Monitoring (NILM) will be held in Vancouver, Canada from May 14 to 15, 2016. The venue for the workshop is still under consideration. Last workshop was held June/2014 at the University of Texas, Austin, in Austin, TX.

Fix headphone audio output on Linux on HP ProBook 6450b

The headphone output on my HP ProBook 6450b on Ubuntu Linux stopped working a few updates ago and I think I've found how to fix it:

cd /usr/share/pulseaudio/alsa-mixer/paths
sudo cp analog-output-headphones.conf analog-output-headphones.bak
gksudo emacs analog-output-headphones.conf

Turn on the master volume (it was 'mute'):

[Element Master]
switched = on

Convert Reveal.js + D3 to PDF

The last two presentations I've given have been created using reveal.js and d3.js. This combination of technologies allows for some nice data-driven animation. For example, my BuildSys presentation on Neural NILM is available to view here and the source code is here. I was honoured to learn that my BuildSys presentation won joint best presentation!


Subscribe to RSS - Jack's blog