Current Cost and EDF EcoManager RF protocols almost fully decoded

Thanks to the enormous help of Graham Murphy, Matt Thorpe and Paul Cooper, the wiki pages for the EDF EcoManager RF protocol and the Current Cost RF protocol are nearing completion. Of course, jump in if you have anything to add. Anyone with a github account can contribute.

And my EDF EcoManager C++ AVR code is ticking along. Still some distance from being usable "in the field" but getting there.

One quick random thought: for those of us who have been tinkering with the Current Cost RF protocol, it appeared rather odd that the data is "manchesterised". It occurred to me this afternoon that we can take advantage of this structure in the data to validate the data. A little more discussion on the wiki.

UPDATE 30/10/2012

Current Cost have asked me to remove the protocol documentation from the wiki. More details here.

UPDATE 5/1/2016

Current cost have kindly allowed me to put the docs back onto the github wiki!

Comments

Thanks for the credit!
It sounds like a great idea to check that the manchester encoding is valid, but I've found that it doesn't help much. It would have been much nicer to have a CRC or even forward error correction (FEC) in there. Ok, so the CRC would make more sense for this application perhaps.

The EDF implementation makes more sense to be honest.

Jack - great work - am I right in thinking all the CC IAMs have to be paired first with an EnviR, so they are assigned an address, or not? If so, what happens if need more than 10 IAMs ?

Anyone else - if you get chance to provide more info on newbie instructions for Arduino IDE code, rather than Jacks hex, would be much appreciated!

Hi Andy,

Thanks for dropping by.

To answer your questions:

"[Do] the CC IAMs have to be paired first with an EnviR, so they are assigned an address?"

The short answer is that no, I don't think you need to pair a CC IAM with an EnviR prior to using them with a DIY RF receiver like mine. I'm pretty sure CC IAMs are delivered with an ID already enabled. I think the "tuning" button just prompts the CC IAM to pick from a small set of possible IDs. I think each CC IAM can pick from about 3 possible IDs. The CC pairing process does not require an ACK response from the EnviR (the EnviR only has an RX and each CC IAM only has a TX).

Note that the pairing process is completely different with EDF IAMs. The EDF IAMs require an ACK from the base station during the pairing process (which my EDF code will soon mimic).

what happens if need more than 10 IAMs

The short answer is that I'd highly recommend that you don't try to use more than a handful of CC IAMs in a single house. CC IAMs suffer quite badly from RF collisions. The more CC TXs you have, the higher the chance of a collision. I've seen "blackouts" of single channels lasting an hour or two when I tried using 30 CC IAMs in a single house.

If you really want to use >10 CC IAMs in a single house then you can use multiple EnviR receivers (which is what I tried. My iam_logger python code logs data from multiple EnviR devices). But you need to manually check to make sure that each RF ID is only assigned to a single IAM (my iam_logger code does this check across an arbitrary number of EnviRs).

My recommendation would be that if you want to use lots of IAMs then investigate the EDF IAMs. These don't suffer so much from RF collisions because the EDF IAMs politely wait to be asked for data.

if you get chance to provide more info on newbie instructions for Arduino IDE code, rather than Jacks hex, would be much appreciated

I do plan to convert my code to be usable with an Arduino IDE soon. If you can't wait then you can try robomotic's port of my code to Arduino IDE available on github. (currently untested). If you do try robomotic's code then please let him know how you get on.

Caution!
My experience with the CurrentCost digital dev boards was that the address was set to 0xFFF when they first turned up. Effectively, the address was stored in the last few bytes of the EEPROM, and from factory, it was blank. If this is the same with the IAMs, then you might find they transmit on the same address.

Or they might not - I have no experience with them to know!

Ive currently got 6 CC IAMs through the EnviR, and posting to COSM through a Home Automation Hub.

Im not noticing any collisions (as evidenced by loss id data only), but resolution is only 6 seconds, so might be too large? Ive also got 15+ other transmitters (emonTXs) in same frequency blasting out at nearly real time - how are you detecting collisions?

Hiya,

Startup ID
First of all, to address the question of the IAM's start-up ID: I completely defer to Graham's experience. If Graham has found that his dev boards use an ID of 0xFFF when brand-new then it's probably safest to assume that the CC IAMs do likewise. My post was perhaps a little over-confident: I should've said that I haven't actually sniffed any data from a brand-new IAM, I was just guessing based on my experience with CC IAMs. But, in either case, the fix is simple: just press the "pairing" button once on your brand-new IAM which should force it to pick a valid ID and then transmit rapidly for a minute or so. The CC IAMs don't require any form of ACK during pairing.

RF collisions
Onto the interesting topic of RF collisions: ultimately, I need to capture the power consumption of every appliance in my house and this data needs to be as reliable as possible. In particular, if we expect each IAM to report once every six seconds then I'm not happy if an IAM "goes dark" for any longer than six seconds. When I had 30 IAMs, I found that individual IAMs would "go dark" for up to several hours, even if that IAM was physically right next to the EnviR. After lots of experimentation, I'm pretty confident that this loss of data is caused by RF collisions between two or more IAMs drifting into sync with each other.

To be even more explicit about what I count as "dropped data": for each CC TX, record the timecode whenever we hear from it. If the timecode between two consecutive readings from a specific TX is more than 6 seconds apart then I count this as lost data.

Sometimes a lost packet is caused by the RF packet being lost in transmission due to "unexplained interference" (I experimented with a single EnviR and a single CC TX. Even if the TX and EnviR are sat right next to each other, I was seeing a packet loss of about 10%.). But we'd expect these "unexplained" losses to be evenly distributed though out the data. As such, it should be very, very, very unlikely to see a single transmitter "go dark" for several hours due to "unexplained interference" (if the chance of a single packet being lost is 0.1 then the chance of seeing every packet lost over a 1 hour period is $0.1^{600} = 10^{-600}$ (we should expect 10 packets per minute, hence 600 packets per hour from a single IAM)).

The only explanation I can think of for a CC IAM to go dark for an hour or two is that two or more IAMs have drifted into sync with each other and one or more of the IAMs involved in this collision will have every one of its packets destroyed until these IAMs drift out of sync with each other again. (Individual IAMs don't transmit precisely 6 seconds apart; some IAMs transmit every 6100milliseconds, some every 6050milliseconds etc; and of course they don't have very accurate clocks. Hence multiple IAMs can drift in and out of sync with each other).

If you want to see some evidence for these collisions then please see the data I posted on the CurrentCost forum here.

Just a very quick update: my rfm_edf_ecomanager C++ AVR code now should compile within the Arduino IDE.

(apologies if links dont work, kept getting errors trying to preview)

Thanks for your updates - Im also on a quest to get every appliance monitored.
Ive currently (pun intended) got all my consumer box circuits monitored with Open Energy Monitor EmonTXs, part-documented here: http://openenergymonitor.org/emon/node/1025.

These pump out virtually real-time, and I only lose data (that Ive seen) due to my poor arduino coding/processing power of the ATM328 - if you're a whizz, you might be able to programme better directly. Alternatively, a 1:1 ration of TXs to EMON BASEs would work, but be expensive.

open.sen.se/andygodber

The accuracy is quite good (if finely tuned) and above 12W, as documented here: http://openenergymonitor.org/emon/node/974

Im considering opening up the socket behind wall face plates, and hooking the clamps there, but it becomes messy, and hence the desire for IAMs.

Have you considered the AEON Labs Z-wave energy monitoring/switching devices? http://www.vesternet.com/z-wave-aeon-labs-inline-smart-energy-switch-schuko

Hi Andy,

That's very cool that you've got all your consumer box circuits monitored. I might well investigate that option too so thanks loads for documenting your approach.

When you say your Open Energy Monitors pump out data "real time", how fast do you mean? One sample per second?

Can the Open Energy Monitor Base ask the OEM TX to re-send data if a packet gets lost in transit? I had a quick look at the OEM code and it seems to use the JeeLib code for handling the RFM12b; and I think the JeeLib code uses reasonably advanced stuff like checksums and ACKs ("reasonably advanced" relative to the Current Cost stuff, anyway!) to increase the reliability of data transmission.

Those AEON Labs devices look nice but rather expensive. The EDF IAMs are only about £14 and should be fairly reliable.

Just out of interest, why would you like to monitor the power consumption of every appliance in your home?!

"When you say your Open Energy Monitors pump out data "real time", how fast do you mean? One sample per second?"

It should be a bit quicker - there are no significant timed delays in the code, so the TX will read and push through the RF as quickly as its able to process. In a stand alone test (1 TX:1 RX, simple code) I reckon it was doing about 3-4 samples per second. However, the pushing of the data from the RX side, to COSM etc, is where the delays are, and is dependant on ISP speed, COSM processing time, RX code, and so on. In practice, Im seeing about 1 per second.

"Can the Open Energy Monitor Base ask the OEM TX to re-send data if a packet gets lost in transit?"

Theoretically, yes. As you surmise, JeeLib code can do this. It would be a really good evolution to ACK - I have a water monitoring TX as well, and (as these are discrete rather than continuous transmissions) can get get lost in my poor attempt at coding, so the ability to retransmit if no ACK, would ensure accurate recording.

Ive pushed the TX and RX code onto Github - I make no excuses for the butchered code - The last programming I did was in COBOL, so most is hacked from Glyn and others good work at OpenEnergyMonitor.org
https://github.com/andygodber/NanodeRF_Cosm5nodes_GitCopy
https://github.com/andygodber/TX_3CTFuseBox3AP_GitCopy

"Just out of interest, why would you like to monitor the power consumption of every appliance in your home?!"

A balance between interest, and trying to reduce energy usage/cost, help the environment, remind the family they've left the lights on etc.

Hey Jack,

Not sure if you are aware of these, but you can pick up a software defined radio receiver for ~$12-30 off of eBay that will let you see the ISM band transmissions and decode them on a PC or higher powered embedded platform (rpi for example). Look up 'rtlsdr', get ones with the E4000 or R820T tuner (the latter are the cheapest).

Great site!

This is what two separate transmissions look like when the signal from a Current Cost Sensable transmitter is passed through wideband FM demodulation and passed to audacity: http://i.imgur.com/saSpr.png The dongle is from nooelec on eBay and the RF software is sdrsharp, which is open source.

Hi Bob,

That looks very interesting, thanks for letting us know about it!

Hi Jack,

Sorry for spamming you so much! Do you have any suggestions for forums where I might be able to interact with others on this? I've posted twice about trying to build my own direct RF demodulator on the currentcost.com/support forums and the posts get rejected.

Here's where I'm at so far. First three columns are temp, chan1 and chan2 from the serial interface. The rest is what I'm decoding from the air using two frequency shift keying.

http://pastebin.com/W0iGuA0Q

I've got the rows lumped by temperature. I'm having difficulty figuring out a correlation between the RF data and the data from the EnvirR receiver serial port. I'm wondering if the HopeRF does some kind of bit interleaving or whatnot.

Thanks for any pointers you might have!

Hi. just a quick note to say that Bob and I are discussing his project on http://currentcost.com/support/viewtopic.php?p=37020&sid=5569d4bba184c98...

Add new comment