information theory

Free Stanford course on Probabilistic Graphical Models

One of the main research directions for my PhD is likely to be experimenting with bispoke probabilistic graphical models for representing multi-state appliances like washing machines.  As such, I need to learn about existing probabilistic graphical models.  For the past few days I've been reading a textbook called "Probabilistic Graphical Models" by Koller and Friedman.  So far I've really enjoyed the book.

Over lunch today, a friend of mine told me that Stanford are running a free on-line course on Probabilistic Graphical Models, presented by Koller. I've signed up - it looks like a great course; it starts in Feb. Stanford are also running a course on Information Theory, which I've also signed up for.

My copy of David MacKay's "Information Theory, Inference and Learning Algorithms" arrived

My copy of David MacKay's book "Information Theory, Inference and Learning Algorithms" arrived yesterday and I started reading it last night. It looks absolutely fascinating.

I first came across Professor MacKay when I read his other book, "Sustainable Energy Without the Hot Air" and  I was struck by how lucid, readable and entertaining it is. There is every reason to expect that his information theory book will be at least as readable.

Why get a book on "Information Theory, Inference and Learning Algorithms"? The main reason is because it should be very useful for my PhD in smart meter disaggregation. One of the god-fathers of disaggregation, George Hart, wrote the following in Hart 1992:

It is insightful to consider the [disaggregation problem] in the context of a communication model. Appliances can be thought of as “transmitters”, inadvertently broadcasting information as a by-product of their operation. The communication “channel” here is the house wiring. Any of the many signatures... may be the “codes” used in this communication scheme. Our task is to design a “receiver” for these codes which can decode them in terms of appliance state-change “messages”.

In other words, the disaggregation problem can be considered in an information-theoretic framework. This conceptual step allows us to take advantage of the tools developed in communication technology (i.e. coding theory).

Also, for a while now I've thought of information theory as one of those "sexy but mysterious" things which I'd love to learn more about. I read "Decoding Reality: The Universe as Quantum Information" a couple of years ago and found it fascinating (although I only understood about half the content of the book).

So, hopefully MacKay's information theory book will be a great self-study book.

Subscribe to RSS - information theory