Disaggregate
Disaggregate Smart Meter data

PowerStateGraph Class Reference

This class does most of the work behind the "graphs and spikes" disaggregation approach. In particular, this class has responsibility for maintaining the "powerStateGraph" and the "disaggregation tree". More...

#include <PowerStateGraph.h>

Collaboration diagram for PowerStateGraph:

List of all members.

Classes

struct  Disag_edge_writer
 used for write_graphviz for DisagGraph. More...
struct  Disag_vertex_writer
 used for write_graphviz for DisagGraph. More...
struct  DisagVertex
 A vertex for the directed acylic graphs used by getStartTimes() More...
struct  Fingerprint
struct  LikelihoodAndVertex
struct  PowerStateEdge
struct  PowerStateVertex
struct  PSG_edge_writer
struct  PSG_vertex_writer
struct  TimeAndPower
 A simple struct for pairing timestamp and meanPower. Used in Fingerprint struct. More...

Public Member Functions

 PowerStateGraph ()
void update (const Signature &sig, const bool verbose=false)
 Update or initialise Power State Graph.
void writeGraphViz (std::ostream &out)
 Information about an entire device 'fingerprint' found in the aggregate data.
const std::list< Fingerprintdisaggregate (const AggregateData &aggregateData, const bool keep_overlapping=false, const bool verbose=false)
 This is the main public interface to the disaggregation algorithm.
void setDeviceName (const std::string &_deviceName)
const Statistic< double > & getEnergyConsumption () const

Private Types

typedef boost::adjacency_list
< boost::vecS, boost::vecS,
boost::directedS,
PowerStateVertex,
PowerStateEdge
PSGraph
 A graph where each vertex(node) is a Statistic<Sample_t> and each edge is a PowerStateEdge .
typedef boost::graph_traits
< PSGraph >::vertex_iterator 
PSG_vertex_iter
typedef boost::graph_traits
< PSGraph >::edge_iterator 
PSG_edge_iter
typedef boost::graph_traits
< PSGraph >::out_edge_iterator 
PSG_out_edge_iter
typedef boost::property_map
< PSGraph,
boost::vertex_index_t >::type 
PSG_vertex_index_map
typedef boost::property_map
< PSGraph, boost::edge_index_t >
::type 
PSG_edge_index_map
typedef boost::adjacency_list
< boost::setS, boost::vecS,
boost::bidirectionalS,
DisagVertex, double > 
DisagTree
 Tree structure used to keep track of all the possible solutions during disaggregation.
typedef boost::graph_traits
< DisagTree >
::out_edge_iterator 
Disag_out_edge_iter
typedef boost::graph_traits
< DisagTree >::edge_descriptor 
Disag_edge_desc

Private Member Functions

PSGraph::vertex_descriptor updateOrInsertVertex (const Signature &sig, const Statistic< Sample_t > &postSpikePowerState, const Statistic< Sample_t > &betweenSpikesPowerState, const bool verbose=false)
 Attempts to find an existing vertex which is statistically similar to stat. If an existing similar vertex is found then the vertex's stats are updated with the new data points. If a similar vertex is not found, and new vertex is inserted. Either way, a vertex_descriptor is returned to the new or existing similar vertex.
PSGraph::vertex_descriptor mostSimilarVertex (bool *success, const Statistic< Sample_t > &stat, const double ALPHA=0.0000005) const
 Find the vertex statistically most similarto stat.
const bool rejectSpike (const Statistic< Sample_t > &before, const Statistic< Sample_t > &after, const bool verbose=false) const
 Decides whether or not to reject this spike based on the before and after statistics.
void printSpikeInfo (const std::list< Signature::Spike >::iterator spike, const size_t start, const size_t end, const Statistic< Sample_t > &before, const Statistic< Sample_t > &after, const Signature &sig) const
 Useful for diagnostics.
void updateOrInsertEdge (const PSGraph::vertex_descriptor &beforeVertex, const PSGraph::vertex_descriptor &afterVertex, const size_t samplesSinceLastSpike, const double spikeDelta, const bool verbose=false)
 Update or Insert a new edge into powerStateGraph.
void updateEdges (const Signature &sig)
 Update (or create) directional edges between power states (vertices) so the edges are consistent with the observed transitions between power states in sig.
const Fingerprint initTraceToEnd (const AggregateData::FoundSpike &spike, const size_t deviceStart, const bool verbose=false)
void traceToEnd (DisagTree *disagTree, const DisagTree::vertex_descriptor &vertex, const size_t prevTimestamp, const bool verbose=false) const
void addItemToEdgeHistory (const PSGraph::edge_descriptor &edge)
const bool edgeListsAreEqual (const std::list< PSGraph::edge_descriptor > &a, const std::list< PSGraph::edge_descriptor > &b, const bool verbose=false) const
 Check if edge list a and edge list b are equal.
std::list
< PSGraph::edge_descriptor > 
getEdgeHistoryForVertex (const DisagTree &disagTree, const DisagTree::vertex_descriptor &startVertex) const
 Trace the disagTree backwards.
void findListOfPathsThroughDisagTree (const DisagTree &disagTree, const DisagTree::vertex_descriptor vertex, const LikelihoodAndVertex lav, std::list< LikelihoodAndVertex > path=std::list< LikelihoodAndVertex >(0))
 Trace the tree downwards from vertex recursively finding every path which successfully completes (i.e. reaches an off state).
const Fingerprint findBestPath (const DisagTree &disagTree, const size_t deviceStart, const bool verbose=false)
 Iterates through each path in listOfPaths to find the one with the highest likelihood.
void removeOverlapping (std::list< Fingerprint > *disagList, const bool verbose=false)
void displayAndPlotFingerprintList (const std::list< Fingerprint > &fingerprintList, const std::string &aggDataFilename) const
const size_t indexOfNextSpike (const std::list< Signature::Spike > &spikes, std::list< Signature::Spike >::iterator spike, const Signature &sig) const
 Determine the index of the next spike after spike.

Private Attributes

PSGraph powerStateGraph
 store the power state graph learnt during training.
PSGraph::vertex_descriptor offVertex
size_t totalCount
 the total number of times any edge has been traversed during training.
AggregateData const * aggData
std::list
< PSGraph::edge_descriptor > 
edgeHistory
 a "rolling" list storing the previous few edges we've seen.
std::list< std::list
< LikelihoodAndVertex > > 
listOfPaths
Statistic< double > energyConsumption
 Energy consumption in Joules obtained from training signatures.
std::string deviceName
 All PowerStateGraphs are associated with a single device.

Static Private Attributes

static const size_t EDGE_HISTORY_SIZE = 5
 Set to 0 to disable.

Friends

std::ostream & operator<< (std::ostream &o, const PowerStateGraph &psg)

Detailed Description

This class does most of the work behind the "graphs and spikes" disaggregation approach. In particular, this class has responsibility for maintaining the "powerStateGraph" and the "disaggregation tree".

Definition at line 26 of file PowerStateGraph.h.


Member Typedef Documentation

typedef boost::graph_traits<DisagTree>::edge_descriptor PowerStateGraph::Disag_edge_desc [private]

Definition at line 231 of file PowerStateGraph.h.

typedef boost::graph_traits<DisagTree>::out_edge_iterator PowerStateGraph::Disag_out_edge_iter [private]

Definition at line 230 of file PowerStateGraph.h.

typedef boost::adjacency_list< boost::setS, boost::vecS, boost::bidirectionalS, DisagVertex, double > PowerStateGraph::DisagTree [private]

Tree structure used to keep track of all the possible solutions during disaggregation.

Implemented using the Boost Graph Library.

Definition at line 228 of file PowerStateGraph.h.

typedef boost::property_map<PSGraph, boost::edge_index_t >::type PowerStateGraph::PSG_edge_index_map [private]

Definition at line 138 of file PowerStateGraph.h.

typedef boost::graph_traits<PSGraph>::edge_iterator PowerStateGraph::PSG_edge_iter [private]

Definition at line 134 of file PowerStateGraph.h.

typedef boost::graph_traits<PSGraph>::out_edge_iterator PowerStateGraph::PSG_out_edge_iter [private]

Definition at line 135 of file PowerStateGraph.h.

typedef boost::property_map<PSGraph, boost::vertex_index_t>::type PowerStateGraph::PSG_vertex_index_map [private]

Definition at line 137 of file PowerStateGraph.h.

typedef boost::graph_traits<PSGraph>::vertex_iterator PowerStateGraph::PSG_vertex_iter [private]

Definition at line 133 of file PowerStateGraph.h.

typedef boost::adjacency_list< boost::vecS, boost::vecS, boost::directedS, PowerStateVertex, PowerStateEdge > PowerStateGraph::PSGraph [private]

A graph where each vertex(node) is a Statistic<Sample_t> and each edge is a PowerStateEdge .

See boost::graph bundles tutorial

Definition at line 115 of file PowerStateGraph.h.


Constructor & Destructor Documentation

PowerStateGraph::PowerStateGraph ( )

Definition at line 17 of file PowerStateGraph.cpp.

References offVertex, and powerStateGraph.


Member Function Documentation

void PowerStateGraph::addItemToEdgeHistory ( const PSGraph::edge_descriptor &  edge) [private]

Definition at line 28 of file PowerStateGraph.cpp.

References EDGE_HISTORY_SIZE, and edgeHistory.

Referenced by updateOrInsertEdge().

Here is the caller graph for this function:

const list< PowerStateGraph::Fingerprint > PowerStateGraph::disaggregate ( const AggregateData aggregateData,
const bool  keep_overlapping = false,
const bool  verbose = false 
)

This is the main public interface to the disaggregation algorithm.

Each candidate solution is represented as a tree. Each vertex on this tree is a spike found in the aggregate data. The edge weights are the mean of the probability density functions for the spike size and the timing. If an edge described in PowerStateGraph (from training) cannot be found in the aggregate data then this candidate is discarded. If we successfully get from vertex0 back to vertex0 then the shortest path through the tree is calculated and saved.

  1. Retrieve edge e which connects vertex0 (offVertex) to vertex1. Look through the AggregateData searching for any spike within a certain number of standard deviations of e.delta.
  2. When a spike is found, start a tree structure and start looking for the subsequent edges learnt during training.
  3. Look for the delta corresponding to each out edge from current vertex. Store the UNIX timestamp of each candidate.
Returns:
a list of UNIX times when the device starts
Parameters:
aggregateDataA populated array of AggregateData
keep_overlappingShould we keep or remove overlapping candidates?

Definition at line 612 of file PowerStateGraph.cpp.

References aggData, PowerStateGraph::Fingerprint::avLikelihood, PowerStateGraph::PowerStateEdge::delta, displayAndPlotFingerprintList(), PowerStateGraph::PowerStateEdge::duration, Utils::fatalError(), AggregateData::findSpike(), AggregateData::getFilename(), initTraceToEnd(), Statistic< T >::mean, offVertex, powerStateGraph, and removeOverlapping().

Referenced by main().

Here is the call graph for this function:

Here is the caller graph for this function:

void PowerStateGraph::displayAndPlotFingerprintList ( const std::list< Fingerprint > &  fingerprintList,
const std::string &  aggDataFilename 
) const [private]

Definition at line 691 of file PowerStateGraph.cpp.

References GNUplot::PlotVars::data, DATA_OUTPUT_PATH, deviceName, GNUplot::PlotVars::inFilename, Utils::openFile(), GNUplot::PlotVars::outFilename, GNUplot::plot(), GNUplot::PlotVars::plotArgs, Utils::size_t_to_s(), GNUplot::PlotVars::title, GNUplot::PlotVars::xlabel, and GNUplot::PlotVars::ylabel.

Referenced by disaggregate().

Here is the call graph for this function:

Here is the caller graph for this function:

const bool PowerStateGraph::edgeListsAreEqual ( const std::list< PSGraph::edge_descriptor > &  a,
const std::list< PSGraph::edge_descriptor > &  b,
const bool  verbose = false 
) const [private]

Check if edge list a and edge list b are equal.

Definition at line 418 of file PowerStateGraph.cpp.

References EDGE_HISTORY_SIZE, and powerStateGraph.

Referenced by traceToEnd(), and updateOrInsertEdge().

Here is the caller graph for this function:

const PowerStateGraph::Fingerprint PowerStateGraph::findBestPath ( const DisagTree disagTree,
const size_t  deviceStart,
const bool  verbose = false 
) [private]

Iterates through each path in listOfPaths to find the one with the highest likelihood.

To be called after listOfPaths has been populated by findListOfPathsThroughDisagTree().

Returns:
details of the best path. Confidence is set to -1 if no paths are available.

Definition at line 1060 of file PowerStateGraph.cpp.

References PowerStateGraph::Fingerprint::avLikelihood, PowerStateGraph::Fingerprint::duration, PowerStateGraph::Fingerprint::energy, energyConsumption, listOfPaths, Statistic< T >::normalisedLikelihood(), PowerStateGraph::Fingerprint::timeAndPower, and PowerStateGraph::Fingerprint::timestamp.

Referenced by initTraceToEnd().

Here is the call graph for this function:

Here is the caller graph for this function:

void PowerStateGraph::findListOfPathsThroughDisagTree ( const DisagTree disagTree,
const DisagTree::vertex_descriptor  vertex,
const LikelihoodAndVertex  lav,
std::list< LikelihoodAndVertex path = std::list<LikelihoodAndVertex>(0) 
) [private]

Trace the tree downwards from vertex recursively finding every path which successfully completes (i.e. reaches an off state).

Parameters:
pathDeliberately called-by-value because we want a copy.

Definition at line 866 of file PowerStateGraph.cpp.

References PowerStateGraph::LikelihoodAndVertex::likelihood, listOfPaths, and PowerStateGraph::LikelihoodAndVertex::vertex.

Referenced by initTraceToEnd().

Here is the caller graph for this function:

list< PowerStateGraph::PSGraph::edge_descriptor > PowerStateGraph::getEdgeHistoryForVertex ( const DisagTree disagTree,
const DisagTree::vertex_descriptor &  startVertex 
) const [private]

Trace the disagTree backwards.

Definition at line 1152 of file PowerStateGraph.cpp.

References EDGE_HISTORY_SIZE, and offVertex.

const Statistic< double > & PowerStateGraph::getEnergyConsumption ( ) const

Definition at line 42 of file PowerStateGraph.cpp.

References energyConsumption.

Referenced by Device::trainPowerStateGraph().

Here is the caller graph for this function:

const size_t PowerStateGraph::indexOfNextSpike ( const std::list< Signature::Spike > &  spikes,
std::list< Signature::Spike >::iterator  spike,
const Signature sig 
) const [private]

Determine the index of the next spike after spike.

Definition at line 171 of file PowerStateGraph.cpp.

References Array< T >::getSize().

Referenced by update().

Here is the call graph for this function:

Here is the caller graph for this function:

const PowerStateGraph::Fingerprint PowerStateGraph::initTraceToEnd ( const AggregateData::FoundSpike spike,
const size_t  deviceStart,
const bool  verbose = false 
) [private]
Returns:
DisaggregatedStruct.likelihood will be set to -1 if this looks like it's not a good candidate match.
Parameters:
deviceStartThe possible time the device started.

Definition at line 796 of file PowerStateGraph.cpp.

References EDGE_HISTORY_SIZE, findBestPath(), findListOfPathsThroughDisagTree(), PowerStateGraph::LikelihoodAndVertex::likelihood, AggregateData::FoundSpike::likelihood, listOfPaths, offVertex, powerStateGraph, AggregateData::FoundSpike::timestamp, traceToEnd(), and PowerStateGraph::LikelihoodAndVertex::vertex.

Referenced by disaggregate().

Here is the call graph for this function:

Here is the caller graph for this function:

PowerStateGraph::PSGraph::vertex_descriptor PowerStateGraph::mostSimilarVertex ( bool *  success,
const Statistic< Sample_t > &  stat,
const double  ALPHA = 0.0000005 
) const [private]

Find the vertex statistically most similarto stat.

Returns:
vertex descriptor of best fit. success is also used as a return parameter.
Parameters:
successreturn parameter. Did we find a satisfactory match?
statstat to find in graph vertices
ALPHAsignificance level (what constitutes as a "satisfactory" match?)

Definition at line 277 of file PowerStateGraph.cpp.

References Utils::highest(), Statistic< T >::mean, powerStateGraph, Statistic< T >::tTest(), and Utils::within().

Referenced by updateEdges(), and updateOrInsertVertex().

Here is the call graph for this function:

Here is the caller graph for this function:

void PowerStateGraph::printSpikeInfo ( const std::list< Signature::Spike >::iterator  spike,
const size_t  start,
const size_t  end,
const Statistic< Sample_t > &  before,
const Statistic< Sample_t > &  after,
const Signature sig 
) const [private]

Useful for diagnostics.

Definition at line 134 of file PowerStateGraph.cpp.

Referenced by update().

Here is the caller graph for this function:

const bool PowerStateGraph::rejectSpike ( const Statistic< Sample_t > &  before,
const Statistic< Sample_t > &  after,
const bool  verbose = false 
) const [private]

Decides whether or not to reject this spike based on the before and after statistics.

Definition at line 189 of file PowerStateGraph.cpp.

References Utils::highest(), Statistic< T >::mean, Statistic< T >::stdev, and Utils::within().

Referenced by update().

Here is the call graph for this function:

Here is the caller graph for this function:

void PowerStateGraph::removeOverlapping ( std::list< Fingerprint > *  disagList,
const bool  verbose = false 
) [private]

Remove any overlapping list entries and leave the one with the highest likelihood

Parameters:
disagListInput and output parameter

Definition at line 746 of file PowerStateGraph.cpp.

Referenced by disaggregate().

Here is the caller graph for this function:

void PowerStateGraph::setDeviceName ( const std::string &  _deviceName)

Definition at line 1171 of file PowerStateGraph.cpp.

References deviceName.

Referenced by Device::Device().

Here is the caller graph for this function:

void PowerStateGraph::traceToEnd ( DisagTree disagTree_p,
const DisagTree::vertex_descriptor &  disagVertex,
const size_t  prevTimestamp,
const bool  verbose = false 
) const [private]

Trace from startVertex to the off state in PSGraph

Parameters:
disagTree_pinput and output parameter
prevTimestamptimestamp of previous vertex

Definition at line 909 of file PowerStateGraph.cpp.

References aggData, EDGE_HISTORY_SIZE, edgeHistory, edgeListsAreEqual(), AggregateData::findSpike(), offVertex, powerStateGraph, and AggregateData::readingGoesBelowPowerState().

Referenced by initTraceToEnd().

Here is the call graph for this function:

Here is the caller graph for this function:

void PowerStateGraph::update ( const Signature sig,
const bool  verbose = false 
)

Update or initialise Power State Graph.

Definition at line 50 of file PowerStateGraph.cpp.

References Signature::Spike::compareIndexAsc(), edgeHistory, energyConsumption, Signature::getDeltaSpikes(), Signature::getEnergyConsumption(), Signature::getID(), Array< T >::getSize(), indexOfNextSpike(), J_PER_KWH, offVertex, printSpikeInfo(), rejectSpike(), Statistic< T >::update(), updateOrInsertEdge(), and updateOrInsertVertex().

Referenced by Device::trainPowerStateGraph().

Here is the call graph for this function:

Here is the caller graph for this function:

void PowerStateGraph::updateEdges ( const Signature sig) [private]

Update (or create) directional edges between power states (vertices) so the edges are consistent with the observed transitions between power states in sig.

Basic strategy is to:

  1. retrieve gradient spikes from sig, extract the most salient and put the spikes into temporal order.
  2. Create stats for the data points between each spike. Use mostSimilarVertex() to determine which of the existing power state vertices this belongs to.
  3. Check to see if there's an existing edge representing the power state transition, if so then check the delta and duration stats for the edge and update if necessary. If no edge exists then create one with the necessary delta and duration, with a hard-coded stdev.
Deprecated:
just leaving this here to illustrate one of the strategies I tried. This has been superseded by update() (which used to be updateVertices) and updateOrInsertEdge()

Definition at line 494 of file PowerStateGraph.cpp.

References Signature::Spike::compareIndexAsc(), Signature::getDeltaSpikes(), mostSimilarVertex(), offVertex, powerStateGraph, and totalCount.

Here is the call graph for this function:

void PowerStateGraph::updateOrInsertEdge ( const PSGraph::vertex_descriptor &  beforeVertex,
const PSGraph::vertex_descriptor &  afterVertex,
const size_t  samplesSinceLastSpike,
const double  spikeDelta,
const bool  verbose = false 
) [private]

Update or Insert a new edge into powerStateGraph.

  • If an edge already exists between beforeVertex and afterVertex then
    • check all out-edges from beforeVertex
    • if any out-edge has the same history as the current history and the same sign delta then and and is within 3 stdevs of the existing delta then:
      • update that edge's statistics
    • else
      • create a new edge.
  • else
    • create a new edge

Definition at line 329 of file PowerStateGraph.cpp.

References addItemToEdgeHistory(), edgeHistory, edgeListsAreEqual(), powerStateGraph, Utils::sameSign(), totalCount, and Utils::within().

Referenced by update().

Here is the call graph for this function:

Here is the caller graph for this function:

PowerStateGraph::PSGraph::vertex_descriptor PowerStateGraph::updateOrInsertVertex ( const Signature sig,
const Statistic< Sample_t > &  postSpikePowerState,
const Statistic< Sample_t > &  betweenSpikesPowerState,
const bool  verbose = false 
) [private]

Attempts to find an existing vertex which is statistically similar to stat. If an existing similar vertex is found then the vertex's stats are updated with the new data points. If a similar vertex is not found, and new vertex is inserted. Either way, a vertex_descriptor is returned to the new or existing similar vertex.

Parameters:
sigsource of the raw data
verbosecout debugging messages?

Definition at line 221 of file PowerStateGraph.cpp.

References mostSimilarVertex(), offVertex, and powerStateGraph.

Referenced by update().

Here is the call graph for this function:

Here is the caller graph for this function:

void PowerStateGraph::writeGraphViz ( std::ostream &  out)

Information about an entire device 'fingerprint' found in the aggregate data.

Produce a graphviz output.

Definition at line 576 of file PowerStateGraph.cpp.

References powerStateGraph.

Referenced by Device::trainPowerStateGraph().

Here is the caller graph for this function:


Friends And Related Function Documentation

std::ostream& operator<< ( std::ostream &  o,
const PowerStateGraph psg 
) [friend]

Definition at line 1176 of file PowerStateGraph.cpp.


Member Data Documentation

Definition at line 285 of file PowerStateGraph.h.

Referenced by disaggregate(), and traceToEnd().

std::string PowerStateGraph::deviceName [private]

All PowerStateGraphs are associated with a single device.

Definition at line 309 of file PowerStateGraph.h.

Referenced by displayAndPlotFingerprintList(), and setDeviceName().

const size_t PowerStateGraph::EDGE_HISTORY_SIZE = 5 [static, private]

Set to 0 to disable.

Definition at line 287 of file PowerStateGraph.h.

Referenced by addItemToEdgeHistory(), edgeListsAreEqual(), getEdgeHistoryForVertex(), initTraceToEnd(), and traceToEnd().

std::list< PSGraph::edge_descriptor > PowerStateGraph::edgeHistory [private]

a "rolling" list storing the previous few edges we've seen.

Definition at line 288 of file PowerStateGraph.h.

Referenced by addItemToEdgeHistory(), PowerStateGraph::PSG_edge_writer::operator()(), traceToEnd(), update(), and updateOrInsertEdge().

Energy consumption in Joules obtained from training signatures.

Definition at line 306 of file PowerStateGraph.h.

Referenced by findBestPath(), getEnergyConsumption(), and update().

std::list< std::list<LikelihoodAndVertex> > PowerStateGraph::listOfPaths [private]

the total number of times any edge has been traversed during training.

Deprecated:
not actually used.

Definition at line 281 of file PowerStateGraph.h.

Referenced by updateEdges(), and updateOrInsertEdge().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Friends