Blog
OpenEnergyMonitor

OpenEnergyMonitor, emonPi and openHAB


Using the emonPi in it's current default configuration works great to post data to Emoncms for logging and visualisation. However there may be times when you want some more flexibility and or ability to interface with other hardware or services 

In my last few blog posts I have been taking a look at making the emonPi the heart of a smart energy optimised home and how live energy monitoring data from the emonPi can easily be used to inform other services:   


North Wales Tech Meetup Presentations

Yesterday evening Trystan and I attend a local meetup in Bangor, North Wales hosted by NorthWalesTech It was a good event with some interesting talks. Here are the slides from the 5min lightning talks we gave:
 
My slides give a brief overview of the OpenEnergyMontor project before diving in to MQTT and node-RED with a live demo controlling LightWave RF plugs switching Christmas lights over the web using MQTT and nodeRED:



Slides from N.Wales Tech meetup Dec 15. Brief overview of OpenEnergyMonitor and MQTT, nodeRED and LightWaveRF demo.

Related blog posts:
http://openenergymonitor.blogspot.com/2015/10/emonpi-nodered-and-mqtt.html 
http://openenergymonitor.blogspot.com/2015/11/remote-control-of-lightwave-rf-plugs.html 
http://openenergymonitor.blogspot.com/2015/11/node-red-emoncms-node.html 
http://openenergymonitor.blogspot.com/2015/11/ambient-wind-energy-indicator-using.html

---------------------------------------------------------------------------------------------------------

Trystan's lightning talk was focused on a heatpump which he has recently installed with John Cantor to heat his home. Trystan gave an overview of heatpump technology, his monitoring setup and the preliminary performance results:



Remote Control of LightWave RF plugs via MQTT using emonPi with OOK Tx

Using a Raspberry Pi as an emonPi / emonBase web-connected base station for energy monitoring has many advantages (maintainability, remote access, debugging, updated etc.) but it does seem rather wasteful to have a 900Mhz 1GB RAM machine being used to post a couple of integers to a web-server! The always-on nature of a web-connected base station and plenty of spare processing power makes it ideal to use the emonPi for more than just energy monitoring; home automation and heating control optimisation are obvious candidates.


Hardware

Many home automation products such as RF plugs, relays and heating remotes use a simple OOK (On-Off-Keying) wireless protocol. UK company LightWaveRF produce a variety of RF plugs and relays which can be controlled via OOK RF. The protocol is also compatible with some cheaper OOK learning receiver relays.  While maybe not strictly open-source the protocol and been reverse engineered allowing plugs to be easily controlled from Arduino / Raspberry Pi. There is an active LightWaveRF online community. Using off-the shelf hardware like this is a 'safe' way to control lights, heaters and appliances  around a home without getting our hands dirty dealing with with high voltages. These plugs and relays can be used to control anything from lights to immersion heaters, most LightWaveRF plugs/relays will switch up to 13A / 3kW.

Note: OOK protocol by it's simplistic nature is not particularly secure, I would not recommend controlling anything you don't mind getting accidently switched.

An OOK transmitter can easily be added to the emonPi.  We have started stocking OOK transmitter modules in the OpenEnergyMonitor store as well as adding an option to have OOK module factory installed on new emonPi's.

See emonPi Technical Hardware Wiki for details how to retrofit OOK TX module to existing emonPi's.

Software

We have been working with Lawrie Griffiths from Geek Grandad blog to develop a LightWaveRF MQTT service to run on a Raspberry Pi to allow LightWaveRF plugs to be controlled via MQTT.


Once running a plug can be controlled by publishing to the 'lwrf' MQTT topic:

E.g "1 1" switches on plug 1 while "1 0" switches off plug 1. Plugs can be paired in the usual LightWaveRF plug.

See emonPi technical Wiki for how to install & setup.

Existing LightWaveRF remotes can be emulated allowing plugs to be controlled from either emonPi MQTT or via the remote.

Now our plugs can be controlled via MQTT there are a many of options open as to how to control either via interface openHAB (blog post coming soon..), nodeRED, android Tasker etc.

I have been using nodeRED with input from android OwnTracks running on my phone to detect when I'm home and turn on a couple of lights:

Demo:

Using NodeRED and OwnTracks to turn on LightWave RF sockets when I arrive home: 




Using android Tasker with  AutoVoice and MQTT plugins to control LightWaveRF sockets:



Next Steps

Now we can control appliances remotely via MQTT the next logical step is to add a mobile user interface. In my next blog post I plan to explore setting up openHab to control appliances. Here's a early preview: 




We plan to include out-of-the-box support for OOK LightWave RF, nodeRED and openHab pre-configured on our next emonPi pre-built SD card. Due for release in December 2015.


UK Grid Carbon Intensity API


Following on from my blog post creating an ambient indicator to indicate wind turbine renewable electricity generation we have extended the Emoncms UK electric grid statistic aggregation to calculate and record the UK electric grid carbon intensity i.e. the amount of carbon emitted per kWh (CO2/kWh). As before we have created a dashboard open API for easy access to this data.

http://emoncms.org/ukelectricitygrid/ukgrid

http://emoncms.org/ukelectricitygrid/ukwind

Dashboards:

Open-API:
UK grid carbon intensity: http://emoncms.org/feed/value.json?id=97715
UK wind: http://emoncms.org/feed/value.json?id=67088

Other feeds are also available including grid fuel mix and current energy demand, just place 'id' in the API with the feed ID's below: 

FeedNameCurrent Value (22/11/15 22hrs)
97689CCGT_val11592
97690CCGT_prc0.31
97691OCGT_val0
97692OCGT_prc0
97693OIL_val0
97694OIL_prc0
97695COAL_val10785
97696COAL_prc0.29
97697NUCLEAR_val7798
97698NUCLEAR_prc0.21
97699WIND_val1308
97700WIND_prc0.03
97701PS_val397
97702PS_prc0.01
97703NPSHYD_val533
97704NPSHYD_prc0.01
97705OTHER_val1976
97706OTHER_prc0.05
97707INTFR_val1495
97708INTFR_prc0.04
97709INTIRL_val0
97710INTIRL_prc0
97711INTNED_val975
97712INTNED_prc0.02
97713INTEW_val0
97714INTEW_prc0
97715gridintensity435.6
97736totalsupply36859
If you use the API in your project please drop us a line and give Emoncms some credit :-)

The data is captured from the UK Balancing BM Reports XML. Our figure for grid intensity matched the figure generated by http://www.gridcarbon.uk/

Node-RED Emoncms Node


I have recently been experimented using Node-RED for a number of tasks. Thanks to the work of others a Node-RED flow node already existed for Emoncms making it very easy to post data to Emoncms. For my Ambient Wind Energy Indicator I had the need to extract real-time feed data from Emoncms feeds using the Emoncms API. I have now contributed this functionality to the official Node-RED Emoncms node. Using the updated Emoncms node it's now super easy to post data to extract latest real-time feed values from any Emoncms server including Emoncms.org.

Get the latest version from: http://flows.nodered.org/node/node-red-node-emoncms. If you're interested in viewing and improving the functionality the open-source code behind the node is up on the Node-RED flows GitHub

Emoncms Node-RED node

Install with:

$ cd ~/.node-red
$ npm install node-red-node-emoncms

Node Documentation

Emoncms in Node Config

Emoncms Server Config



OpenEnergyMonitor November 2015 - Overview Presentation

Here's a few slides giving an overview of the OpenEnergyMonitor project as it stands currently in November 2015. It features some of the work we have been doing looking at the embodied energy involved in producing and shipping some of our products as well as latest developments on Emoncms application dashboards



OpenEnergyMonitor Overview November 2015 from OpenEnergyMonitor

Notes to accompany the slides can be found here.

Interpreting Emoncms Dashboards


The emonPi / emonTx standard install is designed to monitor whole house power consumption. An excellent learning exercise that we recommend everyone undertake soon after installing a whole house energy monitor is to turn off every appliance in the house then slowly turn on each appliance keeping a close eye on the real-time Emoncms dashboard. Walking around the house using a handheld mobile device to display the Emoncms dashboard works great for this. For fastest response we recommend connecting to the local Emoncms running locally on the emonBase / emonPi which will update ever 5s (emonPi default) or 10s (emonTx default).

After this exercise you should have a good idea what the trace signature of each of your appliances looks like on the graph. The easiest to identify are the large power consumers e.g kettle 3kW spike and shower 8kW spike. However with a bit of observation it's possible to identify subtler power signatures. Here's an excellent example by posted by @Zapaman showing a Solar PV dashboard annotated to identify his fridge / freezer and central heating pump running over night. It's great to see the solar PV kick in providing more then the demand when the sun hits the panels in the morning :-)



Ambient Wind Energy Indicator using Node-RED and Blink(1) USB

As we move to a low carbon electric grid with large amount of input from renewable sources we will need to match our consumption of energy to generation. The wind will not always blow and the sun doesn't always shine. Grid energy storage will play a role, however the closer we can match our consumption of energy to generation the less storage will be required. In the future we can expect dynamic pricing to act as as an incentive for shifting consumption patterns. 

As an experiment I have been playing around with creating a real-time ambient indicator of UK wind energy generation. Since wind energy is the greatest input of renewable energy into the UK grid we can assume that when wind generation output is high this is the best time for us to use power e.g. run the washing machine, dish washer etc.. 

I've used emoncms.org,  Node-RED and a ThingM blink(1) USB RGB LED plugged into an emonPi energy monitor to create the indicator.

Real-time wind energy indicator

Blink USB Indicator

First let's install the software to control the Blink(1) USB on the emonPi / RaspberryPi:

$ git clone https://github.com/todbot/blink1
$ sudo cp ~/blink1/linux/51-blink1.rules /etc/udev/rules.d/
$ sudo udevadm control --reload-rules
un-plug then re-plug device
$ cd blink1/commandline
$ make

Note: if Node HID fails to compile you may need to install gcc-4.8. I did not experience this problem using Raspbian Wheezy: https://github.com/node-hid/node-hid/issues/115

Test it's operation by turning on a white LED with
$ ~/blink1/commandline./blink1-tools --on

To make the Blink(1) USB work with Node-RED we need to install the Blink(1) node-RED flow and libusb driver. If you haven't already got it installed this example also requires the emoncms noe-RED flow to pull data from emoncms.org

$ sudo apt-get install libusb-1.0-0.dev
$ cd ~/.node-red
$ npm install node-red-node-blink1
$ npm install node-red-node-emoncms

Then restart node-RED.

Node-RED makes it super easy to link the required software components together, with pre built nodes available for emoncms and Blink USB it's just a case of linking the nodes together and adding some logic. Here is now node-RED flow, see node-RED export at bottom of this post:


We have a script running on emoncms.org to pull in real-time UK wind energy generation. We have been logging this data for almost a year now. Checkout the dashboard here: http://emoncms.org/mywindpower/ukwind.

http://emoncms.org/mywindpower/ukwind

The read API key for this account is 8f5c2d146c0c338845d2201b8fe1b0e1 and the feed ID is 67088. To pull the real time data into node red just plug in the  api key and feed ID into the node-RED emoncms in node. Node-RED will then execute the following GET request:

The blink usb node requires RGB csv to represent a colour output. A node is used to map the generation 0 -  7000 MW (currently!) to 0 - 255 RGB representation. 





[{"id":"98c24074.673dc","type":"emoncms-server","server":"http://emoncms.org","name":"emoncms UK windpower"},{"id":"5a3294d0.a5cd6c","type":"function","name":"Create RGB values CSV","func":"// convert to integer \n//var wind = +msg.payload;\n\n// green = 255 @ max wind output\nvar g = msg.payload\n\n// red = 255 @ zero wind output\nvar r= 255 - msg.payload\n\n// Round to whole number\ng = Number((g).toFixed(0));\nr = Number((r).toFixed(0));\n\nvar csv = r + \",\" + g + \",\" + 0;\nreturn {payload:csv};","outputs":1,"noerr":0,"x":617,"y":348,"z":"c0b0c402.3f4f38","wires":[["52d8719f.ad279","f3dd00d0.0c23"]]},{"id":"6ec040ed.913fc","type":"inject","name":"Trigger every 10min 7AM-11PM","topic":"","payload":"1445415635","payloadType":"string","repeat":"","crontab":"*/10 7-22 * * *","once":false,"x":297,"y":261,"z":"c0b0c402.3f4f38","wires":[["68f8d790.970728"]]},{"id":"52d8719f.ad279","type":"debug","name":"","active":false,"console":"false","complete":"false","x":878,"y":334,"z":"c0b0c402.3f4f38","wires":[]},{"id":"b5d43139.4a2bd","type":"inject","name":"Test 100MW","topic":"","payload":"100","payloadType":"string","repeat":"","crontab":"","once":false,"x":223,"y":421,"z":"c0b0c402.3f4f38","wires":[["2da76a85.d25896"]]},{"id":"9122dd65.6edd2","type":"inject","name":"Test 2000 MW","topic":"","payload":"2500","payloadType":"string","repeat":"","crontab":"","once":false,"x":225,"y":469,"z":"c0b0c402.3f4f38","wires":[["2da76a85.d25896"]]},{"id":"b5c8e6e5.4a3718","type":"inject","name":"Test 5000MW","topic":"","payload":"5000","payloadType":"string","repeat":"","crontab":"","once":true,"x":224,"y":514,"z":"c0b0c402.3f4f38","wires":[["2da76a85.d25896"]]},{"id":"f3dd00d0.0c23","type":"blink1","fade":"600","name":"Blink(1) USB","x":847,"y":440,"z":"c0b0c402.3f4f38","wires":[]},{"id":"68f8d790.970728","type":"emoncms in","name":"Get UK wind generation output from emoncm.org","emonServer":"98c24074.673dc","feedid":"67087","x":263,"y":343,"z":"c0b0c402.3f4f38","wires":[["2da76a85.d25896"]]},{"id":"2da76a85.d25896","type":"range","minin":"0","maxin":"6000","minout":"0","maxout":"255","action":"clamp","round":true,"name":"Range: Scale wind output to RGB value","x":551,"y":417,"z":"c0b0c402.3f4f38","wires":[["5a3294d0.a5cd6c"]]},{"id":"64a3fa26.9b5c04","type":"inject","name":"Turn off after 11PM","topic":"","payload":"0,0,0","payloadType":"string","repeat":"","crontab":"01 23 * * *","once":false,"x":613,"y":488,"z":"c0b0c402.3f4f38","wires":[["f3dd00d0.0c23"]]}]

emonPi, NodeRed and MQTT

Using the emonPi in it's default configuration works great to post data to Emoncms for logging and visualisation. However there may be times when you want some more flexibility and or ability to interface with other hardware.

Using a tool called Node-Red the emonPi can become a central hub for home automation, control and notification. Node-RED is a tool for wiring together hardware devices, APIs and online services in new and interesting ways. At the heart of Node-RED is a visual editor allowing complex data flows to be wired together with only a little coding skills.  Here are some of my ideas of things I might like to achieve with Node-RED, For more inspiration check out Martin's IoT Google hangout 29min in:


Install Node-RED on emonPi / emonHub / Raspberry Pi 

Make emonPi file-system RW:
$ rpi-rw

Install Node-RED (taken from node-RED RaspberryPi install guide):
$ curl -sL https://deb.nodesource.com/setup_0.12 | sudo bash - 
$ sudo apt-get install -y build-essential python-dev python-rpi.gpio nodejs
$ sudo npm install -g --unsafe-perm node-red

Open up port 1880 so we can access Node-RED editor:
$ sudo iptables -A INPUT -p tcp -m tcp --dport 1880 -j ACCEPT

Open up port 1883 so we can access MQTT server:
$ sudo iptables -A INPUT -p tcp -m tcp --dport 1883 -j ACCEPT

Make iptables rules persistent
$ apt-get install iptables-persistent

Choose yes to save rules, if needed edit the persistent rules:
$ sudo nano /etc/iptables/rules.v4


Start Node-RED with:
$ node-red-pi --max-old-space-size=128

To view the node-RED graphical editor browse to:
http://emonpi:1880/ (or use IP address if this does not work)

To use Node-RED as a permanent setup you will want to set it up as a service to autostart at boot and enable secure authentication access to node-RED editor.

When you are done don't forget to put the emonPi back to read-only mode to preserve SD card lifespan:
$ rpi-ro

Getting data from the emonPi into Node-RED

There are two options to get our energy data into Node-RED:

1. Real-time data from emonHub using MQTT (see example below):

MQTT is a lightweight communication protocol for small web-connected devices. The latest version of emonHub as setup on the emonPi (and emonHub from July 15) uses MQTT as a link to Emoncms and also to provide data to the emonPi LCD script. Since MQTT is already running all we need to do is to point Node-RED MQTT input block to subscribe to the 'emonhub/rx/#' MQTT topic on port 1883. The '#' topic includes data received from all nodes. To subscribe to just one node use e.g. emonPi use: 'emonhub/5/values' or 'emonhub/10/values' for emonTx.

To subscribe the the emonPi's MQTT topics externally (if your running node-RED on another machine) you will need to open up the MQTT port on the emonPi:

$ sudo iptables -A INPUT -p tcp -m tcp --dport 1883 -j ACCEPT

2. Historic data from Emoncms using HTTP API: 

To pull historic data already logged to Emoncms (e.g KWh data) into Node-RED we can use Emoncms's API. Here is a simple example to pull in the latest value from the KWh feed, just enter your RW API key and feed ID:

http://emoncms.org/feed/value.json&apikey=APIKEY&id=FEED_ID

To view all Emoncms API see: http://emoncms.org/input/api

Node-RED example: send warning email when emonTH battery is low


Simple but useful example, use Node-RED connecting to emonPi in real-time data via MQTT (localhost) and sending email (via Gmail SMTP) if emonTH battery drops below 1.7V. Also in this flow as an example is a node to decode emonPi power data.


Here is the contents of each node: 
MQTT Node subscribing to emonhub mqtt topic on localhost (running node-RED on emonPi)
Separate node to split the MQTT data up into nodes based on ID: e.g emonTx = 10, emonPi = 5, emonTH = 19
CSV node to split up MQTT CSV string
If function node
email node, enter your Gmail (or otherwise) SMTP login

Node-RED flow export, import by copy and pasting into import>clipboard in Node-RED editor:

[{"id":"eb57a13e.14a86","type":"mqtt-broker","broker":"localhost","port":"1883","clientid":""},{"id":"44c7eefa.bb381","type":"mqtt in","name":"MQTT: emonhub/rx/#","topic":"emonhub/rx/#","broker":"eb57a13e.14a86","x":159,"y":386,"z":"fa6ca080.05936","wires":[["2c7fcc53.d38034","eb7e04d9.1481f8"]]},{"id":"2c7fcc53.d38034","type":"debug","name":"RAW MQTT output: emonhub/rx/#","active":false,"console":"false","complete":"topic","x":242,"y":461,"z":"fa6ca080.05936","wires":[]},{"id":"eb7e04d9.1481f8","type":"switch","name":"Seperate Nodes","property":"topic","rules":[{"t":"cont","v":"5"},{"t":"cont","v":"19"},{"t":"cont","v":"20"}],"checkall":"true","outputs":3,"x":343,"y":317,"z":"fa6ca080.05936","wires":[["b0618044.4f9e8"],["9077dea0.6f882"],[]]},{"id":"9077dea0.6f882","type":"csv","name":"emonTH 1: emonhub/19/values > Decode MQTT CSV","sep":",","hdrin":"","hdrout":"","multi":"one","ret":"\\n","temp":"temp,temp_external,humidity,battery,pulsecount","x":664,"y":373,"z":"fa6ca080.05936","wires":[["75ee731f.8a118c"]]},{"id":"b0618044.4f9e8","type":"csv","name":"emonPi: emonhub/5/values > Decode MQTT CSV","sep":",","hdrin":"","hdrout":"","multi":"one","ret":"\\n","temp":"power1,power2,power1_plus_2,Vrms,temp1,temp2,temp3,temp4,temp5,temp6,pulseCount","x":647,"y":236,"z":"fa6ca080.05936","wires":[["cda897fa.325768"]]},{"id":"cda897fa.325768","type":"debug","name":"Debug: emonPi Power1 output","active":false,"console":"false","complete":"payload.power1","x":1015,"y":235,"z":"fa6ca080.05936","wires":[]},{"id":"75ee731f.8a118c","type":"function","name":"If emonTH battery < 1.7V","func":"if (msg.payload.battery < 1.7) {\n   return {payload:'WARNING: emonTH battery low  '+msg.payload.battery+'V'  };\n}","outputs":1,"noerr":0,"x":1015,"y":363,"z":"fa6ca080.05936","wires":[["f8d8fd72.0727"]]},{"id":"f8d8fd72.0727","type":"delay","name":"Limit 1 email per day","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","rateUnits":"day","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"x":948,"y":447,"z":"fa6ca080.05936","wires":[["1d4154b2.e2beab"]]},{"id":"1d4154b2.e2beab","type":"e-mail","server":"smtp.gmail.com","port":"465","name":"","dname":"Send warning email","x":1198,"y":439,"z":"fa6ca080.05936","wires":[]}]