OpenEnergyMonitor Meetup

We had a very enjoyable OpenEnergyMonitor meetup last weekend. Ken Boak and Paul Allen (MarsFlyer)  came up from London, Matt Fawcett from Manchester and Carlos Alonso Gabizó from Machynlleth.

From left to right: Glyn, Matt, Ken
We did a surprising amount of development in such as short time as well having some great discussions. Here's a summary of what we worked on:

Controlling wireless nodes from emoncms
Ken Boak and I worked on furthering a way for emoncms to be able to send commands back to the gateway and wireless nodes, which could be used as part of a system for turning on and off heating systems and setting set point temperatures. There's now a new optional emoncms module called command where you can add commands to a queue that can then be requested by the gateway. Download the module here: https://github.com/emoncms/command



Gas monitoring 
There was a lot of discussion around gas monitoring. Paul brought with him a jeenode with a hall effect sensor connected for sensing the magnetic pulse given off by certain types of gas meter. The jeenode goes to sleep between pulses so that it can run on a single AA battery for a long time which is a big benefit for installations where there are no near electrical sockets. Glyn worked on adding this to the gas monitoring page on the website here: http://openenergymonitor.org/emon/buildingblocks/gas-meter-monitoring

Paul also implemented a pulse to power input processor for emoncms to allow conversion of pulse data coresponding to energy use increments to power values for use with the low power gas pulse counter.

EmonTx accuracy experiment replication
Carlos replicated the EmonTx accuracy experiment, with the results comparing well with the previous investigation take a look at his results here http://openenergymonitor.org/emon/emontx/accuracy



An open source SAP calculator - Matt and I made a start on an open source web based SAP worksheet - SAP is the UK Government’s Standard Assessment Procedure for Energy Rating of Dwellings produced by BRE the Building Research Establishment. We've put the progress we have made so far up on github here: https://github.com/emoncms/sap 
Please get in touch if your interested in helping to develop this, we could do with more help with completing the form and checking and implementing the remaining calculations.
Matt is working on a really exciting project in Manchester with Carbon Coop to do building retrofits of which monitoring and sap calculation is an important part, its well worth a look at their work here http://carbon.coop/



We also looked at humidity sensing and built up a couple of low-power wireless humidity nodes using an emonTx's with a DHT22 humidity sensor.

Throughout the weekend I think we had a good emphasis on how the tools that we are developing are applied with much discussion of how it all fits into the wider picture of sustainable energy. 

Finishing the first day with well earned homebrew ale thanks to Carlos
From left to right: Paul, Carlos, Matt, Ken, Trystan (Glyn taking the photo)
It was a great weekend and I'm really looking forward to the next one already, we're thinking maybe next April on a bank holiday weekend.

RFM12B Wireless Transmission Explained


All OpenEnergyMonitor hardware modules (emonTx, emonGLCD emonBase etc) currently use the RFM12B 433/868/915Mhz module to communicate via wireless. 

We use the JeeLib Arduino library from JeeLabs as the software driver for the RFM12B.

Robert Wall has recently contributed a well written bit of documentation to explain the basics RFM12B wireless transmission:




RFM12B - Part 2 -
Sending data by radio between emonTx, emonBase and emonGLCD
The data is sent by radio between the sensor nodes, the base station (including the RFM12Pi) and the display. This note describes the way in which the data is assembled and addressed in order to pass values between units.

The Arduino-based emon modules are programmed with application sketches that make use of the JeeLib library to handle the transmission and reception of the data, using the function calls provided by the library. The RFM12Pi is programmed in PHP but uses exactly the same methods and data structure. Appendix 1 contains instructions for configuring the Raspberry Pi module.


Configuring & Addressing
All the devices that communicate with each other must all use the same frequency. For the Arduino-based devices, this is specified in the software and must be one of the pre-defined constants. It must match the frequency of the hardware. (You should check which frequencies are permitted in your locality. It is possible to operate at the wrong frequency, e.g. 868 MHz hardware at 433 MHz, but the range can be very small, maybe less than 1 m).

The addressing scheme has two parts, the “Network Group” and the “Node ID”. The RFM12B module supports 250 network groups, the RFM12 module can have only one network group, 212. Only devices that belong to the same group can talk to and listen to each other. If you like, you can think of a network group as being like a room: everyone in the room can talk to anyone else, but can neither talk to nor hear someone in a different room.

Within the network group, there can be 31 Node Ids, but nodes 0 and 31 are special. Node 0 is used for On/Off Keying (OOK) and node 31 is for broadcast messages. The way we use it, each device must have its own unique ID. You can think of the ID as the name of a person in the room. Appendix 2 lists our standardised Node IDs.

There is one further but very important restriction: all the devices in all the network groups all use the same frequency, so only one device is allowed to transmit at any time.

This documentation forms part 2 of RFM12B documentation. Part 1 details an overview of the RFM12B module.

I would also like to take this opportunity to thank ukmoose for his contributions to the Raspberry Pi RFM12pi documentation wiki page.

Public and private feeds

Another new feature in the latest commit is what I hope is clearer feed authentication levels:

Public access to feeds - no authentication required accessible to everyone - these are for use in public dashboards which also require no authentication, but is wider than dashboards only as it also means sharing feed data publicly in general either directly through the feed api or visualisations.

Private feeds

Read only access to feeds - accessible via read apikey (but not public unless you share your read apikey publicly) useful for sharing with someone privately, ie sending a link via email to a trusted person. To do this you make the feed private and send the link to whatever it is you want to share privately with an &apikey=YOURREADAPIKEY at the end.

Write access to feeds - accessible via write apikey, strictly a private apikey (not to be shared as this would give others write access to your account). Write access is needed for connected devices such as the basestation to upload data to emoncms.

All feeds are by default private. To make a feed public click on the lock icon in the feed list, the globe icon represents public access:


This is my feed list, I have granted public access to my solar hot water system temperatures and made the house power private as this could be particularly sensitive data as it would be possible to see my daily activity in detail.

Private feeds and public dashboards

If you include a private feed in a public dashboard, the dashboard will not be able to retrieve the feed data and your dashboard is likely to look like this:


As all feeds with the implementation of this feature have been set to private all public dashboards will either have dials that are 0 and possibly visualisations that show the above message. If your visualisation is still working it may be because the javascript from the page is cached in your browser and you need to clear it.

If you see any bugs or to see the latest status of bug fixing and features on their way have a look at the bug and dev list


Multiple multigraphs

A long asked for feature for emoncms has been the ability to create multiple multigraph type visualisations. One might be a comparison of room temperatures another for hot water system temperatures, maybe another for room temperature and electrical or gas energy input?

This feature has now been implemented and is integrated fully into the visualisation selector page. If you have your own emoncms server you can download the latest emoncms from the usual place: github.com/emoncms/emoncms  and emoncms.org has also been updated.

Unfortunately due to the large rewrite of the multigraph code the new multigraph is not backwards compatible with the old multigraph, but I hope you will persist with the new implementation as the gains are well worth it. Old multigraphs appear as a list of undefined feeds, my advice is to delete the old multigraph and start with a new one.

Here is a brief walk through of the feature:

1) Start by going to the Vis tab and select the multigraph visualisation from the first drop down menu. In the second box you should now see a select multigraph drop down menu and a new + button. Click on the plus icon to create a new multigraph:


This will refresh the page, navigate again to multigraph in the first drop down menu, you should now see the new multigraph id in the drop down menu. Click/select the new dashboard id:


Select a feed from the add feed drop down menu and click on Add, this will add a feed to the multigraph feed list table. Put a tick in the left or right checkbox to specify the axis you wish the feed to appear on, you should now see the feed appear:


Add a few more, put a tick in the fill box to fill in the area below the curve:


Select a default time range by clicking on the day/week/month/year buttons. When you reload the multigraph it should now appear at the time range given. If you'd like the multigraph to load a specific fixed time range just zoom in on the area you want it to show, the next time you load the multigraph it should load your last view.


Add a multigraph to your dashboard by selecting multigraph from the dropdown visualisation menu. 
Click on the blank dashboard and click on the Options button. Select the multigraph you wish to show from the drop down menu and click save.

Love to hear any feedback on it, whether it worked or didn't work for you, may be worth clearing the cache if it does not work as a few of the javascript scripts have been updated. Enjoy!


OpenEnergyMonitor Shop Christmas Ordering

Looking up at Snowon 1085m from Llyn Glasyn yesterday (20 min north of OEM labs), North Wales UK

The first snow of the year has fallen onto the Welsh mountains and the emoncms Bee Hive Temperature Monitor has recorded sub-zero temperatures in the valleys. This can only mean that winter is on its way! 

The OpenEnergyMonitor online shop has been nice and busy recently; breaking the 1000th order mark a couple of weeks ago was a big milestone. Thank you to all of you who've ordered from the shop, you're directly helping to support the project. The success of the shop has enabled us to contribute to the project full time; we have lots of ideas for 2013. 

We will be running the shop and shipping orders as fast as we can right up until Christmas, if you're hoping to receive your items before Christmas we recommend you place your order before the following dates: 

UK:
1 st class: 18th Dec
Special Delivery: 20th Dec

Europe:
Airmail/Airsure: 10th Dec 

Outside Europe (USA):
Airmail/Airsure: 8th Dec 

Outside Europe (NZ/Australia):
Airmail/Airsure: 3rd Dec

If you don't manage to get your order in before the dates above, don't worry there is still a good chance it will arrive, we will be shipping as fast as we can right up until our local post office shuts on 23rd Dec.  

Merry Christmas or as we say in Welsh Nadolig Llawen.


The view from Plas Y Brenin early this morning looking out over Llyn Mymbur  (30 min north of OEM labs), North Wales UK





OpenEnergyMonitor meetup

Were having an openenergymonitor meetup, hack weekend the weekend after next (8th-9th of December) up here at the openenergymonitor lab in Snowdonia, North Wales. This is an open invitation, if you have the weekend free (I appreciate the short notice) and would like to come please send me an email: trystan.lea at openenergymonitor.org It will be a fairly low key event with plenty of building, coding, chatting, probably a pub visit and enjoying the mountain air.

For a taste of what its like have a look at the blog posts for the last meetup we had:

http://openenergymonitor.blogspot.co.uk/2011/05/snowdonia-build-april-2011.html

And the video of a day at openenergymonitor:



Here's the location:


View Larger Map

If your interested please get in touch.

Hope to see you there!

Trystan





Solar PV excess power into immersion water heater using emonTx, emonGLCD & MKII triac code - in development

From the forums: Solar PV excess power into immersion water heater using emonTx, emonGLCD & MKII triac code

jrobbie (on the forums) writes:

As there appears to be a considerable amount of interest in getting an emonTx unit running Robin’s MK2 code (calypso_rae on this forum) and communicating with emonGLCD / emon Base … I thought it was about time I publish where I am at with my project.

The emonTx with CT1 connected to the Grid, and CT3 connected to the Solar PV generation meter, AC psu and 5v USB supply are also connected.

emonGLCD with modified code
emonTx with additional tri-color LED to indicate immersion heater status

Consumer unit containing MOC3041 triac
So many thanks to Robin for the MK2 code, Ian for the Calibration code, Glyn and Trystan for the building blocks as well as Paul Reed, Stuart and Robert Wall who have all made major contributions via this forum and get us to where we are today.
Read full post including emonGLCD and emonTx software sketches on the forums here

Queuing packets on the basestation and bulk sending to emoncms

As we connect more and more wireless nodes to a basestation, with each wireless node transmitting every 10s or so we quickly start to have a situation where the basestation is trying to make several connections to the server every second.

The question is, does the rate of connections matter? Is it bad news for the emoncms server and what is the effect on the local internet connection, router?

Although I have not investigated this in detail, Im pretty sure Im seeing an adverse effect on my internet connection if the post rate gets too high, I noticed this especially the other day when I changed my router to a bt homehub after a lighting strike took out the old router. It would be great to get some good data on this.

Then there is the question of server load which needs more investigation.

Jumping to a solution before the problem is properly characterized, the solution that comes to mind is to queue packets received from wireless nodes in the basestation and then do a bulk post at a slower rate, such as once every 20s or longer.

I have put together an initial working example of this, including code for both the NanodeRF basestation end and the emoncms server end.

Starting with the NanodeRF code:
https://github.com/openenergymonitor/NanodeRF/tree/master/NanodeRF_multinode_bulksend

The NanodeRF code queues packets received, including a time offset between packets so that they can be stored in emoncms at the right time when they get there (although the implementation for this is a little rough at the moment).

The packets are queued in a string that can be sent via the ethernet and then decoded simply in emoncms using json_decode();

A string for 3 packets from the different nodes looks like this:
[[0,16,1137],[2,17,1437,3164],[4,19,1412,3077]]

The first node: [0,16,1137]

The first number of each node is the time offset, so for the first node it is 0 which means the packet for the first node arrived at 0 seconds. The second node arrived at 2 seconds and 3rd 4 seconds.

The second number is the node id, this is the unqiue identifer for the wireless node.

All the numbers after the first two are data values. The first node here (node 16) has only once data value: 1137.

This string is sent to emoncms every 20s in the example and there is enough room in the string buffer to store up to 400 characters or about 22 low power temperature node packets, this may be extendible by another 200 characters for more storage, and maybe a lot further with an SD card or other larger storage medium.

The emoncms api for sending bulk data is emoncms.org/input/bulk?data=

Emoncms then decodes the string, registers and processes the inputs and returns an ok.

If your interested in testing this out, here's the NanodeRF bluksend example again, if your using the NanodeRF_multinode code already this is just a straight switch to the bulk send code:
https://github.com/openenergymonitor/NanodeRF/tree/master/NanodeRF_multinode_bulksend

If your running your own server you will need the latest emoncms version:
https://github.com/emoncms/emoncms or if you are using emoncms.org its all ready to go there.

There's a couple of question marks at the moment as to the stability of this initial working example, I had a few problems with the queue string getting malformed but only once every 20-30 successful string builds and at the moment the nanodeRF does not reset itself using the watchdog if there are more than 10 unsucessful attempts. A few extra eye's on the code would definetly help to improve it and I will keep testing here.

Raspberry Pi emonBase

The Raspberry Pi needs no introduction, it's a great piece of hardware. A low power, low cost Linux credit-card sized computer made in the UK. As soon as we heard of the Raspberry Pi we knew it would make a great emonBase energy monitoring base station. 

Raspberry Pi with RFM12Pi Expansion

The Raspberry Pi is powerful enough to run a web server running emoncms open-source logging and visualisation web-app while being low power enough to be left running 24/7.  

Using the Raspberry Pi to log energy monitoring data locally (onto it's SD card) has advantage of not having to be reliant on a stable web connection when logging to a remote sever and gives you total control over your data. 

Working with Martin Harizanov we have designed an open-source expansion board for the Raspberry Pi which enables the Pi to receive data via wireless from other OpenEnergyMonitor modules e.g emonTx energy and temperature monitoring node or an emonGLCD wireless LCD display. The RFM12Pi is now available to buy through the OpenEnergyMonitor shop.

RFM12Pi Raspberry Pi GPIO Wireless Expansion Board  

A Raspberry Pi running emoncms with an RFM12Pi expansion board can be used as a powerful emonBase base-station to log, process and visualise energy, temperature and other environmental data. Data can be logged locally to the Raspberry Pi's SD card and/or to a remote emoncms server. Emoncms graphs and dashboards are served from the Raspberry Pi's web-server. Checkout the documentation links below for more information and setup instructions. In the future we plan to release a 'ready to go' Pi SD card image with emoncms pre installed and configured to work with the RFM12Pi.  

Thanks to Baptiste Gaultier for this nice emoncms dashboard example

Documentation Links:




Updated installation and upgrade guides for latest emoncms release

Following on from my previous post on the new modular version of emoncms: http://openenergymonitor.blogspot.co.uk/2012/10/emoncms-development-update-modules.html

Here are the updated installation and upgrade guides:


Linux installation Ubuntu/Debian/PI

Windows installation

Here are some of the new features, interface improvements in the latest version:

New feeds interface with tag based grouping and in-line editing:


New visualisation page interface that makes it easier to explore the different visualisations available including drop down menu's for selecting feeds.


Database schema upgrade script can now be called from within the admin account (first account created).


Modular implementation as detailed in the previous blog post:
Several optional modules available for extending emoncms such as the raspberry pi module for communication with the RFM12PI board.



I have created another bug and dev list forum post for this month, November here: 

Emoncms bug and dev list NOVEMBER 2012