Multiple multigraphs

A long asked for feature for emoncms has been the ability to create multiple multigraph type visualisations. One might be a comparison of room temperatures another for hot water system temperatures, maybe another for room temperature and electrical or gas energy input?

This feature has now been implemented and is integrated fully into the visualisation selector page. If you have your own emoncms server you can download the latest emoncms from the usual place:  and has also been updated.

Unfortunately due to the large rewrite of the multigraph code the new multigraph is not backwards compatible with the old multigraph, but I hope you will persist with the new implementation as the gains are well worth it. Old multigraphs appear as a list of undefined feeds, my advice is to delete the old multigraph and start with a new one.

Here is a brief walk through of the feature:

1) Start by going to the Vis tab and select the multigraph visualisation from the first drop down menu. In the second box you should now see a select multigraph drop down menu and a new + button. Click on the plus icon to create a new multigraph:

This will refresh the page, navigate again to multigraph in the first drop down menu, you should now see the new multigraph id in the drop down menu. Click/select the new dashboard id:

Select a feed from the add feed drop down menu and click on Add, this will add a feed to the multigraph feed list table. Put a tick in the left or right checkbox to specify the axis you wish the feed to appear on, you should now see the feed appear:

Add a few more, put a tick in the fill box to fill in the area below the curve:

Select a default time range by clicking on the day/week/month/year buttons. When you reload the multigraph it should now appear at the time range given. If you'd like the multigraph to load a specific fixed time range just zoom in on the area you want it to show, the next time you load the multigraph it should load your last view.

Add a multigraph to your dashboard by selecting multigraph from the dropdown visualisation menu. 
Click on the blank dashboard and click on the Options button. Select the multigraph you wish to show from the drop down menu and click save.

Love to hear any feedback on it, whether it worked or didn't work for you, may be worth clearing the cache if it does not work as a few of the javascript scripts have been updated. Enjoy!

OpenEnergyMonitor Shop Christmas Ordering

Looking up at Snowon 1085m from Llyn Glasyn yesterday (20 min north of OEM labs), North Wales UK

The first snow of the year has fallen onto the Welsh mountains and the emoncms Bee Hive Temperature Monitor has recorded sub-zero temperatures in the valleys. This can only mean that winter is on its way! 

The OpenEnergyMonitor online shop has been nice and busy recently; breaking the 1000th order mark a couple of weeks ago was a big milestone. Thank you to all of you who've ordered from the shop, you're directly helping to support the project. The success of the shop has enabled us to contribute to the project full time; we have lots of ideas for 2013. 

We will be running the shop and shipping orders as fast as we can right up until Christmas, if you're hoping to receive your items before Christmas we recommend you place your order before the following dates: 

1 st class: 18th Dec
Special Delivery: 20th Dec

Airmail/Airsure: 10th Dec 

Outside Europe (USA):
Airmail/Airsure: 8th Dec 

Outside Europe (NZ/Australia):
Airmail/Airsure: 3rd Dec

If you don't manage to get your order in before the dates above, don't worry there is still a good chance it will arrive, we will be shipping as fast as we can right up until our local post office shuts on 23rd Dec.  

Merry Christmas or as we say in Welsh Nadolig Llawen.

The view from Plas Y Brenin early this morning looking out over Llyn Mymbur  (30 min north of OEM labs), North Wales UK

OpenEnergyMonitor meetup

Were having an openenergymonitor meetup, hack weekend the weekend after next (8th-9th of December) up here at the openenergymonitor lab in Snowdonia, North Wales. This is an open invitation, if you have the weekend free (I appreciate the short notice) and would like to come please send me an email: trystan.lea at It will be a fairly low key event with plenty of building, coding, chatting, probably a pub visit and enjoying the mountain air.

For a taste of what its like have a look at the blog posts for the last meetup we had:

And the video of a day at openenergymonitor:

Here's the location:

View Larger Map

If your interested please get in touch.

Hope to see you there!


Solar PV excess power into immersion water heater using emonTx, emonGLCD & MKII triac code - in development

From the forums: Solar PV excess power into immersion water heater using emonTx, emonGLCD & MKII triac code

jrobbie (on the forums) writes:

As there appears to be a considerable amount of interest in getting an emonTx unit running Robin’s MK2 code (calypso_rae on this forum) and communicating with emonGLCD / emon Base … I thought it was about time I publish where I am at with my project.

The emonTx with CT1 connected to the Grid, and CT3 connected to the Solar PV generation meter, AC psu and 5v USB supply are also connected.

emonGLCD with modified code
emonTx with additional tri-color LED to indicate immersion heater status

Consumer unit containing MOC3041 triac
So many thanks to Robin for the MK2 code, Ian for the Calibration code, Glyn and Trystan for the building blocks as well as Paul Reed, Stuart and Robert Wall who have all made major contributions via this forum and get us to where we are today.
Read full post including emonGLCD and emonTx software sketches on the forums here

Queuing packets on the basestation and bulk sending to emoncms

As we connect more and more wireless nodes to a basestation, with each wireless node transmitting every 10s or so we quickly start to have a situation where the basestation is trying to make several connections to the server every second.

The question is, does the rate of connections matter? Is it bad news for the emoncms server and what is the effect on the local internet connection, router?

Although I have not investigated this in detail, Im pretty sure Im seeing an adverse effect on my internet connection if the post rate gets too high, I noticed this especially the other day when I changed my router to a bt homehub after a lighting strike took out the old router. It would be great to get some good data on this.

Then there is the question of server load which needs more investigation.

Jumping to a solution before the problem is properly characterized, the solution that comes to mind is to queue packets received from wireless nodes in the basestation and then do a bulk post at a slower rate, such as once every 20s or longer.

I have put together an initial working example of this, including code for both the NanodeRF basestation end and the emoncms server end.

Starting with the NanodeRF code:

The NanodeRF code queues packets received, including a time offset between packets so that they can be stored in emoncms at the right time when they get there (although the implementation for this is a little rough at the moment).

The packets are queued in a string that can be sent via the ethernet and then decoded simply in emoncms using json_decode();

A string for 3 packets from the different nodes looks like this:

The first node: [0,16,1137]

The first number of each node is the time offset, so for the first node it is 0 which means the packet for the first node arrived at 0 seconds. The second node arrived at 2 seconds and 3rd 4 seconds.

The second number is the node id, this is the unqiue identifer for the wireless node.

All the numbers after the first two are data values. The first node here (node 16) has only once data value: 1137.

This string is sent to emoncms every 20s in the example and there is enough room in the string buffer to store up to 400 characters or about 22 low power temperature node packets, this may be extendible by another 200 characters for more storage, and maybe a lot further with an SD card or other larger storage medium.

The emoncms api for sending bulk data is

Emoncms then decodes the string, registers and processes the inputs and returns an ok.

If your interested in testing this out, here's the NanodeRF bluksend example again, if your using the NanodeRF_multinode code already this is just a straight switch to the bulk send code:

If your running your own server you will need the latest emoncms version: or if you are using its all ready to go there.

There's a couple of question marks at the moment as to the stability of this initial working example, I had a few problems with the queue string getting malformed but only once every 20-30 successful string builds and at the moment the nanodeRF does not reset itself using the watchdog if there are more than 10 unsucessful attempts. A few extra eye's on the code would definetly help to improve it and I will keep testing here.

Raspberry Pi emonBase

The Raspberry Pi needs no introduction, it's a great piece of hardware. A low power, low cost Linux credit-card sized computer made in the UK. As soon as we heard of the Raspberry Pi we knew it would make a great emonBase energy monitoring base station. 

Raspberry Pi with RFM12Pi Expansion

The Raspberry Pi is powerful enough to run a web server running emoncms open-source logging and visualisation web-app while being low power enough to be left running 24/7.  

Using the Raspberry Pi to log energy monitoring data locally (onto it's SD card) has advantage of not having to be reliant on a stable web connection when logging to a remote sever and gives you total control over your data. 

Working with Martin Harizanov we have designed an open-source expansion board for the Raspberry Pi which enables the Pi to receive data via wireless from other OpenEnergyMonitor modules e.g emonTx energy and temperature monitoring node or an emonGLCD wireless LCD display. The RFM12Pi is now available to buy through the OpenEnergyMonitor shop.

RFM12Pi Raspberry Pi GPIO Wireless Expansion Board  

A Raspberry Pi running emoncms with an RFM12Pi expansion board can be used as a powerful emonBase base-station to log, process and visualise energy, temperature and other environmental data. Data can be logged locally to the Raspberry Pi's SD card and/or to a remote emoncms server. Emoncms graphs and dashboards are served from the Raspberry Pi's web-server. Checkout the documentation links below for more information and setup instructions. In the future we plan to release a 'ready to go' Pi SD card image with emoncms pre installed and configured to work with the RFM12Pi.  

Thanks to Baptiste Gaultier for this nice emoncms dashboard example

Documentation Links:

Updated installation and upgrade guides for latest emoncms release

Following on from my previous post on the new modular version of emoncms:

Here are the updated installation and upgrade guides:

Linux installation Ubuntu/Debian/PI

Windows installation

Here are some of the new features, interface improvements in the latest version:

New feeds interface with tag based grouping and in-line editing:

New visualisation page interface that makes it easier to explore the different visualisations available including drop down menu's for selecting feeds.

Database schema upgrade script can now be called from within the admin account (first account created).

Modular implementation as detailed in the previous blog post:
Several optional modules available for extending emoncms such as the raspberry pi module for communication with the RFM12PI board.

I have created another bug and dev list forum post for this month, November here: 

Emoncms bug and dev list NOVEMBER 2012

Ciseco Visit

Earlier this week I visited Miles Hodkinson who runs Ciseco.

Ciseco supply us with the Open Kontrol Gateway web-connected base station, however Ciseco are best known for their range of wireless modules based on the Texas Instruments CC1110 chip. Most of Ciseco's products are manufactured by themselves in the UK. It was a fascinating visit to see the manufacturing process in action. I took a few photos which might be of interest to those of your who are interested in how things are made.

The surface mount electronics assembly process has got three main stages:

1. Apply solder paste on the PCB pads, this is done using a scraper and carefully aligned laser cut stencil.

Solder pate application

2. The pick and place machine places the SMT components in their correct locations and orientation on the board

Pick and place - the components are supplied on the white reels

3. After a quick visual check the board with the placed components is put in the reflow oven for a precise amount of time at a carefully monitored temperature profile. This melts the solderpaste and solders the components into place.

Reflow Oven
The finished boards are then taken away to be flashed, tested then finally the through-hole components such as the antenna socket are soldered on by hand.

The finished boards 
 View the full galley of photos including some videos of the pick and place in action:

Ciseco visit

Pre-assembled Nanode RF

We're happy to announce that we are now selling Nanode RF web-connected emonBase gateways pre-assembled. This makes it even easier and quicker to get started with energy monitoring: 

The pre-assembled Nanode RF using surface mount electronics has been designed and assembled in the US by Wicked Devices. The Nanode was originally designed by our friend Ken Boak. Ken has written  an interesting blog post giving a brief history of the Nanode.

Ken writes: 

Nanode was conceived back in the summer of 2010, when I returned from a trip to Shenzhen, China. I was between jobs, and had a bit of time on my hands for tinkering with some new product ideas.
Nanode arose out of the need to find a cheaper way of connecting simple open source hardware, such as Arduino to the internet, so that Arduino could be used for a range of sensing and control applications.

Early Nanode prototype controlling an RGB lamp
Read his full blog post here:

Emoncms development update: Modules

I've been a little quiet over the last few weeks, emoncms blogposts and progress updates have been far and few between but that does not mean there has been no development progress happening, to the contrary there has been a lot of significant development work been going on.

I outlined at the start of the month in the october emoncms bug and dev list the idea of re-factoring/reorganising emoncms along more modular lines, pointing to some earlier thoughts on it. Since then I made the leap and started the re-factoring process, Ilde gave me a lot of help with splitting up the language files for translations, and good ideas for menu system implementation, Baptiste Gaultier  gave me some impetus to look at modularising the dashboard widget code and now we are almost there.

The old structure had a folder for Controllers, Models and Views, with all the different controllers, model and view scripts for a feature mixed in together in these folders and also some files such as setup.php which had code in it for all the different features. As the application was growing this was getting a bit unwieldy, harder to understand/read the code, add new features and keep track of development.

This is how the old structure looked like:

  feed_controller.php (part of feed module * )
  input_controller.php (part of input module * )
  feed_model.php (part of feed module *)
  input_model.php (part of input module *)
  feed_list_view.php (part of feed module *)
  input_list_view.php (part of input module *)
setup.php (mixed code for input, feeds, etc) - if you add a new module you had to add db schema code in here.
notify.php (part of notify module but in the root emoncms directory)
* Module or feature specific files are all in different locations 

By changing the structure:

    feed_list_view.php (includes no feed message)
    input_controller.php (merged with enum.php)
    input_list_view.php (includes no input message)

and following the following design principles:

  • A module add's functionality in a self-contained way - not requiring (or minimizing as much as possible) modification of any other modules or the core framework.
  • A module can depend on another module. ie dashboards depend on feeds but feeds does not need to depend on dashboards.

I think this change gives us the following benefits:

  • Clearer application structure - easy to find module specific code
  • Adding a new module is really easy, you dont have to put the controller in the controller folder, the view in the views folder, model in the models folder, database setup in setup.php, all you have to do is download the module folder and place in the modules folder.
  • We can then use git to develop particular modules independently of the rest. A developer looking at commit logs therefore only needs to concern themselves with that particular module rather than the application as a whole.
  • Its easier to test modules and the framework independently of each other.
  • More developer autonomy, project self organisation. You can add a module to the optional modules list without having to coordinate closely with the main emoncms bundle.
The new modular emoncms version can be found here:

Given the potential escalating number of repositories as new modules are added I though it would be best to create a dedicated github organisation for emoncms hence the new location. If you go to: you will see quite a few repositories:

The emoncms repository is the main emoncms bundle which is build up from the emoncms_framework and the other core modules such as input, feed, dashboard and so on. Then there are repositories for the framework and modules independently of the main bundle including modules that are not in the main bundle such as the raspberry pi module which I will come back to in the next post.

The idea being that the main bundle could be built automatically from the module repositories and the framework and that the commit history could be on a module basis.

Adding a module
To add a particular non core feature such as the raspberrypi interface module to your emoncms installation its now just a matter of downloading the raspberrypi repository and placing it in the modules folder of your emoncms installation, much the same way as you add libraries to an arduino project.

Things to complete
As I mentioned at the start most of the refactoring work has been done but there are still some parts that are not complete such as the notify, admin and statistics module and there are some missing message boxes such as when no inputs, feeds or dashboards exist that need to be re-added.

The feed module implementation is also a little different in that the there is no longer a feed_relation table that used to link a user to their feeds, this is now handled in the feeds table which simplifies things but means a simple database migration process needs to take place if your upgrading from an old emoncms version, I will write a short script to do this soon and document the migration process.

Another important change is the API used to post data from basestations to emoncms what used to be api/post is now input/post as this action is handled by the input module. You can if you want sidestep input processing completely and just use the feed module which now has a full api. The api calls for both the feed and input module are documented in helper pages linked to from the top of the input and feed pages.

That's it for now, in the next few post I will discuss the raspberrypi module and also how to migrate from the old emoncms version.