How to contribute on OpenEnergyMonitor dev + development plans


If your wondering how best to get involved with OpenEnergyMonitor development here are a couple of thoughts and possible pointers.

First a few insights and ideas from an another open source project we follow: DIYdrones. Thanks to Amin Zayani for the heads up on these: [1] [2] [3]
In Chris Anderson of DIY drones's words, who we think has articulated it well:
“The best way to participate is to do something cool on your own and share it here. All of our code bases and hardware design files are open. If you see an opportunity to improve something, just do it, post it and then tell the community about what you've done. If it's good, people will use it, help improve it, and we'll get a sense of what you can do. That makes it much easier for us to figure out what your skills are and where you'd fit in best.”

If your looking for ideas here are a list of things we would like to do but no one is currently leading on them, Many of these problems are quite large, it would probably good to break them down into smaller sub projects. If you have a feature you would like to implement or would like to take the lead on any of the features below that would be really great, please post with your thoughts.
  • Appliance inference - infer from a single power feed of a house what each appliance uses and so providing a energy use by appliance breakdown.
  • PCB and casing for the 12 input pulse counter (a DIN rail and non din rail option)
  • Plug based energy monitor - for per appliance monitoring  - an alternative method to appliance inference.
  • Larger monitoring PCB's: maybe 12 input CT?
  • 3 phase monitoring sketch for emontx - quite a few people are asking for this so this would be really useful.
  • Develop a series of control modules, that extend the emon system into an open source monitoring and control system that can be used for domestic and industrial situations. Control modules could include:
    • multiple relay board (for dump load control, immertion heaters, pumps, lighting)
    • solenoid valve driver board for central heating control.
    • motor control
    • This emon monitoring and control system could link together by either a wireless bus say rfm12 or a wired bus (maybe rs485). The modules could be designed to fit in both DIN rail style casing or non-DIN rail casing.
  • Merge emonTx firmware examples into an Arduino library.
Short term goals
For a general overview here are the short term goals that are being worked on at the moment and in the very near future:
  • New casing design for emontx and emonbase
  • Transfer emonbase sketches over to EtherCard library
  • Compatibility of main sketches with Arduino 1.0
  • Complete watchdog addition
  • Emoncms development see feature plans and discussion page on forums.
  • Complete and publish energy audit software v1.0
  • Improve full emon system documentation.
This blogpost is repeated as a forum post here, lets discuss this on the forum as its a better place for it.

Thankyou
Trystan


How to move feed data between servers

There is often a need to transfer large feed tables between different servers. Here is one way to do it via ssh that does not require downloading the data to a local machine. This makes it particularly fast especially if your local internet connection is relatively slow.

1) Login via ssh to the source server
ssh [email protected] (source server ip)

2) Export feed table from source instance database
mysqldump -u username -p database feed_1 > feed_1.sql

3) Transfer feed table via ssh to destination instance server
tar -cjvf - feed_1.sql | ssh [email protected] tar -xjf - -C /home/user (destination server ip)

4) Login via ssh to the destination server
ssh [email protected] (destination server ip)

5) Import feed table into destination instance database
mysql -u root -p database < feed_1.sql

It would be great to have some kind of utility within emoncms that makes it possible to move large feed tables between emoncms instances efficiently and easily without having to use terminal and ssh. If you know of an efficient way of doing this, advice would be most welcome.

Open source and software as a service

In addition to developing emoncms as an application that can be downloaded and installed on any server, interest willing, I would like to work towards offering emoncms as a service. So if you just want to build up your hardware and connect to a ready to go, maintained service with support, the option will be available to you.

One of the reasons I wanted to develop emoncms in the first place was that I was frustrated with fact that all available services where proprietary, I wanted to be able to download, install, modify, take part in development because I very much enjoy doing so and wanted to take part in collaborative innovation on important problems as Irving Wladawsky-Berger described so well in his post on the essence of open source which I blogged about last week.

As this article by mygnulinux.com and another article here by William Judd describe, there is a concern that as more an more of our applications move into the cloud and are run by organisations who adopt the proprietary software as a service business model we will loose the ground that we have gained in the open source software movement, there is also a complication to do with licensing.

There are however a great deal of advantages in having applications in the cloud many of which are also very beneficial for energy monitoring, such as: access to your data from any location and any device and possibilities that open up by using shared platforms.

The answer: cloud based software as a service does not have to be proprietary software, it can be open source and this opens up some really exciting opportunities.

So the idea is that the installed web application's code is placed on github or other code sharing location and is a mirror of the installation on the main server/servers. The service can be used as one would normally use a software service. However if you want to improve the code, modify, add something you can. You just need to fork the repository, do something cool, share it with the community and if its adoption is agreed it can be added to the service install. So this allows for taking part in development, for modification and improvement. It is a little more complicated than editing your own instance of a particular piece of software as changes need to be agreed on by those using the service as any changes will effect everyone, but this is not a prohibitive problem as there are systems that have already been worked out for determining what makes it into the final release in many large open source projects.

As mygnulinux.com and William Judd's articles explain, the other important problems in software as a service is: lock in, data ownership and licensing.

The first can be addressed by creating a comprehensive API that makes it possible to move data from one service to another or to a local installation if one desires to do so.

The second can be addressed through good policy by the service providers.

Licensing is important. The GPL up to and including v3 does not ensure openness for software as a service provided over a network as the requirement to release the source code and any modifications to that source code is only triggered when the software is distributed not when that program is run. This essentially as I understand turns the GPL into BSD like licence as there is no requirement for open sourcing when providing the software as a service across a network.

In response to this loophole the Free Software Foundation bought out another licence called the GPL Affero licence (AGPL) which essentially requires anyone providing a service using the software across a network to release the source code, the one additional provision over GPL v3 being:
“if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software.”
In conclusion
It's possible to get the benefits of open source and the benefits of the cloud and software as a service platforms. I think its going to be exciting to explore this with emoncms.
  • The code for the latest version of emoncms is available on github: https://github.com/openenergymonitor/emoncms3
  • Its the exact same version as is deployed here: http://vis.openenergymonitor.org/emoncms3/
  • It is licensed under AGPL
  • A comprehensive API and tools for easy porting of data between instances is a core aim and will be an important part of continued feature development.
  • A template terms of service needs to be worked out and written.
Examples of open source software as a service
SugarCRM - a customer relationship management package (From the articles linked above)
Identi.ca - a micro-blogging service (From the articles linked above)
Thingspeak - internet of things application

Drop a comment below if you know of others that can be added to the list.



Essence of open source

I recently came across this post by Irving Wladawsky-Berger where he described what he see's as the essence of open source:
“To me, open source is all about collaborative innovation, that is, working with people all over the world as a community to solve important problems.”
He explains how this collaborative innovation is nothing new, that:
"Collaborating with colleagues in your discipline is how you make progress, whether that discipline is physics, medicine, computer sciences or law." and that open source is essentially "a necessary precondition for, collaborative innovation involving software"
If we think about the scale of what has been achieved by this collaborative innovation involving software, the shared IT infrastructure, the internet and how it has changed and continues to change so much, its quite staggering.

Irving finishes by concluding that
"These are such important and complex problems, that only through the kind of collaboration made possible by open source software can we hope to attain the needed levels of innovation and progress."
I think that this is a useful way to view our work as openenergymonitor. The field that we are working in is sustainable energy which is full of important and complex problems and to paraphrase Irving: it will be through the same open approach in sustainable energy that we can hope to attain the needed levels of innovation and progress required.

AC-AC Adapters

Today we have taken delivery of a load of AC-AC adapters in preparation for opening our online shop in the new year.

Using an AC-AC adapter with the emonTx to measure the AC voltage as well as a CT to measure current is is optional but it does increase accuracy by enabling real power measurements to be made. Using an emonTx with only a CT assumes RMS AC voltage to be fixed at a certain level (240V UK) and power factor to be equal to 1.

Real power measurements take into account the power factor of the AC circuit, inductive loads such as motors reduce the power factor thus giving an artificially higher apparent power reading. It is real power which you are billed for by the utility company. For more information on the differance between real and apparent power see AC power theory 1 and theory 2.

Using an AC-AC adapter also enables the direction of the current flow to be determined by providing a relative point of reference.  Knowledge of the direction of current is crucial in monitoring a domestic solar PV systems when due to how the generated power from the solar PV is connected into the fuse box consumption cannot be monitored separately to generation. More on this to come...

Our tests have shown that typically the apparent power reading of in a domestic home is 5-20% higher than a real power reading. Although this seems like quite a difference in reality the purpose of domestic energy monitoring is to increase understanding of how energy is consumed in the home, apparent power readings are accurate enough for this. Also installing an AC-AC with the emonTx next to the utility meter can be tricky since there are often no power sockets to hand. This offset can also be calibrated out by calibrating the apparent power kWh/d readings against the readings from the utility meter.

Instantaneous real and apparent power readings of a domestic house AC circuit during the night, notice the fridge (compressor motor) reduces the PF thus reducing the real-power reading. Power in W is on the left axis, PF is on the right axis 
As I mentioned at the beginning of the this post, we have just taken delivery of a load of AC-AC adapters, now time to test their quality.


The blue trace is the new adapter (pictures below) as you can see it's slightly better than the old one were were using (yellow trace), it is more like a perfect sine wave without the slight 'kink' in the middle that the old (yellow trace) adapter exhibits. Also the new model does not have the +2V offset, switching to the new model will require re-calibration of the emonTx in software.

The new AC-AC adapters. We have plenty in stock 

Testing these AC-AC adapters also gave us the chance to test out our new DSO Quad mini-digital scope. Very nice! 




Emoncms v3 beta

It is time for another big update to emoncms, our openenergymonitor open source energy visualisation web application!

Over the last four and a half weeks several different threads have come together all centering around the idea of widgets and dashboards, last week it came to a head with an in your browser dashboard editor, it was clear that a major release was called for and so here it is emoncms version 3 in good time for the holidays.

It is also optimised (well nearly) for viewing on mobile devices and there is for the first time a hosted demo site available for trying it out, see the launch page the link and use notes.



Ubuntu podcast Interview


A couple of weeks ago Laura Cowen and Tony Whitmore from Ubuntu UK podcast interviewed Trystan and myself at the HomeCamp 4 event in London. 

A recording from the interview can be heard 27 min into this weeks edition of the podcast. Download it here: http://podcast.ubuntu-uk.org/2011/11/23/s04e20-run-to-the-hills/

Trystan and I have both use Ubuntu as our main operating systems for the last few years. We rate it highly  and it keeps on getting better all the time. Its especially good for programming, we experience a lot less problems that windows users to when uploading code using the Arduino IDE. 

Thanks a lot to Laura and Tony, I'v added the Ubuntu UK podcast to my RSS reader. Its a good listen. 

HomeCamp4 Presentation Video

Here's a video of our presentation at the HomeCamp 4 a few weeks ago. Many thanks to Debbie Davies and Mike the Bee for filming editing and uploading the video:



The videos of the other presentations from the event can be viewed here: http://lanyrd.com/2011/homecamp4/video/

See our previous blog post for a copy of our slides, their not too clear in the video.

Home Camp 4 presentation slides

Trystan and I are currently traveling back to North Wales after a busy but enjoyable long weekend in London. We have been presenting the latest developments at OpenEnergyMonitor at the Open-Source Hardware Camp and Home Camp 4It was awesome to meet lots interesting and enthusiastic people often with similar interests and working on similar projects to ourselves. Ian from dangerous prototypes will be posting some blog posts with more details from the event. Check out his blog and super cool open-source hardware development tools here See below for a copy of the slides we presented at the events. OpenEnergyMonitor Home Camp 4, London Oct2011