emonTx V3 Progress Update

Since my last post introducing the emonTx V3  prototype design progress has been steady. The design has undergone several iterations and has now reached a stage of maturity.

The main features of the emonTx V3 have stayed the same:
  • ATmega328 Arduino IDE compatible microcontroller
  • RFM12B or Ciseco SRF 433/868/915Mhz RF wireless compatiable with RFM12Pi Raspberry Pi emoncms basestation
  • 3 x Standard (23kW max) CT channels
  • 1 x High sensitivity (4.5kW max) CT channel 
  • Integrated AC-DC power supply to enable powering the unit from a single 9V AC adapter while also sampling the AC voltage to calculate Real Power and AC Vrms readings 
  • Low power design with option to power from 3 x AA batteries for Apparent Power (current ony) measurement
  • Enclosed in wall mountable extruded aluminium enclosure
  • Terminal block connection for optical pulse sensor and DS18B20 temperature sensors
  • Pre-assembled SMT electronics
emonTx V3 with 3 x AA battery's and 1 x CT (Apparent Power setup)
Fully assembled with antenna in wall-mount enclosure

I am currently in the process of obtaining assembly quotes from manufactures as well as performing lots of testing.

emonTx V3.1 PCB Design 
emonTx V3.1 Schematic

The AC-DC circuit that was initially designed with the aid of simulation then bench tested is performing as expected.

Blue = output from 9V AC adapter Yellow = input to voltage regulator when unit is drawing 7.7mA  @ 3.3V, sudden dip is caused by RFM12B firing up to transmit four integer data packets (approx 24mA for 2.7ms)
If all goes well we're expecting to get the emonTx V3 into production in the next few of months.


emonGLCD 433 / 868 Mhz RF Scanner & Signal Strength Meter

Martin Roberts has developed a fantastic bit of firmware for the emonGLCD to enable it to be used as an RF scanner and signal strength meter on the 433 / 868 Mhz RF band.

 This makes the emonGLCD running this firmware a useful tool for debugging RF transmission and checking out signal strength. To run the firmware a small hardware modification is required to the emonGLCD to access the analogue signal strength output signal from the RFM12B.

Martin's sketch has been added to the emonGLCD github repo.

Full details including build-guide and discussion can be found on the original forum thread.






Notify on inactive feeds

Here's a simple feature that sends an email whenever emoncms feeds become inactive for more than two hours, designed to stop those instances where you login to emoncms after a month and realise that something went wrong and it hasn't been updating for a while.

The notify control page can be accessed via the Extras menu. It can be disabled or enabled and an email address that is different from you main account email can be used if needed.

A cron job runs every 2 hours on the server to check if any feeds have become inactive for more than 2 hours but less than 4 hours. This ensures only one email is sent and historical inactivity is ignored.

Notify is now running on emoncms.org, you may want to check that your email address is correct before turning it on.

To install notify on your raspberrypi or own server see installation details here:
http://github.com/emoncms/notify



PCB Manufacturer Meeting

A couple of months ago I met up with Stacey Driver who is the Business Development Manager of PCB manufacturer Stickleback Manufacturing Ltd. We have been working with Stickleback for several years now, over that time they have grown in size and moved to a new factory. It's great to support (in the small way we can) a UK based manufacturer.

Here's a copy of Stickleback's recent newsletter where our meeting is mentioned alongside details of their new water treatment plant they have recently installed. It's great to hear that they are continuing to invest in their manufacturing processes and reducing their environmental impact. This links in well with my previous blog post on Ethical and Sustainable Electronics 


Stickleback Chat - Official news of Stickleback PCBs

Image

Water Recycling Treatment Plant

As part of our continuing energy efficiency scheme we are thrilled to announce that we have installed a water recycling plant for our factory in Rochester, Kent. The introduction of the unit will significantly reduce the volume of waste water produced as a direct result of our processing.
Acids and alkalines used in our manufacturing processes are neutralised under controlled conditions. The ph. of the solution is carefully controlled allowing the water to be reused. This reduces the volume of water drawn by our company and removes the possibility of any polluting effects.
More and more companies are investing in waste water treatment. For Stickleback, we see it as part of our corporate responsibility to preserve and protect the natural resources we have wherever possible. We can’t wait until it is all put together and up and running.
Image

Open Energy Monitor and Google Campus

Stacey Driver had a fantastic meeting with our long-standing, valued customer, Glyn Hudson, at the Google Campus.
Glyn is a highly ambitious and motivated young entrepreneur, who is proud to be manufacturing a British product. The Open source hardware product, manufactured by Open Energy monitoring, allows us to relate our use of energy to our energy systems and embrace the challenge of sustainable energy.
We’ve worked with Glyn from the prototype stage through to production and his quantities continue to grow as his products are sent worldwide.
We are proud to support growing business’; start-ups and SME’s and have developed smart prototyping systems to help minimise the required outlay at these costly development stages.
Contact Stacey today on [email protected] or 01634 272416 if you think we could become part of your businesses fantastic journey.
We would be delighted to discuss where Stickleback may be able to help!



Ethical and Sustainable Electronics

Like many of us I love technology and enjoy using the latest gadgets, but this often sits uncomfortably on my conscience since I also care deeply about the environment and try and do my best to reduce my energy consumption and carbon footprint where possible.

The choices we make as consumers do make a difference, the stuff we buy accounts for about 40% of our current energy consumption, this is for the average person, it's probably a fair bit higher for 'techies'! The energy used to make the stuff we buy is more than the energy we use for personal transport and more than our energy demands for heating and cooling [1 – P.94]. Hi-tech electronic items have a large embodied energy carbon footprint and often contain 'conflict materials' such as gold (contacts) , tantalum (capacitors), tungsten etc. Manufacturing a PC requires 11 times it's own weight in raw materials to produce compared to 1-2 times for a car [1 - P.94]

Fair Trade Graphic from Nager IT

In many areas of purchasing there is an ethical and lower carbon choice available, like buying local vegetables and fair trade coffee and chocolate etc. However when purchasing electronic items I have always assumed there was not this option, that no product is more ethical or lower carbon than another (they are all just as bad!).

I would like to highlight two projects that I have discovered lately that are working together to try and improve the state of electronics manufacture. I should state that I have no connection with these projects other than reading their websites.

The first is a German project called Nager-IT who have produced a "Faire Maus" or Fair Mouse.

They have designed the USB mouse using low carbon materials and have gone to great lengths to source ethically produced components and have pushed for transparency throughout the supply chain (something which is almost non existent in electronic production at the moment). Checkout this graphic they have produced illustrating their transparent supply chain:



 The mouse they have produced can be purchased through their website.

The second project is called FairPhone, they are a social enterprise in the Netherlands with the aim of " designing, creating and producing our first smartphone and taking the next crucial step in uncovering the story behind the sourcing, production, distribution and recycling of electronics." They are developing and aiming to start stating in Autumn this year a 'fairer' android powered smartphone.

Graphic from http://www.fairphone.com website

It's uplifting to hear of small independent companies taking on the big players in the consumer electronics industry by offering a 'fairer' alternative. We as consumers should do our best to show that there is demand for more ethically produced electronic projects.

I have also discovered that GreenPeace produce an independent ranking of the large electronic companies based on how 'green' they are.

Graphic from GreenPeace


[1] – Sustainable Energy Without the Hot Air – David MacKay [2009]

How this relates to OpenEnergyMonitor 

Steps we have taken so far

  • Choose RHOS and components which are free of conflict materials
  • Choose low embodied energy materials for enclosures which can be easily recycled
  • Use standard parts which can be easily re programmed and used for other purposes in the future
  • Use recyclable packaging (paper bags which are naturally antistatic) for our shipping 
  • Choose low power equipment, LED lighting, recycled paper for our office and switch things off at the end of the day! 
  • Choose surface freight (boat) rather than planes to ship stock (mainly power adapters) from China, this takes 3 months rather than 3 days but worth it for the 32 times less energy consumed [1 - P.92]. 
  • Choose a PCB manufacture who are based in the UK who uses lead free techniques, complies to the highest environmental industry standard and is actively investing in techniques and equipment to reduce wastage and environmental impact (e.g water treatment and recycling) 
  • Strive to optimise electrical consumption in our hardware to be as low was possible 

Steps we would like to take in the future 

  • Produce an end-to-end supply chain map of our manufacture (possibly using http://sourcemap.com)
  • Try and calculate the embodied energy that goes into producing our designs and constantly try and find ways we can reduce this
  • Sell through local distributors to reduce air based shipping and investigate possibility of local manufacture
  • Use recycled plastic / other low embodied energy martial in the moulding of the housing for our next version of the emonGLCD and temperature monitoring node 
  • Investigate how we can build recyclability into the design, checkout the video below from the Great Recovery Project:


Preparing emoncms.org feeds for conversion to timestore

One of the central principles behind the Timestore data storage approach is that datapoints are stored at a fixed interval. Because you know that there is a datapoint every 10s you dont need to store the timestamp for each datapoint, you only need to store the timestamp for the first datapoint. The timestamp for every other datapoint can be worked out i.e:

timestamp = start + position * interval.

Storing time series data in this way makes it really easy and very fast to query. The tests so far have shown timestore to be several magnitudes faster while also using significantly less disk space.

If your logging to emoncms.org there is now an interface that can be used to select the interval rate that you would like your feeds converted too, its linked from the feeds page.

On the feed/convert page:

The new interval column states the average interval rate of the existing feed and is calculated simply as the end time minus the start time divided by the number of datapoints in the feed. This interval rate can be skewed if the monitor dropped off for a period, so its worth double checking that it is correct.

You may wish to change your interval rate, if your logging temperature data at 5s intervals and 60s is enough to see the changes in temperature you want to see then select 60s as this reduces the disk use of the feed considerably.

To set the interval rate you wish your feed to be converted to, click on the pencil button to bring up in-line editing:

Click on the drop down menu under convert to and select from the list the interval you wish to use. Click on the tick button to complete.

Repeat the above steps for every feed you wish to convert and then once your done click on Add feeds to conversion queue button that is at the bottom of the conversion page.

As mentioned in the last post, its going to take up to a month for the server to convert through all the feeds so you wont see anything straight away. I will update with conversion progress.

I've put all the code for the above on github under an emoncmsorg branch of emoncms, the conversion scripts can also be found in the repo usefulscripts

From 1.3 minutes to 196ms: timestore on emoncms.org


John Cantor who uses the openenergymonitor system for heatpump performance monitoring got in contact the other day saying that graphs on emoncms.org where taking a really long time to load. I said that the work on using timestore will soon speed things up significantly. He asked if there was a chance to convert the feeds before he went up to work on a system in Scotland as being able to zoom with speed through the graphs would make a big difference.

Clicking on the Y (Year) button in the rawdata visualisation has slowed down so much over the last year that it now takes over a minute to load the view.


The same request on a raspberrypi takes 2.9 seconds, The thinking to why its so much slower on emoncms.org is because of the use of the php loop doing individual requests to mysql which are added to the mysql process queue and have to wait in line amongst all the other mysql inserts and updates caused by all the updating feeds, the php loop method was initially faster than an equivalent mysql implementation but clearly not for long.

Anyway I figured we could get timestore up and running on emoncms.org straight away if we run timestore in parallel with the current mysql storage and then slowly convert feeds over to timestore over time. The present input processing and visualisation implementation would be kept as is for the time being only initially using timestore to replace the realtime datatype rather than both realtime and daily.

Conversion from mysql to timestore takes a long time especially as the process has to be throttled down in order to allow enough resources on the server for normal operation to continue. For example the conversion of a 568 day long, 5s datapoint interval feed (~152mb) takes about half an hour loading the server to an average 15min (quite high) load of 4.69 at that rate it will take about a month to convert all feeds over to timestore.

The results though are worth it!:


196ms!

1.3 minutes down to 196ms is a 398x speed increase! and its also possible to see another advantage of timestore the effect of the average layers showing a slight winter summer curve. A big thankyou and credit to Mike Stirling for developing timestore enabling this kind of performance, if you haven't already check out the timestore project website:


John Cantor has shared the Scottish heatpump dashboard publicly, take a look to try out the new timestore feeds:


Im currently working on an interface in emoncms to allow selection of the feed datapoint interval (5s, 10s, 30s etc) to convert to before the feeds are added to the conversion queue, more on this soon.

OEM Gateway by Jerome Lafréchoux

I would like to highlight the good and interesting work that Jerome Lafréchoux has been doing on developing a gateway that is intended to go beyond the current RFM2Pi interface towards something that can be used on any linux system whether a pi or any computer to take say input from a serial RFM12Pi and forward it either to local instances of emoncms or remote instances of any service (without necessarily needing a local install of emoncms).

Here's an exerpt from Jerome's last post on the github topic on gateway development:
"The standalone gateway seems complete enough for me to be published, although I'd rather have a little feedback before recommending it.
It works basically the same way. Here are the main features:
  • Can be parameterized either via emoncms GUI (with current limitations of the interface) or through a configuration file. Useful for those who don't want to install a local emoncms just for the configuration of their gateway. This also allows to add features without the need for complex GUI edition.
  • Can read inputs from the RFM2Pi, but also inputs from the serial port of the form "NodeID val1 val2 ...", or even from a socket. This socket possibility means any application on the same machine or on the network can send data to the gateway. This even solves the inter-applications issue. It is better than writing to a file, in my opinion.
  • Extensibility. By design, it should be easy to create new inputs (data sources) or outputs (another server's API). It uses classes, so a new input (listener) or output (buffer) can be created through inheritance/overriding.
See readme file for more explanations.
I should add a socket use sample, but it's pretty basic and require very few code. Here's a python example:
import socket
HOST = 'raspberrypi' # The remote host, can be 'localhost' just the same
PORT = 50011 # The port chosen in the gateway config
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.send('14 15 26\r\n')
s.close() 
Currently, each buffer instantiated in the gateway buffers a given amount of data when network is down. This is limited (we don't want to eat up all memory). Next step could be to add the possibility to buffer samples in a file (with care for SD card wearout)."

To try it out see the github repository here: https://github.com/Jerome-github/oem_gateway. Jerome is looking for testers and would appreciate feedback on it.

EmonTx Shield SMT progress

The latest emontx shield SMT board revision (v2.4) arrived a couple of days ago.

This revision carries on from the developments described in the previous blog post here: http://openenergymonitor.blogspot.co.uk/2013/06/emontx-shield-smt-with-new-layout-in.html

The improvements in v2.4 are:
  • The DS18B20 connector is bought out to the edge with screw terminals.
  • There's an FTDI connector as the shield covers the FTDI connector on the NanodeRF.
  • The jumpers are 2.54mm headers rather than surface pads that are soldered, making them easier to change.
  • The RFM12 footprint is moved further in on the board so that there is space for the Ciseco SRF with its on board chip antenna


There was one small but crucial mistake, I got the RX and TX on the FTDI connector the wrong way around!!

Next step: switch the RX and TX around and send the board off for production.

DIY Casing

We are working on new designs which which will be better designed to fit into cases in the meantime John Cantor  (from heatpumps.co.uk) who has been using a modified OpenEnergyMonitor system for monitoring the performance of air and ground source heatpumps shares a photo of his experiences of boxing up the emonTx and Nanode RF SMT

emonTx setup for temperature monitoring and nanode RF SMT cased up using standard RS cases

These cases from RS were used:

ABS moulded box, 120x65x40mm, ivory
RS Stock No.502-641
ABS moulded box, 112x62x31mm, ivory
RS Stock No.502-635
both £3.45+vat.