Faulty Batch of MCP1702 Voltage Regulators

We have noticed recently that a number of people have been having problems with the Nanode RF. The problems were brought to our attention through this thread on the forums.

Many people were getting the message "Failed to access Ethernet controller" when running an EtherCard example.

After much testing we have identified the problem. It turns out that the MCP1702 3.3V voltage regulator which supplies 3.3V to the Ethernet Chip (ENC28J60) was not supplying enough current to power the chip. The voltage was dropping down to 2.6V under load.

Swapping the voltage regulator for one from a different batch solved the problem. it looks like we have been sent a faulty batch of voltage regulators, our apologies for not spotting this sooner.

Note: this problem only applies to the MCP1702 voltage regulators in the latest batch of Nanode RF kits. All emonTx and emonGLCD kits are un-effected, they have always used regulators from another source.

If you have this problem we will send out two replacement regulators free of charge, one for the Ethernet chip and the other for powering the Atmega 328, it may be ok to leave the Atmega328 voltage regulator in place as the current requirements are not as high and it seems to work fine here without swapping it out, but we will send two out so that you can swap out as you prefer.


We have created a free voltage regulator entry in the online shop, just quote your previous OpenEnergyMonitor order ID when ordering and we will send them out: http://shop.openenergymonitor.com/nanoderf-mcp1702-voltage-regulator-replacement/.

All the Nanode RF's kits sold through the shop from today onwards will contain the non-faulty voltage regulators.

De-soldering tips
The easiest way to de-solder the regulator is to if possible cut two of the regulator legs, you can then pull out the third by pulling on the regulator body itself while holding the soldering iron tip on the solder surrounding the leg. Then use pliers to pull out the other legs one by one. It is usually easier to pull the legs out if there is actually plenty of (molten) solder around the leg. To clear the holes of solder with a solder sucker again make sure there is plenty of solder in the holes to start with, this usually leaves the cleanest result in our experience. An extra pair of hands can help make the whole job easier. 

elektro:camp(«2012.05») Workshop | Offenburg-Germany

Hackathons are great! The next one I am attending will be the electro:camp in Offenburg, in Germany, close to the french border. I think Trystan and Glyn will also be there as well as a number of cool hackers and makers, all interested in energy and automation.

Many great developments will result from this event and hope to see some of you there as well.

Here is the official website: www.elektro-camp.de




Speeding up emoncms feed data requests


If you log 5 second data you quickly amass a huge number of datapoints (about 6 million a year). If you then tried to visualise a years data by loading all 6 million datapoints you would be waiting a long time. The most we would want to load is the same number of datapoints as the pixel width of the visualisation, lets say around 1000 datapoints. So we need some way of picking out of our 6 million datapoints 1000 datapoints at equal interval.

When I first wrote emoncms I searched for mysql queries that could pick out table rows at given intervals and came across the following query that does this:

SELECT * FROM (SELECT @row := @row +1 AS rownum, time,data FROM ( SELECT @row :=0) r, $feedname) ranked WHERE (rownum % $resolution = 1) AND (time>'$start' AND time<'$end') order by time Desc

It seem to work really well. Fast forward 5 months my feed tables are approaching 2.5 million rows  and I'm starting to notice query times getting really quite long. After a bit of searching again, I came across a suggestion to a similar problem suggesting the use of an index. So I tried adding an index and creating a php for-loop to request a single row at given intervals:

$range = $end - $start;
$interval = $range / 1000;

for ($i=0; $i<1000; $i++)
{
  $qtime = $start + $i * $interval;
  $result = db_query("SELECT * FROM $feedname WHERE `time` >$qtime LIMIT 1");
  if($result){
    $row = db_fetch_array($result);
    $data[] = array($row['time'] * 1000, $row['data']);    
  }
}

The following table shows the typical times for requesting data from both the indexed and non indexed tables and the different methods. At the same time I also tested whether request times where affected by the data type that time is stored as: mysql datetime or a unix timestamp stored as an unsigned int.



So it looks like the optimum configuration is primarily the addition of an index and use of the php data request method and then to reduce disk space use, switching to unix timestamp. The next step is to create a script to automate table conversion from datetime unindexed to timestamp indexed.




emonGLCD V1.3 Switches - fix!

It was mentioned in this blog post regarding the current version of the emonGLCD (V1.3) that a hardware error rendered the push switches un-usable. This is not the case any more!



It turns out that this error can be fixed in software. The switches can be made to work with no hardware modifications. The emonGLCD tester sketch has been updated with the required fix.



Here are the details:

The push switches are connected ADC 1,2 and 5 (digital 15,16 and 19). When pressed they are pulled low through resistor R5.
The Atmega328 has internal pull-up resistors (20K) which can be enabled by setting the port as output and doing a high digital write:


pinMode(switchpin, INPUT);


digitalWrite(switchpin, HIGH);

The switches are active low, meaning a logic 0 will be read on digital 15,16 and 19 when the relevant switch is pressed. 

Thanks to forum member Drsdre for bringing this elegant fix to our attention. This is a real open-hardware success story! If you've bought an emonGLCD from us we will send out on request the three switches to be added free of charge.

We have created an item for the extra switches in the shop, just click buy (it's free) and quote your original order number: http://shop.openenergymonitor.com/displays/

All emonGLCD kits bought from the shop from now on will include the switches.

Happy days.

Open source hardware and the free rider dilemna

I wrote a post on my personal blog about using content from open source projects in general and the Open Energy monitor in particular. Here is an extract of it:


There are now more than 960 registered members in the community, not all of them are active and quite a few contribute actively to the development of the project. But does that mean that the others are just passive members? free riders? is that fair?
I started toying with the idea of using an arduino to monitor electricity consumption about two years ago, and I found out that someone else (Trystan and Glyn and quite a few others) already started developing that idea and sharing their work and findings online. Sharing their work and code online does not only save other people (actually humanity in general) a lot of time and effort, but it also allows for a much wider scope of fellow tinkerers to review and improve that work.
You can read the post on my blog.

Heat Pump Monitoring



Over the past few months we have been working with John Cantor to develop, build and test the potential of a heat pump monitoring system. John has a lifetimes experience of installing, optimising and troubleshooting all types of heat pumps. John runs heatpumps.co.uk and has recently written a very informative book Heat Pumps for the Home

The rest of this post will be a guest post by John, over to you:

A heat pump works by transferring heat form one place to another. The electrical input power should be only a fraction (maybe 1/3rd or 1/4) of the available heat delivered. However, the energy efficiency can vary dramatically.  One of the biggest factors that affect this is the working temperatures on the cold (source) and the hot (sink) sides of the system.  This is mainly dictated of system design (radiators/underfloor-heating details etc), but it is also dependent on the way that the system is operated, see www.heatpumps.co.uk

It has become evident that a significant number of the UK’s heat pumps are not reaching their energy-efficiency potential.  In ordered for us to learn how best to install, set-up and operate heat pump systems, some form of real-time monitoring is vital.

We have developed a heatpump-specific variant of the OpenEnegyMonitor wireless web-connected emonBase and emonTx system. Emoncms was used as the web logging and graphing application.

This tool is as valuable as the stethoscope and X ray is to a doctor.  It allows a heating system to be analysed in detail and observed over time so that settings can be modified, use-patterns changed, and system modifications be made.



Our first test was on a heat pump in Scotland.  The owner initially had some very high electricity bills. After a brief spell of monitoring, we were able to pinpoint a few issues.
These related to the electric boost heater coming on unnecessarily and the under-floor heating poorly balanced causing frequent compressor ‘cycling’ (stop/start).  However, this installation was generally a good example. The data can be seen here: http://vis.openenergymonitor.org/pages/heatpump.php

Our second test was on a suspect system in Wales. This showed up some alarming temperatures on the source, and it is apparent that the refrigeration circuit is in poor health.  We plan to repair the system, then compare the graphs before and after.

The graphs below show just one aspect of the analysis.
 

There are many things that can be viewed using the web-dashboard. This includes the refrigeration circuit, the source, the underfloor heating and the compressor power-factor.  

It seems likely that there would be 2 main uses for this monitoring.
1)      For the installer or technically-minded owner to learn how best to operate their system, or to pinpoint any areas for attention and possible modification.
2)      For the owner to see some basic data relating to running cost etc. It would also be used as a remote monitor for holiday accommodation.

The diagram at the top shows the full version, but we are working on a simpler version that focuses on the user-control aspect of a system.


OpenEnergyMonitor Shop is open!

It's taken a bit longer than we expected but finally we can proudly announce that our on-line shop is open for business!

Last week we 'soft launched' the shop to friends, members of the community and our twitter followers. We would like to say a big thank you to everyone who's supported the project by making a purchase through the shop. We have been a little overwhelmed by the level of interest. Again, thank you, you guys are amazing!

In the shop we sell all the components needed to build an Arduino based wireless, web-connected open-source energy monitoring system.

shop.openenergymonitor.com


Component kits for the main OpenEnergyMonitor modules are available in the shop:

emonTx - wireless energy monitoring transmitter unit

emonGLCD - wireless graphical LCD display unit with on-board LED's & sensors

emonBase - NanodeRF wireless, web-connected base station
We also sell CT clip on current sensors, optical utility meter pulse detector sensors, temperature sensor and power supplies.

This system is still in development, it's not an end user consumer product. It is however functional and very flexible in it's current form. Being Arduino based and using standard Arduino software libraries, it's easy to customise the system to meet your needs.

One motivation behind opening the shop is that by selling the units in development kit form it removes barriers to entry and empowers members of the OpenEnergyMonitor community to to get involved with the project. Revenue raised through the shop will go directly towards funding more development and sustaining ourselves so we can continue working on the project.

In the near future we are planning to move to surface mount electronics to enable pre-assembled units. See our newly updated vision page for more information on how we see the project progressing in the future.

emonGLCD V1.3 PCB

Introducing the latest version of the emonGLCD PCB.

This latest version (V1.3) fixes a few minor issues with the previous version (V1.2) but also has some issues of its own. It's a case of two steps forward and one step back.

The good news:

  • The LDR light sensor no longer suffers from reduced sensitivity due to sharing a port with one of the LCD pins 
  • Both tri-colour LEDs are on ATmega328's hardware PWM ports allowing the brightness to be controlled with some nice dimming effects 
  • New new rear silkscreen design featuring a handy RFM12B frequency note area and a large Welsh dragon graphic! 
The bad news:

  • The new feature in V1.3 was suppose to be two extra push switches to allow for future screen navigation. Unfortunately due to a PCB layout error non of the switches on V1.3 are functional. While this is frustrating since it was an easily avoidable mistake the display is still very functional. The current firmware example for single CT energy monitoring and solar PV monitoring don't really make much use of the switch. 
The good news!

Everything else LCD, wireless, ATmega 328, Light sensor, Temperature sensor and multi-colour LED's work great. We have decided to go with the switches not working for now as it still makes a very functional energy monitor display. 


emonGLCD V1.3 front
emonGLCD V1.3 rear 
V1.3 assembled
The schematic and board design for emonGLCD V1.3 are up on solderpad: http://solderpad.com/openenergymon/emonglcd/

The emonGLCD documentation page has been updated with the new port mappings: http://openenergymonitor.org/emon/emonglcd

Documentation reorganisation proposal

The current front page


It has been pointed out several times that the documentation on this site is hard to follow. We've been giving this some thought and have come up with a plan for hopefully what will be a clearer structure.

Here are our initial thoughts:

1) We start with the following headings, these form the main menu:

Community | Applications | Modules | Fundamentals | Tools | Archive

These headings contain:

Community
  • Forums
  • Development plans
  • Links to community builds and blogs
Applications
Applications are step by step guides of how to assemble modules to create a particular system. They start with a short description then a step-by-step type structure. Detailed application discussion would be on a second page or blog post with link through.
  • Power monitoring
  • Solar PV
  • Solar hot water
  • Pulse counting
  • Heatpump monitoring
Applications reference through to modules below.

Separate github folder for each application containing sketches for all the modules that make up the application.

Modules
  • Hardware
    • emontx
    • emonglcd
    • emonbase
    • See-a-watt
  • Software
    • emoncms
    • Pachube
Each module contains the pages:
  • page1: overview + features (maybe use it: application list is referenced here?)
  • page2: built it / install it / buy it
  • page3: detailed design documentation
All emon system sections reference through to fundamentals section below

Fundamentals

Getting started with arduino, uploading sketches etc.

Electricity monitoring
  • AC Theory - Introduction
  • AC Theory - Advanced
  • AC Theory - 3 phase
  • Power in different countries
  • Non-invasive
    • CT sensors - Introduction
    • CT sensors - Advanced
    • Current only - Single Phase
    • Current and voltage - Single Phase
    • 3-phase Monitoring
  • Invasive
    • AC shunt
    • DC Shunt & hall effect sensing
  • Appliance inference - labs
  • Interfacing with inverters
Pulse counting
  • Single pulse counting
  • 12 input pulse counting
Temperature
  • DS18B20 temperature sensing
  • PT1000 temperature sensing
Networking
  • RF
  • RFM12B
  • xbee, zigbee, old rf
  • Master/slave simple serial networking
  • Ethernet
Displays
  • GLCD
  • 7segment, 3310
Storage
  • USB Pen drive data-logging
  • SD Card logging
Control
  • Relay
Firmware
  • Watchdog timer
  • Structures for transmitting data.
  • Sleep mode
  • String constructor
Github sketches for the fundamentals section could be contained in one github folder called fundamentals.

Tools
Tools that can be useful in debugging / development of systems:
  • VI Sampler
  • KST based graphing and statistics
  • Programming nanode with arduino
Archive
Items for archive

Other changes
We've disabled comments on pages, the thinking here is that its getting quite hard to navigate the page comments as conversations at different dates and topics are all getting mixed together and that it would be better to have these discussions on the forum, where each forum thread has its own clear topic.

Every page can be put in a hierarchical order such as modules/hardware/emontx making navigation clearer.

A site map with all of the above.

Do you think this is a good idea? 
Any suggestions on how the website content can be better arranged would be greatly welcomed!
Thankyou!

New emoncms features part 2 - Email notifier

Its good to know as soon as possible if there is an issue with an installed energy monitor, such as the base station dropping offline or a problem with RF connection. This is a rough and ready first implementation of an email notifier for emoncms, that sends you an email to say that a feed has become inactive or a feed is equal to a certain value. No longer are the days when you log in to emoncms just to find that there are two days of missing data as happened to me on feb 6-7 here

A walkthrough

1) Click on a feed in the feed list to bring up the feed view page, scroll down to the bottom and click on set notifications:


2) Select the events that you want notifications for. Notification on value and on inactive both work. Periodic notification is yet to be implemented.




















3) Click on edit mail settings and enter in the recipient email address. This only has to be done once as it is the same for all notifications.














If you want to send the notifications to multiple addresses, just put in a comma between each address.

4) Unplug your emonbase!! and check your email:














Behind the scenes


Main scripts:

notify_model.php
mail_model.php
notify_controller.php
notify_view.php



Installation/Setup
1) To use the notifier on your own server you will need php mail installed, to do this on ubuntu enter the following into terminal

sudo apt-get install php-pear
sudo pear install mail
sudo pear install Net_SMTP
sudo pear install Auth_SASL
sudo pear install mail_mime



2) In emoncms3/Includes copy the default.mail_settings.php file and rename too mail_settings.php and enter in your mail server settings.

3) Open the file cron.php and change the key password to something unique, this is used to ensure that only you can run the cron.php script.

4) Create a cronjob, in ubuntu start by opening crontab: $ crontab -e
Then add a cronjob that calls the cron.php script including the key that you created. To call the cron.php script once a minute try the following:


# m h  dom mon dow   command
* * * * * wget -q -O /dev/null "http://localhost/emoncms3/cron.php?key=YOURKEY


5) If your using the same database as a previous emoncms version, you will need to create the new notify database tables. This can be done in index.php by pasting require "Includes/setup.php"; in the top. Its probably worth removing it afterwards.

6) Configure your feeds as in the screens above!