Simply Dashing!

I’ve finally gotten around to making a web frontend for my Raspberry Pi & Arduino wireless sensor network.

I chose Shopify’s Dashing dashboard, as its opensource and can be installed locally and not via some cloud crap.

Essentially you have a HTML template into which you feed JSON data. I got a bit tied up in the JSON formatting so it took longer than it should have to get working (about 3 hours) but once I realised that most of the job placeholders were a single line of JSON, and not a whole entity, it was quite straightforward. Here’s 90% of my entire dashboard code:

SCHEDULER.every '30m', :first_in => 0 do
    require 'sqlite3'
    db = SQLite3::Database.open "/var/tmp/weather.db"
    
    study = db.execute "SELECT reading FROM weather WHERE node=2 ORDER BY date DESC LIMIT 3"
    send_event('study_temp', { current: study[0][0].round(2), last: study[2][0].round(2) })

    shed = db.execute "SELECT reading FROM weather WHERE node=1 AND type=2 ORDER BY date DESC LIMIT 3"
    send_event('shed_temp', { current: shed[0][0].round(2), last: shed[2][0].round(2) })

    outside = db.execute "SELECT reading FROM weather WHERE node=1 AND type=1 ORDER BY date DESC LIMIT 3"
    send_event('outside_temp', { current: outside[0][0].round(2), last: outside[2][0].round(2) })

    lounge = db.execute "SELECT reading FROM weather WHERE node=3 ORDER BY date DESC LIMIT 3"
    send_event('lounge_temp', { current: lounge[0][0].round(2), last: lounge[2][0].round(2) })
     
    light = db.execute "SELECT reading FROM weather WHERE node=4 ORDER BY date DESC LIMIT 1"    
    send_event('lightsensor', { value: light[0][0].round(1) })

    bedroom = db.execute "SELECT reading FROM weather WHERE node=0 ORDER BY date DESC LIMIT 3"
    send_event('bedroom_temp', { current: bedroom[0][0].round(2), last: bedroom[2][0].round(2) })

    db.close
end

Here’s a screenshot of the finished dashboard, including an additional widget that shows the RPi-1B’s very stretched RAM:

It installed a lot easier on Debian Sid than Raspbian Wheezy, but I hear Raspbian Jessie is easier due to updated bundler/exec_json/nodejs packages – every time I use Ruby I seem to go through dependency hell, Ruby seems to have a lot of issues with backwards/forwards compatibility with Gems – why things like bundler/rvm are necessary I guess. Makes Java look portable.

RF24Network

I’ve been playing with the rewritten RF24 and RF24Network librariy forks and am impressed. The RF24Network library is a much more reliable way of using multiple nodes than just using the same channel/pipe and hoping you won’t get timing conflicts with RF24 e.g. I’d find the data from 2 nodes would merge! Also hardware SPI finally works on the Raspberry Pi.

I’ve updated my weather station scripts, I now have the RPi as the master receiving node0 (with a new 17dBm antenna!); the MOSFET-based Arduino as node1 in the shed, but with the BMP085 pressure/altitude disabled as it was returning absolute rubbish, I think its a hardware issue and not the Adafruit lib; and the older non-MOSFET Arduino as node2 in my office, just reporting its DS18B20 readings in an attempt to prolong battery life – I’ve desoldered the DHT11/BMP085 to see if that helps.

I’ve breadboarded up a node3 that’s powered by a CR2032 coincell and just has the ATmega328P, chip-antenna nRF24L01+, TO92-packaged DS18B20 and two resistors, just to see how the battery life goes. I might even make a 5v board that can be mains powered – or a USB-powered SFE Pro Micro….

My new scripts are here and a graph of the results showing both sensor boards is below:

Weather Station Mk2

I’ve redesigned my Arduino weather station to incorporate a P-channel MOSFET to switch power to the peripherals on and off to hopefully save battery life. The ATmega328P always gets 3.3v, but then uses Sleepy() to conserve power anyway, but the DHT11, BMP085, DS18B20 and nRF24L01+ get powered off when not in use. Oddly enough the voltage seems to drop to 0.7v rather than 0v completely, but I guess no current flows.

I had to change my code a little to re-initialise the radio once per loop and control the MOSFET from a digital pin: download.

I tidied up the veroboard a bit – pink wires are 3.3v constant for the ATmega, red wires are controlled by the MOSFET, black is GND, blue is data:

I hard a hard time finding a through-hole MOSFET than worked at TTL voltage (the FQP7P06 I first tried only worked properly at CMOS 5v) but I eventually settled on the NDB6020P which can be switched using as little as 2.5v.

I’m currently calibrating the DHT11 and keeping an eye on how long the battery lasts, so for the moment the weather station is in the house rather than the shed, here’s some results (its also tweeting using Python/C++ on the RaspberryPi, I didn’t need to update them):

Update: it seems that turning the peripherals on and off is making the readings from them more reliable, as the BMP085 and DS18B20 readings are almost identical and the DHT11 is within one degree – given that it uses integers this is as near as it will get.

Update 2: the weather station has been running on 2xAA’s for over 12 days now, without the MOSFET it would last under 8 days.

Update 3: the board eventually stopped after 30 days, although since then I’ve been having problems with it stopping even though the battery is not dead – perhaps a range problem…. Anyway I’ve added the boards.txt definition I used below:

atmega328bb.name=ATmega328 on a breadboard (3.3v, 8 MHz internal clock)
atmega328bb.upload.maximum_size=30720
atmega328bb.upload.protocol=usbasp
atmega328bb.upload.using=usbasp
atmega328bb.bootloader.low_fuses=0xE2
atmega328bb.bootloader.high_fuses=0xD8
atmega328bb.bootloader.extended_fuses=0x07
atmega328bb.bootloader.path=atmega
atmega328bb.bootloader.file=ATmegaBOOT_168_atmega328_pro_8MHz.hex
atmega328bb.bootloader.unlock_bits=0x3F
atmega328bb.bootloader.lock_bits=0x0F
atmega328bb.build.mcu=atmega328p
atmega328bb.build.f_cpu=8000000L
atmega328bb.build.core=arduino
atmega328bb.build.variant=standard

Weather Data Logging With SQLite

My weather station is still going strong after three days on the Alkaline batteries. Now I’ve got quite a bit of data, I thought I’d better do something more useful that just tweet it.

The problem was I was only storing the current hour’s weather data, so I had to import my data back from Twitter! Now whilst you can get a CSV archive of your tweets, you can only get it about once a month which is stupid. So I ended up using some grep/sed tricks on the web page to get my data, and then wrote an importer to remove the prettifying and get it back into the original data format. That meant that I had a starting point for my database so I didn’t waste three days’ worth of data. I also found that my timestamps weren’t being stored as datetime objects, so had to fix that too.

So now my Python application tweets and updates the SQLite3 database I made from the existing data, every hour from a cronjob. I also added some exception handling for duplicate db-records/tweets and finally made a pretty SVG graph of the data using PyGal:

As is to be expected, the DS18B20 sensor is the most accurate, with the DHT11 being the least, but there’s not a huge variation. Oh and the Alkalines seem to have finally given up after about 62 hours.

Update: I’ve switched to using matplotlib for rendering graphs, as it can output PNG/PDF instead of PyGal’s SVG’s which don’t render well in some viewers and require an internet connection:

The updated code is available here. Oh and the 2900mAh NiMH batteries have lasted for about 12 hours so far…..

Update 2: It got cold last night and I realised that I’d used unsigned ints so couldn’t cope with negative values, possibly that’s also the explanation for some odd altitude readings. All fixed up now (same link above) and the batteries have lasted 84 hours or so now.

Update 3: the 2900mAh batteries last exactly 8 days, I’ve swapped them with the Eneloops now to see if they can beat that.

Saving Battery Life

I found that my weather station only lasted for about 7 hours on 2xAA batteries last night. The culprit could be the boost regulator, but before I investigate that I’ve reworked my Arduino code and moved away from manually tinkering with the avr/power.h stuff to using JeeLib. Its greatly simplified my code too. I removed the code that was measuring the voltage as that always returned 3.37v from the boost, so a bit pointless, plus it kept the ADC awake draining power.

The reduction in packet size meant I could move the dewpoint calculation back to the Arduino side, so all my data was being transmitted to and stored on the Pi via the C++ rather than it being calculated in the Python code temporarily before being Tweeted.

I’ve also made the Arduino and radio sleep for 30mins to reduce power consumption, then it wakes and transmits for about 5secs and then goes back to sleep. Plus the Pi process sleeps for 10mins to reduce the number of disk writes it makes (and CPU time) without missing the transmissions. So hopefully I’ll achieve the months/years of battery life that I keep hearing about.

All the sourcecode including the Pi stuff is here.

Update: I take it back, the culprit is shitty NiMH batteries. I’ve been running for 22 hours on a pair of Alkalines and still have 2.79v coming out of them! So I’m discharging and fully recharging a pair of 2900mAh NiMH’s to see how they go. I’ve also ordered some brand new 1900mAh Sanyo Eneloop 3rd generation NiMH batteries that apparently store their charge very well.