Wednesday, November 25, 2015

RFD900+ now supported by flash-rfd900 utility

We have a bunch of the newer RFD900+ radios, which have 1 - 2 dB better receive sensitivity than the older RFD900/900a (no plus) versions of the radio.

However, we were previously unable to use these in Mesh Extenders, because we couldn't flash them using our little C utility that we wrote for flashing them on Mesh Extenders.

Using the officially supported flash utility was not an attractive option for us for a couple of reasons: (1) it requires all sorts of dependencies, like python, that ended up adding something like 200MB to the Mesh Extender image, which is normally ~50MB; and (2) it wasn't as fast as we would like.

Our flash-rfd900 utility ( worked fine on the old RFD900 / RFD900a radios, but seemed to brick the 900+ radios.  Some very helpful insights from the folks at RFDesign revealed that the new radio uses 24-bit addresses in the bootloader, instead of 16-bit ones.   The intel hex program file can also contain type-04 records in addition to the usual type-00 records that contain the actual data. Type-04 records indicate which 64KB bank is being addressed, and are necessary for the RFD900+ which has an upgraded micro-controller that has 128KB of program flash, and twice the RAM resources of the older radios.

Adding support for these two features was fairly simple, and after that flash-rfd900 is now working for the RFD900+ radios, so we can finish getting the Mesh Extender firmware image to work with that model of radio.

Wednesday, October 21, 2015

Debugging RFD900 Ad-Hoc UHF behaviour

So I am getting closer to having working ad-hoc UHF communications between Mesh Extenders with RFD900 family radios running our custom CSMA firmware (not that for now you need a spectrum license from your local regulator to do this at >3mW in most countries).

Now I am debugging why MeshMS messages are not being delivered in reasonable time. It seems that lbard is not making the most sensible choices in terms of which piece of which bundle to transmit.

However, I need a nice way to see what it thinks it is transmitting, without having to stop lbard and run it in debug mode, which of course loses the accumulated state in lbard, and thus might cause the bug to disappear.

To address this, I am implementing a function to dump the current state of lbard in HTML so that it can be read wirelessly using the web server on the Mesh Extender to serve it up.  This will show the calculated priority of the bundles on each node, as well as their information about neighbours and in-progress transfers from neighbours.  Hopefully with that I will be able to track down the remaining priority bugs quite quickly.

Sunday, September 20, 2015

Shaking the bugs down with ad-hoc communications via RFD900

Last week I was up at Arkaroola again working on getting the Mesh Extenders to communicate via the RFD900 radios running my new CSMA firmware using our new spectrum license that allows us to avoid frequency hopping at (relatively) high power levels.

The CSMA firmware itself had very few problems to be solved.  The main change I made was introduce a couple of escape sequences to allow setting the power level of each packet to either low (requiring no spectrum license) or high (requiring a spectrum license).

This was part of a general effort to make the power level easily controllable so that I could set the Mesh Extenders to either "normal" or "Arkaroola" mode, without having to reflash them.

In the end, I implemented two tests that the Mesh Extenders perform before sending a packet at high power: First, the file "HIPOWER.EN" must exist on the USB memory stick on the FAT32 partition, and second, the slider switch on the MR3020 PCB must be set to the middle position. If either of those are not set correctly, then packets will be transmitted at low power.  Thus I can either replace the contents of the USB memory stick, or slide the switch to disable hi-power mode when I return from Arkaroola.  Ideally I should modify the lid design of the 3D-printed case so that I can access the slider switch without unscrewing it, but that's really just a matter of convenience.

So after fixing that, and having a frustrating day while I realised I was using the wrong binary for the RFD900 radios, I managed to send a MeshMS message between a pair of Mesh Extenders operating in this way.  There are still some scheduling bugs in LBARD that prevented the confirmation reply from getting back to the sending phone. I also have a list of other relatively minor issues to work through to get it all operating stably, all of which I hope to work through this week ready for the next trip to Arkaroola, where I hope to be able to operate 8 or more Mesh Extenders together using the RFD900 in this ad-hoc mode, and to be able to send and receive MeshMS messages over a fairly large area.  This will be very satisfying and a significant step forward if I can achieve it.

Monday, August 31, 2015

Simple CSMA firmware for RFD900 radios

I have been continuing to work on getting multiple Mesh Extenders communicating nicely via the RFD900 radio.  

As I have previously mentioned, the challenge with this is that the frequency hopping firmware that they normally run cannot work with multiple radios without a defined network controller.  This just isn't possible with Mesh Extenders, where we expect nodes to wander in and out of coverage on a regular basis.

Fortunately, it is allowable to use the ISM915 band in Australia, the USA, New Zealand and a few others without frequency hopping.  To do this, you must however use very wide channels, or limit your transmit power to not more than 3mW EIRP.  Fortunately, as described in my last post, the RFD900 can still transmit a long way on a couple of milli-Watts EIRP.  While at Arkaroola, we can of course transmit at upto 1W EIRP under our scientific spectrum license.  In the longer term, the next model of radio from RFDesign will allow the use of very wide channels, so the scientific license is really just an interim solution. Anyway, enough of the spectrum regulation side of things ...

The normal RFD900 firmware's frequency hopper uses a Time Division Multiplexing (TDM) scheme, which is great for a pair of radios, or a small group with a defined controller, who can control who has what slice of time.  But as mentioned, that doesn't make sense for Mesh Extenders.

Instead, we need a simple CSMA (Carrier Sense Multiple Access) or similar scheme that allows disorganised radios to communicate with each other on a transient and dynamic basis. The downside to this is that the effective bandwidth will be probably less than 1/3 of what can be achieved using TDM (which is part of why the original firmware uses TDM).

Because we can now also have potentially a lot of radios talking, we need a clear way to mark packet boundaries in the serial stream -- for both transmit and receive -- so that we can reliably send and receive whole packets.

To this end, I have culled all the TDM code out of the RFD900 firmware, and implemented some simple packet framing.

The framing on the receive side we had already implemented for the Mesh Extenders. This just puts a simple prefix on each received packet in the serial buffer. That prefix contains a magic value, as well as some useful statistics, mostly signal strength.

For the transmit side, I have changed the firmware so that it holds onto bytes in the TX buffer until it sees "!!".  To include a ! in the packet it must be escaped as "!.".

After a bit of fiddling around, I was able to get all this working fairly nicely.

The only other piece was to add a simple Listen Before Talk (LBT) function that holds off sending a packet until the ambient noise drops below a threshold.  This is to stop us talking over the top of other stations.  This is vital now that we no longer have TDM to keep the radios from talking over the top of each other.  For now I have a hard-wired threshold, but I will make this configurable soon, and if I can think of a good approach, I will make it dynamic based on the background noise on the channel, so that it doesn't stop transmission in busy urban areas, but still stops talking over the top of distant stations when in low-noise environments.

(In fact, this whole effort was triggered by problems using the existing LBT code in version 1.6 of the RFD900 firmware, on which our Mesh Extender version is based. Basically if you set the LBT register on that firmware, the radio stops responding.  This may well have been fixed in later versions of the firmware.)

Anyway, while there is more work to be done, you can see the work so far at

Friday, August 28, 2015

Testing RFD900 radio at Arkaroola

Last week I was up at Arkaroola for Science Week, and took the opportunity to test the RFD900 radios up there in preparation for some upcoming Mesh Extender testing there.

The idea of the tests was to try the RFD900s out in single-channel (non-hopping) mode, now that we have an experimental spectrum license issued for Arkaroola.  This means that we can run the RFD900 at full power, without frequency hopping enabled, as a precursor to the new radio the RFDesign folks are working on, that can be operated in an equivalent mode without the special spectrum license.

The benefit of all this is that we can completely avoid the synchronisation issue between Mesh Extenders, and thus allow more than just pairs of Mesh Extenders to communicate. This is a big step forward, and one that has been a long time coming!

So, I configured the radios down here in Adelaide, but set to 1mW, so that we would not be in breach of the LIPD Class License under which the RFD900 is typically operated.  This let me confirm that I had them setup properly.

Then, up at Arkaroola, I put one radio in the window of one of the buildings in the Arkaroola Village, and drove up towards Coulthard's Lookout, about 4 - 5km away, to confirm that we could get that kind of range.  We were not expecting any problems, but just wanted to make sure that they worked in the rocky terrain that is full of iron and odd shaped rocks on the ground that can mess with radio propagation.

A little way before Coulthard's Lookout there is a ridge that has excellent line of sight back to the village, and is almost 4km from the village.  So we stopped there to test performance, which confirmed that we were losing only a small percentage of packets.  We did also discover that horizontal polarisation was pretty much useless, and that it was the vertical polarisation that was doing all the work.

Here I am in my "other" office. The village is just visible in the distance.

We then went up to Coulthard's Lookout, and were receiving only perhaps 50% of packets, which rather surprised us, as we had expected better performance, however, it was adequate.  It was only the next morning driving home that I realised that I had left the radio set to 1mW !  So the RFD900 was still getting 50% of packets through at 1/500th the TX power that we could use.

Given that every 4x increase should double the range, this means that we should expect 50% packet throughput at a range more like 85km -- and this is with both radios on the ground. Previous tests indicating ~80km range of the RFD900 has involved one radio being in the air on a UAV.

Part of the reason for the great performance (apart from the great design and implementation of the RFD900) is the REALLY low level of background noise at Arkaroola.  Based on the RSSI indications of the RFD900 it is probably 10 - 20dB lower than in Adelaide.  But this isn't news to us, as we know from previous work that the noise floor at 2.4GHz - 5GHz is around -100dBm, compared with around -80dBm here in Adelaide.

Apparently it was family day in the office.

Thursday, August 13, 2015

Working on the Mesh Extender UHF Radios and other things

It's been a while since the last post, but we have been busy in the background on some exciting things:

With the help of the NLnet Foundation, we are making the first steps towards an iOS port of the Serval Mesh.

With the help of USAID, we are working on "slow-media transports" for Android, in particular using Bluetooth device names, and hopefully, Wi-Fi Direct service discovery information, to carry mesh traffic. This is really significant, because it allows carriage of ad-hoc communications on non-rooted devices.  The trade-off is that it is much slower than ad-hoc Wi-Fi.

The third thing we are working on at the moment is making Mesh Extenders work in groups of more than two units.

Historically we have been stuck with pairs because the radios we are using are required to frequency hop when using the ISM 915MHz band available here in Australia, New Zealand and the USA.

We are now actively working to remove this limitation, using the new radios that RFDesign are creating that allow broader spectrum occupancy that will allow them to qualify as spread spectrum devices under the relevant FCC and ACMA rules (LIPD class 45 for the ACMA rules).  This means that we can drop the frequency hopping that causes all the synchronisation issues for ad-hoc and transient groups of Mesh Extenders that currently exist.

In other words, this will allow people to carry a Mesh Extender, and for it to automatically mesh with any other Mesh Extenders that come into range from time to time.  The practical limit will probably be 5 - 25 Mesh Extenders in range at any one time -- something I hope to quantify better in the near future.

Now, the only trouble is that those radios are not yet finished and available.

So we have embarked on an interim solution, by obtaining a spectrum license from ACMA that will allow us to operate the RFD900 radios without frequency hopping at upto 1W EIRP in our favourite piece of Outback, the Arkaroola Wilderness Sanctuary.

Our scientific spectrum license which was granted earlier in the week allows us use of 915MHz - 928MHz at upto 1W anywhere within 130km of 29'52"35S, 138'52"25E, which is great because this affords us great flexibility for testing in a variety of settings.  

The license stands for the next 12 months, which will hopefully be time enough for us to transition to the new RFDesign radios.

We are also open to making use of this license open to other people who would like to experiment in similar ways in that area (but note that it is really quite remote, 400km from the nearest large town, and centred about 130km from the nearest settlements with any facilities).  We would also need to make sure that ACMA and the University are comfortable with such an arrangement, but I am happy to explore this if anyone is interested.

Meanwhile, I hope to modify the configuration of some radios, and put the latest servald on a couple of Mesh Extenders and, all things going well, to perform some rudimentary testing at Arkaroola this coming week.

Update: This map shows the approximate area of the license.

Monday, March 16, 2015

Mapping assistance for Vanuatu Cyclone Response

We have had a request from the head of the Disaster Research Centre here at Flinders University asking for volunteers to help re-map Vanuatu following the category 5 cyclone that hit there in the last few days.

Here is the call for assistance:

Dear MicroMappers,

The United Nations has just activated MicroMappers to support their rapid damage assessments in Vanatu following the Category 5 Cyclone. You can accelerate the UN's efforts by donating a few minutes of your time. Simply click here: 

No prior experience necessary. If you can click "Like" on a Facebook picture or video, you can be a Digital Humanitarian.

Thank you!"