Thinking about off grid / grid down networking and ham radio

Hello all, I wanted to talk a bit about the subject of off grid networking and digital ham radio communications that leverage modern technologies that we use and rely on every day. I’ve been doing a number of experiments that leverage Raspberry Pis, phones, tablets, and laptops in the shack, in the field, and while traveling to operate one or more ham radios.

I’ll go through the ways I’ve configured my field computers to operate in: as a single device, as two or more devices, as devices connected to a $25 travel router running OpenWrt, and as equipment connected to a home network. The reality of this setup is that all my off grid computers work in one of two ways given any situation, but with some subtle differences.

The broad scenario 1 is when one of my Pis detect a wireless network they’re configured to connect to. This can mean they’re connected to a familiar Internet-connected wifi network at the QTH or elsewhere as a client, OR they’ve connected as a client to a wifi network generated in the field by another device designed to operate off grid. This means the device is acting as a standard wifi client.

The second broad scenario that one of my Pis supports is that it does not detect a recognized wireless network and generates its own. Only one device should be in this mode in any scenario. Once a single device generates a network any additional Pis, laptops, tablets, phones, etc. should connect to this Pi which will also automatically serve DHCP, DNS, NTP (in some cases), etc. This host then becomes responsible for handling basic network services as well as its ham radio related functions. There is, however, a drawback to this scenario with my current setup. For some reason my Raspberry Pi 4s and Raspberry Pi Zero W will stop passing network traffic after a few minutes. The solution to this problem is just to re-connect to the wireless network and things work again, but reliability is an issue. If you can’t simply log in and reconnect a Pi you have to reboot it. I address my solution in one of the more specific scenarios discussed later.

By supporting these two broad ways of handling connectivity we can create effective on and off grid networks. This makes sense because the first host that gets powered up has the chance of becoming the master of the wireless network. By coordinating the name of your off grid or standalone wireless network and its pre-shared keys you can create a network that can be formed on the fly, and that can have at least some self-healing properties in the event of a device failure. If all the nodes are capable of becoming an access point with all necessary network services in the absence of another we can ensure our services are available, or are as available as possible.

Let’s start with the five very specific scenarios we mentioned earlier. Notice that these five scenarios are mostly combinations of the two very broad ways our devices are configured to behave.

First let’s say that things are normal and we’re doing some digital HF or UHF/VHF work in the shack. Our local infrastructure is working fine, our Internet connectivity is fast and working well. Because all of our devices are clients on our home network we can use any device to access VNC, SSH, HTTP, or Winlink services on our Pis without issue while watching our favorite streaming service. All the Pis we have powered up are clients to an established network.

Secondly let’s talk about a scenario where we’re on battery but no configured wireless networks are available. Maybe we’re in an emergency scenario, camping, or on the road. Our Pi attached to our radio generates a new wireless network that you can connect to and starts the necessary DNS, DHCP, or even NTP service to allow wireless clients to connect and use provided services. Maybe this Pi is running JS8Call/FlDigi or Pat Winlink. You just connect your phone, tablet, or laptop to this network and access the documentation web server, Winlink web page, or use VNC/SSH to control the Pi. We’re now able to communicate using our radio via digital modes and read any documentation we stored on the Pi. It should be pointed out that none of my field-portable Pis have keyboards, displays, or monitors of any kind when in normal use. I rely on other devices for those functions to cut down on power consumption and bulk.

Our third scenario assumes one operator with no configured wireless networks, but is running two radios. Maybe we have one Raspberry Pi attached to a UHF/VHF radio that’s set up for packet Winlink, and another HF rig running JS8Call. The Raspberry Pi that booted first is serving our wireless network and basic network infrastructure services that allow subsequent devices to connect. The operator is now able to connect to to the generated wireless network and use Pat Winlink as well as JS8Call or FLDigi. One person and one device can now manage two radios set up for different purposes! The trick is that we might not have 100% due to the connection issues with Pis when they’re in access point mode.

For our fourth scenario we’ll modify the last scenario a bit. We have one operator with one device connecting to two radios over the wifi connection. This time we’ll introduce our modified $25 wireless access point. It’s now generating the wireless network, serving DHCP, DNS, and NTP if one of our Pis with GPS units is connected. All Raspberry Pis are now in wifi client mode so we no longer have the connection reliability issues in our second and third scenarios! What’s even better is that if we have a wired Internet connection we can now plug that into the travel router and have full Internet connectivity for our Pis and operator’s device. Sweet!

Our fifth scenario builds on the fourth, but we’ll introduce one or more operators using one or more devices. This could also apply to the third scenario with a Pi hosting the wireless infrastructure without an access point but we’ll ignore that. Every radio operator connected to the wireless network can now queue up e-mail on Winlink or take turns using VNC to control our HF rig. All the operators on the network now have access to the documentation server because it’s just a web page that serves files. We now have a field network that can serve more than one radio operator, and we can be more efficient sending email over Winlink by batching, and we can also decide to create an APRS server that allows multiple instances of APRSDroid or some other KISS over TCP client to connect if we’d like.

I hope you came away from this thinking about different ways we can build resilient and flexible systems even when working off grid or in emergency situations. A minimal amount of additional hardware combined with some preparation ahead of time can allow us to use more efficient or sometimes required digital technologies off grid and can even allow a group of operators to save power by sharing radios and batching messages rather than dealing with the overhead of connection setup for modes like Winlink!

Portable UHF/VHF Winlink host

Howdy! I had recently posted about some Raspberry Pi based systems that can be used in the shack or in the field and those posts received a lot of questions about how they were set up. This post is the first in a series that seeks to explain the design, operation, and setup of these hosts. Some of the work I did here was inspired by Julian, OH8STN’s general off grid/grid down operating philosophy and K1CHN’s blog posts.

A Raspberry Pi W Zero in a clear acrylic case connected to a Mobilinkd TNC3 with a MicroUSB OTG cable, host end connected to the Pi's USB OTG port.
Raspberry Pi Zero W connected to Moblinkd TNC3 using a MicroUSB OTG cable.

So, what are we talking about here?

  • A Raspberry Pi Zero W / Raspbian based Pat Winlink host.
  • Designed to work with packet on UHF/VHF radios.
  • Compatible with a wide range of radios.
  • Runs on USB or 12v and has typical 45A Anderson PowerPole connectors in a “right hand red” configuration when running on 12v. This matches my off grid power setup and the ARES + RACES standards.
  • Uses a Mobilinkd TNC3 to work with a variety of UHF/VHF radios and to offload signal processing to a TNC. That means we don’t need as much processing power on the compute node. Multiple adapter cables are sold on the Mobilinkd store and you can make your own.
  • Automatically sets up and tears down the AX.25 port and connection based on the status of the TNC’s USB connection.
  • Nice, tidy web interface that can be used by any device with wifi capabilities and a fairly modern browser to send and receive e-mail.

Parts list:

  • Raspberry Pi Zero W and power supply
  • MicroSD card
  • Optional Pi Zero W case
  • Mobilinkd TNC3
  • Appropriate cable, purchased or constructed to connect from the TRRS jack on the TNC3 to your UHF/VHF radio
  • MicroUSB OTG cable for the Pi to TNC3 connection.
  • Optional USBBuddy (12v to 5v USB down converter)
  • Optionally short USB A to MicroUSB cable to cut down on voltage drop.
  • UHF/VHF radio of choice. This setup has been tested with a Baofeng UV-5R, Yaesu FT-857D, Yaesu FT-3DR, Kenwood TM-D710G, and a Kenwood TM-V71 but should work with many others. The Mobilinkd TNC3 was designed to work with a broad range of radios.

Theory of operation:

Block diagram showing control flows between and connections between the operator, radio, devices, and between the TNC and Raspberry Pi. The diagram also depicts the control flows between the TNC, ax25.service, Linux AX.25 subsystem, and Pat Winlink. Flow charts depict AX.25 service flows and a wifi flow chart describing when the system decides to host its own wifi.
Block diagram showing various system components and their interactions. Flow charts for managing the AX.25 service and self-hosted wifi.

The Pi leverages a number of smaller subsystems to provide e-mail access.

  1. Scripts and applications that provide AX.25 setup and teardown as the TNC is connected or disconnected. These scripts leverage udev and systemd to detect state changes on the TNC. While you can use Bluetooth to maintain these connections it’s easier and more simple to control connections to the TNC using its USB connection status.
  2. A set of scripts, services, and utilities that automatically provide a functional wireless network in the event the Pi is unable to connect to a known network. This includes DNS and DHCP.
  3. Pat Winlink is run as a systemd service so the operator doesn’t have to worry about starting and stopping the service. It can run in the background whether or not the TNC is connected, but it can’t send or recieve e-mail without the TNC connected.
  4. Pat Winlink is accessed via a web interface. You need a phone, tablet, or computer that can connect to the wireless network generated by the Pi, or is on the same network as the Winlink server. This allows an operator to use Winlink.

Setup process

Start by installing Raspbian on your MicroSD card and getting your Raspberry Pi up and running. The steps here should work. Make sure you give your pi a hostname like “winlink” or whatever you’d like. If you configure dnsmasq this becomes important. Once your Pi is up and running with an Internet connection we can pre-install needed utilities and software so we don’t have to do it later. We’ll also make sure dnsmasq and hostapd don’t start automatically (more on this later). These commands can be done from the Raspberry Pi’s terminal application or from an SSH session if you’ve enabled it:

sudo apt-get update -y && sudo apt-get upgrade -y
sudo apt-get install libax25 libax25-dev ax25-apps ax25-tools dnsmasq hostapd git
sudo systemctl disable hostapd
sudo systemctl disable dnsmasq

Once we have all that installed we’ll install and configure Pat Winlink. Download the latest Pat Winlink release from GitHub. You’ll want to make sure you choose the armhf (Raspberry Pi) .deb file. Make sure you note the name of the file as it will be important when running the install command. When you’re ready to upgrade Pat Winlink in the future you can download the newest version of the .deb file from the same page. Assuming you’ve downloaded the file to the Downloads folder you can run the following command to install Pat Winlink, replacing “pat_0.10.0_linux_armhf.deb” with whatever file name you downloaded:

sudo dpkg -i ~/Downloads/pat_0.10.0_linux_armhf.deb
mkdir ~/.wl2k

After we configure our AX.25 settings we can come back to configuring Pat Winlink as we’ll need some values from that setup.

Next we’ll edit /etc/ax25/axports. This file configures our AX.25 ports that get used to build connections to packet Winlink gateways. Mine looks like this. You’ll of course want to replace my callsign with yours. The port we configure below will be called wl2k. This will be needed for the Winlink configuration in a number of spots.

# /etc/ax25/axports
#
# The format of this file is:
#
# name callsign speed paclen window description
#
wl2k K7JLX 9600 255 7 Winlink (9600)

In the next step we’ll create three files – a script that manages AX.25 port connections, a systemd service that manages the AX.25 port, and finally a udev rule that starts and stops that systemd service when the Mobilinkd TNC3 shows up as a USB serial device or is disconnected.

First, let’s install ax25-up, a script in the Winlink project that manages AX.25 connections. Our systemd unit depends on it. The commands in this step come from here, but are slightly modified to put the script in a different location on the Pi.

bin="/usr/sbin/ax25-up"
sudo curl -o $bin https://raw.githubusercontent.com/la5nta/pat/master/share/bin/axup
sudo chmod +x $bin

Next we’ll add the udev rules that work with systemd to start and stop the service. Add the following contents to /etc/udev/rules.d/99-hamradio.rules:

# Mobilinkd TNC3
ACTION=="add", ATTRS{idVendor}=="0483", ATTRS{idProduct}=="5740", ATTRS{manufacturer}=="Mobilinkd LLC", SYMLINK+="ttyTNC", TAG+="systemd", ENV{SYSTEMD_WANTS}="ax25.service"
ACTION=="remove", SUBSYSTEM=="usb", ENV{PRODUCT}=="483/5740/200", TAG+="systemd"

We’ll now create our systemd unit file which should be: /usr/lib/systemd/system/ax25.service Note the ExecStart command where we use our wl2k port, and /dev/ttyTNC as created by our udev rules. That device is created in the event we attach another USB serial device to create a predictable name no matter what the proper udev name of the device is.

[Unit]
Description=AX.25 KISS interface
Before=network.target
Wants=network.target
BindsTo=dev-ttyTNC.device
After=dev-ttyTNC.device
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/sbin/ax25-up /dev/ttyTNC wl2k 9600
ExecStop=killall kissattach
KillMode=none
[Install]
WantedBy=multi-user.target

Configuring and setting Pat Winlink up as a service

First we’re going to put in a configuration for Pat Winlink. We’ll do that by editing ~/.wl2k/config.json. You’ll want to replace the values I have in my configuration with your own. I’m connecting to a station called W7LT-10 most of the time. You’ll also want to remove your password until you’ve set one. Follow these instructions for that process. Replace your callsign and grid square locator with your own. You may also want to add your own connect_alias entries. These can be various Winlink stations you want to “bookmark” for quick connection. I’ve added a number of them for use while I’m out camping. In that list you might have noticed the one beginning with “!W7LT-10”. Since I use that one most commonly I added an ! to the front of the name to keep it at the very top of the list when it’s alphabetized by the Pat Winlink application. The http_addr directive tells Pat Winlink to listen on any address on port 8080. This will be important when constructing the URL to access Pat Winlink with. You may also notice that I’ve configured this to listen on ax.25 and telnet. This allows the Pat Winlink application to listen in peer-to-peer mode for incoming connections. You don’t have to switch between peer-to-peer and CMS mode manually like you would using Winlink Express. You may want to assign a new telnet password if you want to keep the listen entry for telnet.

{
  "mycall": "K7JLX",
  "secure_login_password": "VerySecret",
  "http_addr": "0.0.0.0:8080",
  "listen": ["ax25", "telnet"],
  "locator": "CN85qm",
  "motd": ["K7JLX Pat Winlink stn"],
  "connect_aliases": {
    "!W7LT-10, 144.910MHz, PDX": "ax25:///W7LT-10?freq=144910",
    "K7HWY-11, 145.030MHz, La Pine, OR": "ax25:///K7HWY-11?freq=145030",
    "KB7EOC-10, 144.950, Tillamook, OR": "ax25:///KB7EOC-10?freq=144950",
    "KG7AV-10, 145.030MHz, Bend, OR": "ax25:///KG7AV-10?freq=145030",
    "W7EUG-10, 145.030MHz, Eugene, OR": "ax25:///W7EUG-10?freq=145030",
    "W7GC-11, 144.950, Hebo Lake, OR": "ax25:///W7GC-11?freq=144950",
    "WC7EOC-10, 144.980MHz, Hillboro, OR": "ax25:///WC7EOC-10?freq=144980",
    "WW7CH-10, 145.050MHz, Eatonville, WA": "ax25://WW7CH-10?freq=145050",
    "N7GWK-10, 145.630MHz, Centrailia, WA": "ax25://N7GWK-10?freq=145630",
    "N7DEM-10, 144.920MHz, Longview, WA": "ax25://N7DEM-10?freq=144920",
    "W7BO-10, 144.920MHz, Woodland, WA": "ax25://W7BO-10?freq=144920",
    "KA7CTT-10, 144.920MHz, Vancouver, WA": "ax25://KA7CTT-10?freq=144920",
    "W6TQF-10, 145.055MHz, Underwood, WA": "ax25://W6TQF-10?freq=145055",
    "N7YRC-10, 144.930MHz, Yakima, WA": "ax25://N7YRC-10?freq=144930"
  },
  "ax25": {
    "port": "wl2k"
  },
  "telnet": {
    "listen_addr": ":8774",
    "password": "AlsoVerySecret" 
  }
}

Now that we have a solid initial configuration let’s set Pat Winlink up as a service that starts automatically when the Pi boots. The first step is creating a new systemd unit at /lib/systemd/system/pat.service which will run as the standard pi user that ships with Raspbian. The contents of that systemd service are as follows:

[Unit]
Description=Pat Winlink HTTP server
After=ax25.service
[Service]
Type=simple
ExecStart=/usr/bin/pat http
User=pi
Group=pi
[Install]
WantedBy=default.target

Now we start and enable the service by running some commands in the terminal. The last command should show Pat up and running with a green “active” status. You can press “q” to quit the status display.

sudo systemctl enable pat
sudo systemctl start pat
sudo systemctl status pat

To test all the work we’ve done thus far reboot your Raspberry Pi using the GUI or by issuing the following command in the terminal:

sudo shutdown -r now

Once your Pi is back up and you’re logged back in as the pi user we’ll connect the Mobilinkd TNC3 to the Pi using the MicroUSB OTG cable. The host side connects to the Pi’s USB port, not the power port. After plugging the cable in press the connect button on the Mobilind TNC3. You’ll see a yellow flash on the TNC’s status LED. This momentary button press should trigger the ax.25 systemd service to start. We can check on that by running the following command:

sudo systemctl status ax25

If you see that the service is active and green you’re good to go on the base Winlink functionality. The only thing left to do is connect to it. Use your browser connect access Winlink with a URL derived from this template: http://<your winlink hostname or IP>:8080

If you want an automatic hotspot proceed to the next step.

Automatic hotspot

For this part of the guide just follow the steps that Raspberry Connect lists. You can modify their scripts to create IP addresses and wireless network names/passwords as needed. You can modify the /usr/bin/autohotspotN script and set the IP address there. In the createAdhocNetwork() function modify the ip a add line with the desired IP and subnet mask.

After the scripts scripts have been run and things have been configured you can optionally set up DNS records in dnsmasq. My configuration looks like this but yours will certainly vary. The static IP addresses and DNS records help Android devices or other systems that don’t work well with MDNS find your winlink service. You can use any network you want. The dhcp-host line for winlink.local is commented out because the host we’re running won’t get a DHCP address from itself. The additional entries help other Raspberry Pis or other devices get static IPs and allow them to be found by hosts. In this way we can make sure any devices that connect to your network that offer services show up. Make sure the ‘address’ line matches the hostname of your pi, and that the IP matches the one you set after the RaspberyConnect’s script run in the previous steps.

interface=wlan0
no-resolv
bind-interfaces
dhcp-range=192.168.255.20,192.168.255.250,4h
address=/ft857d.local/192.168.255.1
dhcp-host=dc:a6:32:11:11:11,192.168.255.1
address=/winlink.local/192.168.255.2
#dhcp-host=b8:27:eb:22:22:22,192.168.255.2
address=/tx500.local/192.168.255.3
dhcp-host=dc:a6:32:33:33:33,192.168.255.3

Multicast DNS (MDNS)

To help devices that support MDNS automatically discover the winlink service we can add a configuration file to Avahi. To do that create a file at /etc/avahi/services/winlink.service:

<?xml version="1.0" standalone='no'?>
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
  <name replace-wildcards="yes">Winlink on %h</name>
  <service>
    <type>_http._tcp</type>
    <port>8080</port>
  </service>
</service-group>

Now restart Avahi using the following commands:

sudo systemctl restart avahi

At this point if your pi can’t reach a known wifi network it should start its own. Make sure you test it before you need it to work in a real-life situation.

As a bonus you can create a document server on the Pi to make sure you have information, forms, manuals, etc. when you need them in the field.

Updates to this blog entry:

4 Sep, 2021 – The Pat Winlink configuration section was updated to reflect changes I’ve made since the original draft of this entry.

A random field day (Jan 16, 2021)

I had the opportunity to spend a few hours in the Oregon countryside while my partner had a meeting. Naturally I decided to do deploy my new radio, the Lab599 Discovery TX-500 along with my second purpose-built digital comms Raspberry Pi. The other is used with my Yaesu FT-857D.

Picture of radio equipment in the back of a Prius along with a folding chair facing the hatch back.

I began by setting my rig up in the trunk of my car. Since I wanted to at least simulate running off grid on battery I didn’t connect my radio to the car and opted to use my 40Ah Bioenno LiFePO4 battery. I had intended to bring my smaller 12Ah Bioenno LiFePO4 battery which was actually purchased for the TX-500 kit, but I had spaced it and left it on the charger. Despite the cloudy weather that is typical of Oregon this time of year I also brought my GoalZero Nomad 20 to see if I could extend my runtime even if slightly and to give it a good test. Every little bit of extra juice helps, but I only used 1.8Ah of battery the entire 5.5hr deployment! The solar panel did provide an additional 0.8Ah which is 44% of what the battery provided.

Folding solar panel placed on top of the car roof with cable running to hatch back.

Solar panel on the car and facing south

Buddipole power mini lashed to a 40ah LiFePo4 battery
Buddipole Power Mini and 40Ah Bioenno LiFePo4 battery.

The first antenna I deployed and ran was my Superantenna kit, but instead of using the titanium whip supplied with the kit I added the Chameleon Mil Whip 2.0 to get more efficiency and significantly wider SWR bandwidth. I tuned the antenna up for 20m using my NanoVNA and ran JS8Call on the TX-500’s dedicated Raspberry Pi… using my tablet as a keyboard and screen over VNC. I had a number of successful contacts from the Southwest to AK and managed to relay a text message to a friend in NM via an operator in-state running 9w!

Superantenna deployed on a tripod topped with a Chameleon Mil Whip 2.0.
Superantenna w/ tripod and Chameleon Mil Whip 2.0

I did try to make some SSB phone contacts but there was a contest going so I didn’t really get too far. As the sun started going down I noticed the 20m band was starting to close, so I tuned to 40m and the contest was still going on so I wasn’t able to make any contacts. It can be difficult to raise anyone during a contest because a lot of folks are talking and running high power so it’s very easy to be drowned out.

In general I also like to try more than one antenna or antenna configuration per deployment so I set up my Chameleon EMCOMM Portable III in an inverted “V” configuration with the center point hung using an arborists’s weight and some paracord in a tree. I was able to make some JS8Call contacts and was able to hear a lot of distant operators. Again, I was unable to make a contact using SSB phone despite the fact that the tuned inverted V configuration should technically be more efficient than a loaded vertical. I’ll need to do another test on another day.