big and round

Project Horus is a high altitude balloon project. My first memory of Project Horus was when they launched Tux into near space at Linux Conf Au 2011 - my first Linux Conf Au. It would be awhile before I would start working on high altitude balloon related things. In that time Project Horus moved from using RTTY (radio teletype) for the payload telemetry to a 4FSK mode called horusbinary. Today horusbinary is used by many different amateur high altitude balloon payloads and supports additional custom fields.

Typically horusbinary payloads are decoded using the “horus-gui” app by connecting a radio to a laptop or computer via soundcard interface.

Mark’s done a great job packaging this up to make it easy for receiving stations to get up and running quickly.

Authors

Written by: Python Library - Mark Jessop vk5qi@rfhead.net

FSK Modem - David Rowe

FSK Modem Python Wrapper - XSSFox

And apparently I made the python to c wrapper using clib… I don’t even remember this.

For more casual users or people using phones and tablets could we build something more accessible? Recently I played around making a web frontend for freeselcall using web assembly which prompted Mark to ask the question:

but yeah, this is interesting, and suggests a horus decoder is probably possible

since i think you wrapped fsk_demod in a similar way for freeselcall as you did for horusdemodlib?

Introducing webhorus

webhorus is a browser based recreation of horus-gui allowing anyone to decode horusbinary balloon payloads from a browser providing a “no install” method and usable from mobile phones. It’s even possible to decode telemetry by holding up a mobile phone to the speaker of a radio.

webhorus is fully functional and is fairly close in feature parity to horus-gui.

How it was built

So horus-gui uses python library horusdemodlib which in turn uses the horus_api c library. That means if we build a website we’ll end up with the trifecta of Python, C and JavaScript in a single project.

horusdemodlib

I decided that I wanted to do minimal changes to the existing C and Python library and decided to include horusdemodlib as a git submodule. I had to request some changes via PRs to fix some bugs and make things easier to work with but it was fairly minimal.

One thing I didn’t want to use however was ctypes. The existing module uses ctypes and it has some drawbacks. The two biggest for me is that its very easy to get things wrong with ctypes leading to memory corruption and the other is that the current approach requires a dynamic linked library thats built beside the module, not with.

Since I’m packaging this for the web I decided to use cffi . cffi lets you define a c source file - usually you can just use your .h file¹ - that it uses to build a compiled python module with.

struct        horus *horus_open_advanced_sample_rate (int mode, int Rs, int tx_tone_spacing, int Fs);
void          horus_set_verbose(struct horus *hstates, int verbose);
int           horus_get_version              (void);
int           horus_get_mode                 (struct horus *hstates);
int           horus_get_Fs                   (struct horus *hstates);
....

horusdemodlib is simple enough that I could replicate the cmake process inside FFI().set_source()

ffibuilder.set_source("_horus_api_cffi",
"""
     #include "horus_api.h"   // the C header of the library
""",
      sources=[
        "./horusdemodlib/src/fsk.c",
        "./horusdemodlib/src/kiss_fft.c",
        "./horusdemodlib/src/kiss_fftr.c",
        "./horusdemodlib/src/mpdecode_core.c",
        "./horusdemodlib/src/H_256_768_22.c",
        "./horusdemodlib/src/H_128_384_23.c",
        "./horusdemodlib/src/golay23.c",
        "./horusdemodlib/src/phi0.c",
        "./horusdemodlib/src/horus_api.c",
        "./horusdemodlib/src/horus_l2.c",
      ],
       include_dirs = [ "./horusdemodlib/src"],
       extra_compile_args = ["-DHORUS_L2_RX","-DINTERLEAVER","-DSCRAMBLER","-DRUN_TIME_TABLES"]
     )   # library name, for the linker

Using this approach along with setuptools (for me via poetry) means that the c library is built as part of your normal pip install or package build.

I wrote a very small wrapper around the cffi library

import _horus_api_cffi

.....

self.hstates = horus_api.horus_open_advanced_sample_rate(
	mode, rate, tone_spacing, sample_rate
)
horus_api.horus_set_freq_est_limits(self.hstates,100,4000)
horus_api.horus_set_verbose(self.hstates, verbose)
.....

This is considerably easier than ctypes as you typically have to define the args and the return types. Any mistake will lead to at best a segfault and at worst undefined behaviour.

I utilised existing helper functions from horusdemodlib python library to ensure easy maintainability and decoder parity.

From here I can perform a pip install ./ and obtain a functional modem. So we’ve replicated what the horusdemodlib python library could do, using horusdemodlib and in Python…. wait what were trying to do again… right we needed static link python library - check.

from webhorus import demod
horus_demod = demod.Demod()
frame = horus_demod.demodulate(data)

pyodide

Next step is getting this from a compiled python library into web assemble to call from the browser javascript. Since we have python building the c library for us pyodide pretty much takes care of the entire process.

Once pyodide is installed and activated its literally just a case of doing:

pyodide build 

Everytime I’ve performed this I’ve found a handful of compiler issues to clean up. Nothing major. In this case it was just ensuring that one of the functions always had a return path. The output of this build process is a python whl file compiled as wasm32. I always think this step will be the hardest but its actually been the easiest so far.

Python in the browser

From the Javascript side with Pyodide installed² we can start accessing Python. I opt’d for putting the python code in its own file so that my IDE stopped getting confused. One important note here is that the pyodide library “.wasm” file needs to be served with mime type “application/wasm”

const pyodide = await loadPyodide();

// so we can actually load up micropip and install our packages and dependencies via pip - great for development however for production release I didn't want to rely on a third party server.
pyodide.loadPackage("./assets/crc-7.1.0-py3-none-any.whl")
...
pyodide.loadPackage("./assets/webhorus-0.1.0-cp312-cp312-pyodide_2024_0_wasm32.whl")
pyodide.runPython(await (await fetch("/py/main.py")).text());

What’s really neat is just how good the integration between python and Javascript is. Pyodide have done an amazing job with this. Using things like getElementById from within python just feels so wrong - but works so well.

horus_demod = demod.Demod(tone_spacing=int(document.getElementById("tone_spacing").value), sample_rate=sample_rate)

Adding audio

The last piece of this puzzle is audio. We need to capture microphone input to feed it into the modem. This was so challenging to get working. Jasper St. Pierre wrote a good blog post on “I don’t know who the Web Audio API is designed for” and I definitely feel that frustration on this project.

Now there’s some extra frustrations with what we are doing. We need as clean unprocessed audio as possible. I found out the hard way that Chrome tries to optimise audio for video calls. But before we get to that we need to get access to the audio device. This must happen through user interaction - so we force the user to hit a “Start” button, create an AudioContext and eventually perform a getUserMedia. On most browsers this will pop up a consent dialog requesting access to microphone. Prior to this we can’t even list the audio devices.

Now that we have consent and we’ve opened and audio device, then we list what audio devices the user has. To get the unprocessed audio we have to perform another getUserMedia with constraints set to remove any filters. The ones we care about are echoCancellation, autoGainControl and noiseSuppression. However if we blindly request these constraints turned off we’ll break Safari. So first we do a getCapabilities on the device first to see what we can set.

It’s all a bit of a hassle and getting the order of operations correct so that each browser is happy is a bit frustrating. Opening the microphone was only half the battle, we also need to obtain PCM samples to put into the modem. The only³ non deprecated way to do this seems to be via the AudioWorkletNode. This runs a bit of Javascript in a node / sandbox / thread. This actually sounds ideal - I could run the modem in its own thread and just send back decoded messages. However the sandbox is so limited that loading the modem into that environment is just too hard. Instead we just buffer up audio samples and when we have enough use this.port.postMessage to send buffers data back to the client. Oh and because this is the Web Audio Api we need to convert -1 to 1 floats to shorts. What fun.

Anyway, that’s all the tricky parts glued together. The other 80% is building out a UI with Bootstrap, Plotly, and Leaflet + figuring out how to get Vite to build it into a solution. The code is up on GitHub if you’d like to check it out, including a GitHub Actions workflow so you can see the entire build process.