[Devember2021] A web based real time view of amateur radio communications - [LAUNCHED and LIVE]

[x0r]

Blurb:

There are many windows/linux desktop applications that are used to help monitor and communicate digitally over the airwaves with other amateur radio users.

I would like to move these to a more web based application that will open the digital world to the masses. Allowing everyone amateur or not to be able to listen in to digital stations.

I am recently a new Ham. I got into this hobby due to the wonderful online tools called websdr. This is great for analogue radio, but what about digital?

I will start off small, aiming at either APRS for VHF band. And maybe FT8 for the HF bands.

I have created an empty github that will soon be filled:
https://github.com/x0r9/digiradio-v

A Live demo: https://m1.x0r.uk/digiradio-v/

there are plenty of software tools out there. So I will start working with those to see how they work. I know there are efforts for them to communicate with each other in standard (or not so appropriate standards). So I will probably need to pick up on this first.

This will likely be a python project and my philosophy will be to keep it as simple as possible.

[x0r]

An update:
I haven’t really played with any code so far. I have mostly been tinkering with the current hardware that I have.

Presently I have a Portable VHF/UHF FM rig. I have set that to listen onto the APRS channel. After that the plan is to pipe that sound into a USB sound card, jobs done?

I am walking into a few issues. One is a awful Mains power line hum. This is still there when on batteries, just reduced. This hum is not audible on the in built speaker, but only on the sound output jack.

I tired building an audio isolation transformer, but nothing was going through it. I can scope the sound in, but not out. Not sure if it a loading issue or not, but it’s around 100Ohms the transformer.

Given that didn’t work. I thought I’d try it the old fashion acoustic coupling way. Plugged in a cheap-o bundled headset from a baofeng and electrically taped it in front of the laptop microphone I had.

This has it’s own problem with noise and trying not to much the volume too loud that it clips and distorts. I noticed that some of the signals I was picking up seem to me more ‘crackly’ than others. I don’t know if that’s because my handset or theirs are set to narrow/wide FM so either under or over deviating on these things.

Looking at the captured waveforms I am not getting any clean high tones, although low tones look okay. This may be to do with the way some radios try to boost higher tones to over come hiss noises. I am not sure if that is something my handset is doing.

Either way I bought a second hand cheap VHF radio, very popular for modifying into data modes in repeaters. A Motorola GM340.

So I will get to play with that. And maybe mod to do DMR modes later down the line.

[x0r]

Another Update 11th Dec:

I have recieved my GM340 and I had a DB sub9 port and a sacraficial ethernet patch lead to make a serial programming cable. But searching the pinout quickly told me it wasn’t an RS232 serial standard. But rather Motorola’s own “RIB” or Radio Interface Box. This is their own UART bus. So I bought another usb to RIB serial interface. Do’h! Set me back a few more days until that arrived.

The interface did arrive and I managed to program the radio to talk to the APRS frequencies and also managed ensure the back expansion adapter was spitting out flat unfiltered audio. No pre/post emphasis. I hooked the output to a scope and it looked promising.

Then finally, let’s make the cable into the audio jack and get Direwolf fired up. I pulled out some cheap jumper cables from my project box and got soldering. When i went and neatly heat shrinked it all all to be tidy. I tested for continuity and nothing.
I found the cause of this, it wasn’t my soldering, but rather than jumper cables were pre crimped. But poorly, crimped on the insulation, not the copper. Hence no connectivity. Wasted 30 minutes there, and made sure my next set didn’t have the same build issues.

Finally I power it up and get a waveform capture in audacity, just to check it levels and noise.

Now that’s clean and was absolutely what I was expecting! you can see the mark/space going between the two tones.

Firing up Direworf showed it was decoding stuff, looking good. But to help visualise it, I put it into YAAC (yet another APRS client).

So I was happy. And I also managed to finally prove out that my Anytone 878UV plus was spitting out valid packets. If the weather is good, I may go out for a walk and see if it will track me.

So Hopefully that will be end of the majority of the hardware. I may tidy it up and later deploy the software onto a PI rather than my desktop for a more 24/7 appropriate operation.

Now onto software! I wrote some python script to simply talk to the KISS TNC over TCP. That is effectively the API to direwold decoding the packets off the sound card.

Fortunately there is a python KISS and APRS packet module written. So I’ve imported them and am now saving packets to file for later replay.

The APRS module seems to need a little help with decoding the to/from addresses based on the demo code I’ve seen. But I will see next fully decode the data and extract station geolocations.

The YAAC screenshot is effectively what I would like to mimic in a browser.

Oh and here is a photo of the corner lab I have

[x0r]

Update Dec 12th:

This is my anytone handset if anyone was wondering what it looked like. It has inbuilt GPS and TX’s the position over the air.

I’ve manage to go out with a small walk with my Anytone and it picked a number of packets when i walked about. It really only picked up when i was on hills as my house is buried a little in the middle of the urban environment. Although I was pleasantly surprised it performed better when I did similar tests last year with my DMR hotspot.

With that data captured, time to process it. I had to import another library called aprslib, this seem to understand and parse all the information data. Where as the aprs library really only decoded the X.25 portion of it

Once I got that done, i need to graph the lat/lon data now? Well I decided to generate a GPX file of the track. As I am familiar with those a little. So another python library added in and a GPX file was added, and I parsed out a few call signs and plotted them in gpxsee software.

Here is a few packets my station picked up from some one transmitting from their car.

It’s not got every single point out. But what is curious about these positional updates is that I didn’t hear them coming directly from the car. They were bounced/repeated off 2 digipeaters. So I am curious to know the exact path these messages took, and they are present in path section on the x.25 packet, so that would also be interesting to plot.

[x0r]

Update: 17th Dec 2021

I’ve now been collecting the data, time to start visualising it.

I’ve created a basic web page/app. It runs with FastAPI. I’ve never used it, looks pretty simple to use so far. I’ve used Flask about 10 years ago. Fast API has async support which i’ve never used in python and I hope to try using websockets, maybe in this devember release, maybe not.

For the front end I used leaflet.js for mapping. I’ve used that. Also Jquery because I’m familiar. It does get the job done. I would like to read more on react/vue/angular type of frameworks. But I want to get something visible. I’ll look into those things later.

So far on page load it fetches a bunch of JSON data of the last X minutes/hours of data. And adds those markers to a map.

I’ve done some crude collection of objects and stations. If the station moved to try and draw a line where they moved from.

I’m seeing markers all over the country and I’m wondering how I am capturing it. I.e. what is the path that packet took, and did it go through an iGate? I find it curious how such data could of teleported so far. Be it misleading packet, cheating through the web, or some dump luck of being able to transmit at the right time and place to pick up good conditions.

[x0r]

There still is a number of things I need to improve on (including but not limited to):

• Move captured data from file to a Redis cache
• Update Icons on map to distinguish types of data
• Clicking on a point giving more in-depth information
• The map auto-refreshing
• Consider containerising the app
• Move JS/CSS into their own static files
• Add different mapping layers to help understand terrain better

[x0r]

Update: 18th Dec

I’ve added in icons. Borrowed some from the same guy that made aprs.fi icons. These were two large image files. The icons are in a table format, I wrote some browser code to crop the icon it needs. I’ve never done that, and it was straightforward using canvas. Something i have played with a few years back.

Another upgrade was caching data into redis. I forget that redis is some what primitive in the data it can hold. So It was quite the document database I tried to setup with. I don’t care too much for data persistence. But i have had to put in changes from everything being a bytearray to it’s string/float or other equivalent.

Presently I moved off path history of moving stations. There is something not quite right with the way i was processing it. And i embedded that history in a list under a key for each object. that’s too much nesting types for redis! I will have to come back to this later.

I also quickly change the mapbox streets-v11 to outdoor-v11. It’s a little better. But I realised I need to figure out how to cache some tiles for offline use when using remotely in the field.

Remaining to do:

• Clicking on a point giving more in-depth information
• The map auto-refreshing
• Consider containerising the app
• Move JS/CSS into their own static files
• Add offline/local app tile cache option
• Think of putting this live for the viewers/judges to play with

[x0r]

Update: 30th Dec

Okay, I have 2 days left. But that doesn’t mean I haven’t been busy for the past 12 with now update.

I’ve dona a lot, but now I have moved everything to redis cache now. I have created a websocket for the page to listen to and give real time updates. These updates are sent through redis pub/sub model.

I have also containerised this app now, and spent a bi of time fixing up the docker compose I have orchestrating redis/this app/ nginx reverse proxy and other apps my sites run behind the scenes.

I had a lot of small annoying bugs and fixes I need to make to deploy it live. But it’s there now.

The only main issue I have now is that being live on a linode server, it doesn’t have a direct connection to my radio. So I will have to some how pass those packets/information from my computer at home to my linode instance.

So I may play with websockets once more and see If I can make an application that be a websocket client at home, connect to my linode instance with aide of an API key of sorts and start feeding it live data.

This should keep the firewall and application simple. In that no TCP access to my KISS TNC which may have accidental TX capabilities, or exposing redis which could have it’s own security issues.

But before I got, just before Christmas, around 22nd/23rd in the UK we had some good conditions for the 2M band to the point I was picking up stations not just south of the UK, but also the whole of France and Belgium and a little bit of Germany.

So leftover todo:

• Clicking on a point giving more in-depth information - Later feature post Devember
• Add offline/local app tile cache option - Later feature post Devember
• Think of putting this live - Partially Done
• Steam data to ‘cloud’

[Nic_s]

This looks pretty cool. A friend of mine recently got into the hobby and has for the past few days been catching images from the ISS.

[x0r]

@Nic_s

This looks pretty cool. A friend of mine recently got into the hobby and has for the past few days been catching images from the ISS.

Yeah, there are a lot of things you can do! I’ve been meaning to get a contact with ISS and other amateur satellites.

The ISS even has a APRS digipeater on it! I hope to ping that when I get this station to transmit as well as just receive. It would be pretty straight forward for me to reprogram my GM340 to listen for the ISS APRS packets.

[x0r]

Update Dec 31:

Okay it’s a mad dash to get this up and running. Distracted as I am listening to drunk trio rambling on their NYE live stream.

I added an API that is a websocket for me to stream raw packets from my station. And it works. But it was a little messy getting there as the websockets is all async/await while the KISS client was all blocking sockets. So I had to hack something in to get that to work.

I accidentally forked the python kiss library on github, but If I get frustrated enough with the library I might program more hacks in anger. Looking at other forks of that library, I don’t seem to be the only one! But otherwise it’s a very good start with a few minor imperfections, just a lot of potential for growth.

anyway, we’re launching, and its here on my vanity domain:

https://m1.x0r.uk/digiradio-v/

You’ll notice a green star flashing every 10 seconds. this is just a heart beat from the server saying the websocket is alive. But any other stations that flash, are just a packet coming in to update that point.

The flashing animation could be improved, like on aprsdirect.com it has a radio wave animation that pops out a lot better.

there are a lot of life improvements on this. But I may want to start prioritising automating testing before this grows. As this will catch problems quickly before I end up debugging in a live environment. Not an issue as it’s not a very popular page. But it makes tracking and controlling changes harder.

Anyway, Happy new years folkes. I’ll try and keep the system live for the next week or two. Might setup a raspberry PI to feed the data for a more power efficient and dedicated computer for those things.

But this is me submitting my entry!

[x0r]

You got me thinking @nic_s about the ISS, that too has an APRS digipeater. But it doesn’t work on 144.800Mhz that is used for normal terrestrial signals. it’s on 145.825Mhz.

So I programmed my radio with that extra channel and to scan for both. So when the ISS passes, it picks it up.

And I can say it did work! I picked up a few stations in europe. Given it passed at 2-4 am I doubt many operators were awake!

I wonder if I will see more as the ISS passes during more social able hours.

[x0r]

Update 13th: Jan 2022
The past week I haven’t had much luck picking up from the ISS, i haven’t changed anything on the radio. Although checking today I see Sporadic E again.

I have created another repo GitHub - x0r9/pi-digiradio

This will have the docker build and nginx setup that I used to host it. As you can tell from the name, it’s planning to be a complete self host raspberry pi (or other SBC) setup. I’m in the middle of setting that up.

In the meantime, I bought a new radio, a Tait 2000 series. For UHF. Might fiddle with that for other digital modes.