I made a small system agent that sends output to a remote shadow server (nagent)

So you may be familiar with check_mk, I’ve been using it for almost 8 years now for getting details about system state and have quite a few custom check_mk plugins in the wild.

tmpfs                 tmpfs     3249480     2508   3246972       1% /run
/dev/mapper/data-root ext4    482351616 46396120 411379908      11% /
tmpfs                 tmpfs    16247396   185416  16061980       2% /dev/shm
tmpfs                 tmpfs        5120        0      5120       0% /run/lock
/dev/nvme0n1p1        vfat       508920   253816    255104      50% /boot/efi
/dev/nvme0n1p2        vfat      4186096  2612292   1573804      63% /recovery
/dev/sda1             ext4    491133848       28 466112124       1% /mnt/data
tmpfs                 tmpfs     3249476      988   3248488       1% /run/user/1000
tmpfs                 tmpfs    16247396        0  16247396       0% /run/qemu

I wanted to expand it to make a micro shadow service that takes the output from check_mk and similar styled scripts and turns it into JSON, and stores it in a shadow for each host that I can pull into other services.

So this thing I made called nagent (NATS agent) sends checkmk style output via NATS, which is picked up by nagentd.

That daemon uses script hooks on the incoming data for generic modification/rejection, but then passes each section within to a script with the same name as the section who converts that data to JSON. That JSON output from each script is then combined to make a shadow for each host.

I cheat a little bit here, leveraging the python JC tool to parse many different outputs right out of the box.

Finally the shadow is then sent to ‘sink’ script hooks which allows you to add scripts to send the full JSON shadow to MQTT or webhooks or where ever. The shadow is also available from an HTTP API.

It’s BYOAuth ATM, but I think it’s good enough for an MVP as the kids call it. I usually wrap all this stuff inside a VPN or SSH tunnel anyway.

I am aware that check_mk have some kind of commercial offering these days, but I have not looked into it at all, so may have just reinvented the wheel (and open sourced it).

Maybe someone can find it useful or have some tips or suggestions for me.

1 Like