As a Python backend engineer by trade, but an AI hobbyist at heart, I’ve been exploring small toy programs to run on my Turing RK1 (RK3588)-based Turing Pi 2 cluster, ever since Llama 7B became a thing and llama.cpp could run it. The cluster boasted a 5–6 TOPS NPU (on each rk3588), so I figured I’d experiment.
Unfortunately, not all TOPS are created equal, and the Rockchip RK3588 seems geared more toward the YOLO era of ML compute (iykyk).
But thankfully, I also have an Acer Predator Helios 300 laptop with an RTX 3070 Ti mobile GPU (and I upgraded the system RAM to 32 GB).
Most of the projects below were vibe-coded around the 2023–early 2024 generation of LLMs (so they’re not exactly technical masterpieces).
One of the first apps I developed was a Rudimentary Video Summarizer. I don’t necessarily recommend you try it today or at all…it’s not exactly good code by my standards. The idea was simple: given a YouTube video link, download the audio, transcribe it with whisper.cpp, and pass the transcription to an LLM to summarize it. It did get the job done. What’s sad is that youtube still hasn’t made something like this that passes on the COMPLETE transcript of the video for summarization.
Developing this app got me into learning how to compile whisper.cpp with OpenVINO (was a pain to get it working on Fedora 38/39 back then) to speed it up for my i7-12700H since I couldn’t load whisper and an LLM on 8 GB of VRAM. Nowadays it would be much easier, as there are distilled versions of Whisper and smaller LLMs that are much smarter.
Another small “app” I built was what I called WhisperTypist. I basically wanted voice dictation. This used Whisper and a utility called ydotool to actually type things out (simulating the keyboard). It was fun to watch work, even though it’s pretty hacky and not the right way to build a voice-dictation app. Again, not really suitable for public consumption, to be totally frank.
Probably the most useful project I built for someone else was a custom Chrome extension for my brother: an email summarizer. For any open Gmail email thread, a single click would pass all the email bodies to a local LLM server and summarize everything in a custom pop-up extension window.
Another handy project I made for myself was a Firefox extension that lets me select chunks of text (usually articles), right-click, and pass the text to a Python server running a sentence chunker (thank you to the NLTK project at UPenn) and the wonderful Kokoro TTS model to essentially read articles aloud.
Probably the most useful thing I made for myself was not actually an AI app, but a tool I use in my own AI workflow everyday : a VS Code extension I vibe-coded to combine code files in the project directory and have the combined code available on my clipboard to pass it on to an LLM, since I was NOT going to pay for a cursor or windsurf agentic coding VSCode fork.
Anyway, those were some of the toy apps I’ve made and gotten some use out of. What have others been doing?
If I can get my hands on a system with more vRAM (I was eyeing a GMKTec Strix Halo 128GB that MicroCenter had in stock for $1,899 but has since jacked up the price to 2099) to redo most of these apps and back them up with bigger and better local models aka project “Build Back Better”.
After I’m done reviving and re-architecting the above mentioned projects (I’ve picked up a few pythonista tricks here and there). I’ll probably try and make a epub–>custom audio book app. All my apps, I want to access over tailscale from anywhere in the world.