Seeking guide to run AI on 6950XT for STT conversion

Hello,

I wanted to do some Speech-To-Text conversion locally. I know there are AI code bases, like whisper. I know that there are some pre-trained models out there, like on hugging face.

I don’t know how to make it work on my GPU. Like, the docs talk about installation, not how to get up and running. I know that some AI engines can even run on a modest ARM SoC. But IDK how to do that either.

I wanted to run the AI engine on my RX6950XT. I’m running Linux kernel version 6.12.X.

I’d appreciate any pointers so that I could learn how to do this. I did try googling without effect.

If there are other tools that you’d recommend, I’d be happy to learn about them. IIRC, some time ago Louis Rossman was talking about a STT engine, but I can’t seem to find the video nor the engine.

Anyone know how to setup and run a STT conversion on an AMD GPU?

Thanks!

1 Like

are you on linux? can I recommend handy? https://handy.computer/ it’s pretty robust and works well even on modern cpus/npus/etc.

2 Likes

Sure, but, “It runs completely offline using Whisper,…” So it looks like it just uses whisper in the background which is not very handy. :wink:

Quite frankly, if I’m going to build whisper anyway, why not just use whisper from the beginning?

Whisper is one of the models you can use, but it it’s not the only model. And it’s entirely offline. And it’s not even thebest one (imhho) anymore.

Forgive me but the newbishness of your original post suggests a basic starting point, more basic than trying to DIY something with whisper.

This is a basic starting poing that lets you effortlessly experiment with a bunch of different models that vaguely fit into the “STT” bucket. There’s lots of nuance there.. continuous speech recognition, recognition for short commands, push to talk, etc. Handy sort of gives you an EZ gui for these kinds of things AND you can decide if you’ve vastly overestimated the difficulty of STT AI at this point, or not, for what actually needs to run on your GPU. Or maybe your CPU is capable. Or maybe your CPU has an NPU which leaves your GPU for real AI tasks.

Don’t know till you try… What are you expecting? A bunch of copy-paste commands into the terminal?

2 Likes

I got the quote directly from handy’s website. I’m uncertain why they didn’t mention that there were multiple possibilities where I was reading.

This is a basic starting poing that lets you effortlessly experiment with a bunch of different models that vaguely fit into the “STT” bucket. There’s lots of nuance there.. continuous speech recognition, recognition for short commands, push to talk, etc.

Allow me to clarify then. I’m used to compiling, and sometimes editing, source code. I’m looking to feed in an audio recording and get text out. Not push-to-talk (realtime STT) or anything fancy like that.

Don’t know till you try… What are you expecting? A bunch of copy-paste commands into the terminal?

I was thinking there’d be a guide out there that I could be referred to. You know, RTFM. But where are the Fine Manuals? I tried searching, as I noted above, but didn’t find anything.

Thanks!

To be clear, I already have opencl up and working on my GPU, so I’m not starting from scratch. I just need to understand how to connect all the parts together. That’s what’s confusing me.