Just discovered this little beauty. If you are into AI / LLM experimentation across multiple models, then you need to take a look. Windows, Mac and Linux versions available. Currently beta for Linux, but I’ve had no issues running it on Linux Mint Cinnamon (save a few minor and easy to ignore display bugs) in the last week across three systems. It works amazingly well.
It can search the full Hugging Face catalog and recommends compatible models based on your system specs, then downloads them to your system. It can run multiple models simultaneously along with its own inference server function.
Full help features explaining all functions and a really polished piece of software.
6 Likes
I’ve been using LMStudio combined with AnythingLLM and it’s awesome. Can give it PDFs or websites to cite too! I’d suggest giving it a shot!
1 Like
This is cool! I’m saving this for later
1 Like
Just spent two hours using Anything LLM and concur. Thanks for that suggestion.
I’m a degen that uses SillyTavern and Koboldcpp. You know when people using those tools for inference they are not using it for serious work haha. I wish a non javascript program existed that could read the card format.
Sweet! Looking at it now!
I will note, if you have a system without AVX2 (I did until recently, went from an Ivy Bridge (AVX but no AVX2) to a Coffee Lake, my Ivy Bridge is relaxing running a DVR) then Intel SDE (“Software Development Emulator”) may let you run this anyway. It’ll trap and emulate any instructions your CPU doesn’t support while having the rest run at full native speed. If you want to run “xyz” it’s like “/path/to/sde xyz”. They also have this for Windows. It’s intended so a software developer could have software that can use new instructions for a nice speed boost or whatever developed, tested, and shipped out a bit ahead of the CPUs actually supporting those instructions coming onto the market. It’ll traps and emulates at least back to SSE4, and I think it may go all the way back to MMX (also supporting the odd instructions like TSX, you could use it to run something requiring AVX512 on the newest Intel CPUs where AVX512 has been removed, etc.)
It’s pretty fast, I had an old potato some years back where the Second Life viewer went from only needing SSE2 to requiring SSE4.2 without even a mention in the release notes. Running newer one under SDE I got literally a 0FPS speed difference (so I didn’t get the speed benefits from the SSE4.2, but trapping and emulating the SSE4.2 instructions wasn’t any slower than running the older version that wasn’t using SSE4.2 to begin with.)
If you get a “illegal instruction” on whatever app, it’s worth trying it under SDE!
