Radeon RX 6400 for Home Assistant AI Acceleration

High-Performance Computing Machine Learning, LLMs, & AI
1

Hello. Could anyone advise me as to whether the RX 6400 could be a viable choice for LLM acceleration, to be used in conjunction with Home Assistant? Low power draw and its small size make it an appealing choice for that purpose, as it would always be on, and would ideally be installed into a small form factor case of some kind. Being its only tasks would be initially for simple things like switching stuff on and off, answering simple questions, telling you the weather, and such, I am hopeful that it may do the trick. Oh, and I’m hoping to keep things as local as possible, and only jump the fence for certain functions like for camera alerts, turning on some lights, or things like that… Keeping most of the libraries/training data stored on the machine, with the ability to also use the cloud in certain circumstances, when necessary. Thanks in advance!

2

I don’t know the answer to your question - it’ll hinge on whether there are few enough parameters to fit in 4 gb, I suspect.

Let me pose a quick sanity check, though - is there any reason not to just start out with CPU, though? A use case like that should be just fine, I’d think.

3

Thanks for the reply. I would consider using the CPU, but I plan on putting this all on the ZimaBlade so I’m definitely not going to have much to work with. It runs HA just fine on its own, though once I split things into VMs and docker containers I will be pushing things to the limit, I think.

4

If I understand correctly (and I have only implemented ollama and not the homeassistant integration) it will still take some CPU to run… I have not been able to get any LLM running on my rx550 to test just how much. Do you have any other servers running that could offload?

5

I have an rx6400 paired with an i7-8700, but I’m on Windows 10 and have no idea how to test those workloads for you.

6

PCIe x4 and DDR3 RAM

uhhhhhh

2 Likes
7

Same price, 8 GiB DDR4, actually has a way to power the hard drives you might conceivably plug into those SATA ports, probably not much worse on idle power (because the PSU is 12V-only).

Used equipment, people!

8

umm no… I have an rx6400 in a box. However I am thinking in putting it back in my system whenever I can afford another monitor and using it as a gpu pass through with virt-manager. I would not be gaming on it, however as a display for VM maybe an option.