A Drop-In CUDA Implementation Built On ROCm

Iron_Bound · February 12, 2024, 11:13pm

So this just released, allowing cuda to run on AMD hardware without code changes.

Hows the performance for existing appliations? as it could be interesting for science and video editoring software.

MazeFrame · February 12, 2024, 11:31pm

Would be huge for De-Noising in Davinci resolve (yay, 3 FPS without CUDA)

bedHedd · February 13, 2024, 1:08am

Interesting story that Intel and AMD both sponsored the solo dev initially, but both pulled out later

LinuxNoob1 · February 13, 2024, 1:32am

Given that this requires ROCm 5.7, what are the odds I get this working on Ubuntu 22.04 with the 6.5 HWE kernel and a 7900XTX?

My initial attempts earlier today were a bit rushed as I was busy constantly, but I was unable to get ROCm 5.7 installed at all.

Iron_Bound · February 13, 2024, 4:00am

This guide has help me install ML packages before, Ubuntu jammy is supported too.

github.com

nktice/AMD-AI/blob/main/ROCm-5.7.md

# AMD / Radeon 7900XTX 6900XT GPU ROCm install / setup / config 
# Ubuntu 22.04 / 23.04  
# ROCm 5.7.3
# Automatic1111 Stable Diffusion + ComfyUI  ( venv ) 
# Oobabooga - Text Generation WebUI ( conda, Exllama, BitsAndBytes-ROCm-5.6 ) 

## Install notes / instructions ##

 I have composed this collection of instructions as they are my notes.
 I use to setup my own Linux system with AMD parts.
 I've gone over these doing many re-installs to get them all right.
 This is what I had hoped to find when I had search for install instructions -
 so I'm sharing them in the hopes that they save time for other people. 
 There may be in here extra parts that aren't needed but this works for me.
 Originally text, with comments like a shell script that I cut and paste - 2023-07 - nktice


2023-09-09 - I had a report that this doesn't work in virtual machines (virtualbox) as the system there cannot see the hardware, it can't load drivers, etc.  While this is not a guide about Windows, Windows users may find it more helpful to try DirectML - https://rocm.docs.amd.com/en/latest/deploy/windows/quick_start.html / https://github.com/lshqqytiger/stable-diffusion-webui-directml

2023-09-30 - Updated to use ROCm 5.7 - As it is out now, and does appear to be working much like 5.6...

This file has been truncated. show original

ulzeraj · February 13, 2024, 8:41am

I also wrote a guide for ROCm and popular AGI software through Distrobox (containerized environments) which should be distribution agnostic. It’s a few months old but should still work.

https://habla.news/a/naddr1qqxnzd3ex56rwvfexvurxwfjqgsfam9gjjew3qcwqhkgdax3r80yzx3d6w4uke2jtkmfcjr0ftl93qsrqsqqqa28vfwv5f

As for the project I’ll definitely try it with CUDA-only stuff that gave me a hard time to run in the past such as TortoiseTTS.

PaintChips · February 13, 2024, 2:52pm

Factoring in compute differences of AMD, your mileage is still going to vary as Adobe like Davinci Resolve offloads encoding to NVENC so CUDA is only a small part of the encoding puzzle. Science stuff that leans on floating point the performance per watt on AMD drops off on the memory bus, HBM Radeon series had been the closest AMD had gotten.

Before they sponsored the dev, their OpenAPI stuff had been going on(Intel Compute Stick/Compute PCIe cards) so its more than likely an internal project beat it on the funding side.
CUDA on AMD or Intel GPUs is still a mixed bag due to I/O differences, heck an Arc GPU barely reaches an RTX 3060 if pushed on heavy CUDA stuff but for medical imaging its fairly on par to an AMD Radeon Pro of a similar gen.

MazeFrame · February 13, 2024, 3:38pm

Encoding (for export) depending on format, supports AMD and Nvidia.
The issue that makes me swap my 5700xt for the RTX 4000 for projects that need it is denoising performance. 3 FPS without CUDA, 40-ish with CUDA.
In-Progress system build will hold both AMD and the 4000 because raw crunch

PaintChips · February 13, 2024, 4:56pm

Filtering of any type is still CUDA, been hitting the brick wall on Intel Arc and also learned you need a dummy display adapter if you attempt to use a 2nd GPU headless.