Random reboots and freezes on RX 6800

Hardware Hub GPU
, ,
1

I have been having random issues with my AMD GPU (RX 6800 non-XT). At first, I brushed it off as software bugs because the only major issues I had were random reboots when using ROCm(basically CUDA but AMD) on Linux to run AI and reboots when using ROCm and Blender on Windows, specifically with low rendering output resolution for some reason??? Yeah, super confused on that one. I discovered it when I was bored and set the output resolution to something like 180x144 to see how fast it would be, and spamming F12 to re-render would cause a reboot on my card consistently. All of these were using ROCm, so it was reasonable to assume it was just software bugs; after all, ROCm is a bit(a lot) buggy, I heard. I got a AMD GPU expecting this to some degree, hoping VRAM for cheap would make it worth it in the end. And at the time my GPU was perfectly stable besides these weird reboots when doing specific things. And maybe minor graphical weirdness? But I just thought those where bugs in the game, because afterall they where so minor not like textures going all crazy and lines everywhere, things like that happen even with my old card nothing crazy yet. Games played mostly perfectly, I’ve played though multiple games at this point without noticing the slighist thing infact.

All of these crashes had something in common though, when I reboot, I get an MCE (Machine Check Exception) in the logs. But I still assumed it was software still. And plus it’s kinda weird since MCE errors usually indicate memory corruption in RAM not VRAM, and I never had these before switching from a GTX 1080, plus I ran memtest86+ for hours and got no errors whatsoever or anything.

So before this, my only issues were using ROCm or Vulkan compute (basically the same thing on Linux, they use the same interface anyway). However, as of late, I have been having random freezes(no reboot) and weird glitchy lines appear on my monitor. I assumed an update must have broken something, but I was getting suspicious at this point, so I switched to Windows to be 1000000% sure. After switching to Windows the weird lines where still there, but only when I played games seemingly and so far at least I haven’t had any freezes but that was quite rare and random. on Linux they where always there when doing anything.

I am pretty convinced I have faulty GPU memory probably, and it has degraded overtime to where I am now where it’s always noticable afterall things I talked about before are VRAM intensive. It seems extremely subtle though which I why it has taken me so long to suspect it’s hardware related. My explanation for the MCE errors is maybe it happened when transferring from CPU to GPU, meaning it was hardware afterall.

Anyway, I’m mostly just asking if there is even the slightest chance it could be something else than hardware failure before completely throwing in the towel. And also just general advise because I have never had to waranty a GPU or really anything. For reference I bought my GPU new off Amazon from XFX in 2023, but I don’t have the original box anymore. And maybe someone else has had similar experience.

And MCE errors:

Feb 27 15:09:35 nixos kernel: mce: [Hardware Error]: Machine check events logged
Feb 27 15:09:35 nixos kernel: mce: [Hardware Error]: CPU 9: Machine Check: 0 Bank 5: bea0000000000108
Feb 27 15:09:35 nixos kernel: mce: [Hardware Error]: TSC 0 ADDR 1ffffc0754f4a MISC d012000100000000 SYND 4d000000 IPID 500b000000000
Feb 27 15:09:35 nixos kernel: mce: [Hardware Error]: PROCESSOR 2:870f10 TIME 1709068166 SOCKET 0 APIC 3 microcode 8701030
Feb 27 15:09:36 nixos kernel: MCE: In-kernel MCE decoding enabled.

Error from Linux drivers themselves before reboot:

Feb 27 22:08:15 nixos systemd[1]: Starting Cleanup of Temporary Directories...
Feb 27 22:08:15 nixos systemd[1]: systemd-tmpfiles-clean.service: Deactivated successfully.
Feb 27 22:08:15 nixos systemd[1]: Finished Cleanup of Temporary Directories.
Feb 27 22:08:24 nixos kernel: amdgpu 0000:09:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000028 SMN_C2PMSG_82:0x00000000
Feb 27 22:08:24 nixos kernel: amdgpu 0000:09:00.0: amdgpu: Failed to enable gfxoff!
Feb 27 22:08:29 nixos kernel: amdgpu 0000:09:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000028 SMN_C2PMSG_82:0x00000000
Feb 27 22:08:29 nixos kernel: amdgpu 0000:09:00.0: amdgpu: Failed to enable gfxoff!
Feb 27 22:08:30 nixos kernel: amdgpu 0000:09:00.0: [drm] *ERROR* [CRTC:95:crtc-1] flip_done timed out
Feb 27 22:08:31 nixos kernel: [drm:amdgpu_job_timedout [amdgpu]] *ERROR* ring sdma0 timeout, signaled seq=3400, emitted seq=3401
Feb 27 22:08:31 nixos kernel: [drm:amdgpu_job_timedout [amdgpu]] *ERROR* Process information: process  pid 0 thread  pid 0
Feb 27 22:08:31 nixos kernel: amdgpu 0000:09:00.0: amdgpu: GPU reset begin!
Feb 27 22:08:31 nixos kernel: amdgpu: Failed to suspend process 0x800c

This kinda seems like a race condition in the driver, but again I’ve been having other issues now. And I was going to attach a video of the weird lines, but it seems I can’t upload anything yet since I just made a account:/

2 Likes
3

This is my first time on this site so sorry if I did anything wrong or this isn’t right place. But I have also noticed that the weird lines seems to mess the colors up in a row of pixels, shift some of the pixels, and oddly doesn’t happen in game but on my other monitor on the desktop. The game itself seems fine lol

1 Like
4

I had weirdness with my RX580 crashing and having MCE errors on linux in the past. What I found out was that two seperate problems were at fault.

The first was that the GPU driver would crash whenever a certain number of vertices were drawn in Blender’s viewport. I fixed that by upgrading my kernel/mesa version.

The second issue was a bit trickier. It ended up that I hadn’t touched my motherboard in almost 10 years, and resocketting the CPU fixed the machine-check errors.

If you’re at the point of desperation, I’d give at shot at just resocketting the GPU, CPU and RAM. Maybe even CPU and GPU power plugs as well.

I don’t know about the RX 6800, but I do know that the RX 6900/6950 are quite picky about the power supply they’re plugged into. So if you have a spare PSU laying around, you may give a shot at powering the GPU seperately from the rest of the system.

If all else fails, I would try contacting XFX for an RMA, and send them a copy of your Amazon invoice if you still have it. Even if you don’t have the box, I can’t imagine them not accepting a proof-of-purchase from a reputable seller.

1 Like
5

I’d have to second this, most of the GPU instability I had with my 6950XT were power draw related. Though if it’s managing to reset without having to reboot, it’s more likely to be driver related. Actually having some showstopper problems with ARMA Reforger via Proton due to this, though equally having problems with any UE5 engine game :expressionless:

3 Likes
6

Welcome to the party!

tl;dr;

Have you tried throttling the power limit down, underclocking the GPU, and confirming the voltages and temperatures look good?

Thoughts

I haven’t used an AMD GPU for years now unfortunately because as you mention the software landscape isn’t quite as easy for rocm vs cuda. But there may be some things you could try before returning the card.

Just the Facts

To summarize the facts (as I understand them) from your post and your profile:

  • i7 8700k (assuming stock volt/clocks?)
  • 16 GB RAM (assuming JEDEC/XMP profile with no crazy OC?)
  • Linux Mint 19 (what kernel e.g. uname -a. You are using nix package manager on mint?)
  • Dual booting Windows 10 ?
  • Old GPU was a GTX 1080 (no issues with this build)
  • Removed the GTX 1080 and installed RX 6800 (problems begin)
  • Problems happen both in Linux and Windows, no system freezes in Windows

Debugging

I’m much more familiar with nvidia-smi and not sure how to go about the AMD / Linux GPU driver landscape and tooling. A few questions that may be relevant:

  1. Are you using xorg or wayland environment?
  2. What version of GPU drivers / Linux kernel are you using (is it mesa or AMD drivers??)
  3. Do you notice kernel errors happening during testing or only after rebooting and checking error logs?
  4. Have you installed and learned to use corectrl or whatever the tooling is for AMD GPUs? Might need some kernel boot parameters or /sys/stuff to get that setup just guessing…

You could possibly gather some stats while running your tests e.g.:

## Terminal 1
# install lm-sensors package
$ watch -n 0.5 sensors
$ sensors
amdgpu-pci-5c00
Adapter: PCI adapter
vddgfx:        1.28 V
vddnb:         1.28 V
edge:         +39.0°C
PPT:          11.00 mW

## Terminal 2
sudo dmesg -T --follow

You can monitor power usage, voltage, temperatures, and kernel logs and see if it prints anything before freezing and snap a photo with your phone.

Then you could try similar tests in windows using hwinfo64 to monitor temperatures/voltages etc.

Collect some baseline configuration info with how you have it setup now, then try throttling down the power limit, or underclocking a bit, and confirming the correct voltage ranges and repeat the experiments to see if it still crashes and if you notice differences in temperature / power usage / voltages etc.

References

Conclusion

Good luck! I agree that we need more cheap VRAM in this world! haha… Cheers!

1 Like
7

No idea if this is related, but there’s a nasty bug in recent kernels due to an AMD GPU driver bug that seems to cause freezes.

3 Likes
8

I had 8700K @ 5.2-5.4GHz with 5700XT for a couple of years and when I upgraded to RX6800XT I started experiencing random crashes and finally some cores died on the CPU. Probably it was the PSU fault (Corsair 700W), since after an upgrade to Intel 13th gen got similar crashes and freezes. Also, in my experience, the 6.5 kernel on Mint causes similar trouble

9

For amdgpus take a look at:

GPU Control/Overclocking

Each of these is useful, I mainly stick with CoreCtrl because I know it pretty well.

For monitoring

  • amdgpu_top
  • radeontop
  • btop

I am using a rolling release distro and btop finally started reporting my power usage as of this week which is nice. amdgpu_top has a dark mode if you hit Shift + T and radeontop shows slightly different stats than it.

3 Likes
10

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.