SPDIF configuration for PulseAudio?

I’ve been having trouble with choppy sound in Ubuntu 18.04 and later 20.04 that I’ve been trying to fix for the past couple of years. The sound skips every once in a while or distorts, especially when trying to play a DVD.

Today it got so bad that I switched to my headphones and noticed the sound was much better. No skipping or distortion was there at all. I’ve heard here and there about there being problems with optical output (S/PDIF on my motherboard) so I was wondering, is there any special configuration that I need to get optical out to work correctly? I tried turning on realtime priority in pulse (by default it’s off in Ubuntu), but that didn’t seem to help.

Im using a SPDIF out on my pc running Manjaro to my Cheapo Edifier and it works fine, I never had any problems. This may not be a SPDIF specific problem.