Warning for OBS users: AVOID NVENC HEVC!

OBS has very advanced features for enabling the NVENC chip on Nvidia cards but they haven’t enabled the HEVC encoding for good reason:

IT CRASHES THE DISPLAY DRIVER.

Try it yourself. Try to use the hevc_nvenc encoder in a custom FFmpeg output in OBS and try to stop the recording after starting it. (if you can find the right arguments to get it successfully started) YOUR WHOLE DISPLAY DRIVER WILL CRASH, ALL THE TIME, EVERYTIME.

While HEVC is an awesome codec for 4K 60p and above, and Nvidia GPUs of recent have the capability to use it, only stuff not using FFmpeg like Elgato’s dedicated capture utilities are stable enough to use it for something like their new Game Capture 4K60.

Sorry OBS fans. We’re limited to NVENC H.264 if we want to encode 4K 60p and you don’t have a CPU as powerful as Threadripper for x264. Also, protip, the only functioning way to record 4K 60p in H.264 NVENC is to choose “High” profile, leave the Level on “auto” and use a bitrate of 150mbps with 2 GPUs. One dedicated to OBS and NVENC, and one being the primary rendering GPU for the game. Bitrates above 100Mbps will NOT UNLOCK if you choose a manual Level selection, say Level 5.1. This is the weirdest thing about the NVENC encoder I have discovered to date.

5 Likes

Just a little update. Got NVENC 4K 60p working with NVENC H.264 with 2 GPUs in use, a GTX 1080 for game rendering, and a GTX 1060 6GB dedicated for OBS and NVENC. This is the only way to do this on a lower clock speed and lower core count CPU (6 compared to 16) with tons of PCI-E lanes. Don’t skimp and get a 1050Ti or a 1060 3GB, cause I saw 75% GPU usage while recording 4K 60p on the dedicated OBS GPU.

Been using NVENC HEVC on OBS custom ffmpeg output for quite a few months (half an year actually), without problems so far!

Parameters: profile=main10 tier=high level=6.2 rc=constqp rc-lookahead=60 spatial_aq=1 qp=20

Hardware:
i7 4790k
GTX 950

I mean this has been known for a while now.

Hmm, must be a Pascal only issue then.

Maybe, i don’t know what settings are you using, but if it helps, here its a capture from mine.

The issue isn’t the settings. The issue is it crashes the graphics driver when you stop the recording. This is present on Pascal cards when you do a custom FFmpeg recording. I guess Maxwell cards don’t have the same issue.