Hardware encoding is FAST, not good. If you want a “proper” file, use CPU. If you want it on the fly, in embedded systems or systems with no CPU to speak of or fast in general, GPU encoding “ASIC” is the way to go but you have to limit yourself to not always optimal presets.
I’ve tried to do some hw transcodes to h264 and h265 and av1 on 7900xtx and pretty much gave up immediately.
I dint note down the details, but there were absurd results from seemingly optimal presets in handbrake , like h265 transcode larger and worse looking than original h264.
FAST and LIMITED properties of hw accelerated encoding is pretty succinct observation, whats worse it lack of good documentation and limitations differ across generations and vendors.
Some specific profiles worked without obvious issues, but quality was not good enough for my use. Good enough for streaming, which is probably the primary use case here.
(I didn’t re-compute the datapoints under 100MB, that’s why no log scale)
There is a thing called gop_size that controls how often a frame is recomputed from zero. I don’t care about scrubbing performance, so I just set it at -g:v 10000000
So how does this compare now versus CPU encoding? I have always used my Threadripper to do CPU encoding, and I only use GPU encoding for gathering game clips quickly, or if I had to stream something.
I use Handbrake and I have different options based on the TYPE of video I am transcoding. I do some testing on the quality and make sure that the quality is as “lossless” as I can notice when watching on my 32" 4K Photo monitor’s. So if I cannot notice any artifacts during high stress times of the videos, then I deem it to be fine (and if color is not affected at all).
So it takes me a few runs to optimize the settings for each type of content and then I set it and forget it. However I transcode with CPU using my Threadripper 3960x… I can give it 6-8 cores and just leave it for a week if I have to transcoding everything I need until its done.
Doing this I cut the size of my video library by atleast half and I couldn’t tell the difference no matter which source I watched on.
I haven’t found any video that takes longer than 1 day to re-encode even with CPU. The slowest encoding I have ever encountered (where quality still mattered) was 3FPS on my system. However usually I can encode at 8-18fps and 32-40 fps in 720p (which lots of older Cartoons and such are in)… I get 20-30fps in 1080p usually with my settings. 8-18 is 4K with higher bit rates.
You should be able to get a bit more out of the encoder with preanalysis settings tuned. pacaqstrength set to high gives maximum compression efficiency and using pataqmode 1 for transcodes or 2 for recording with both using a lookaheadbufferdepth of 11 should give good results. For recording you can leave scene change detection disabled, but for transcoding enable it.
Handbrake, being what it is you can really push the encoder hard with it taking so much data into account with the correct parameters if you do not care about speed at all.
The AMF documentation is an amazing resource for getting the most out of the encoder, but it is a really boring read.
I haven’t ever used those specific settings, unless they correspond to the UI.
I have lots of settings in the UI and I can share my configs here if people would like.
