MKV, Handbrake, H264

[Phoenix_Rising]

Recommendations and Advice.

Currently using MKV to backup my blue ray and dvd collections. Just using it to backup the content raw to its default save types.

Being an Audio and Videophile, I am wanting to keep the best quality possible of these backups during the encoding process, sticking with H264.

I am aware cpu gives the best compression, not necessarily the best quality, and it is slow. Using handbrake above 6 cores has diminishing returns, lest you are using Xeon/Epyc/Threadripper cores above 16.

I have looked into using GPU encoding with AMD/INTEL/NVIDIA and seen various stats on the times, size and speed.

Currently, I have various AMD graphics cards up to the RX 6600, an old RTX 1650 super.

If keeping them uncompressed gives me the highest 1 to 1 quality, and the only way to reduce size is to remove certain audio and subtitle encodings, then I can accept that, and just increase my NAS storage accordingly. Which this equates to roughly 500TB of storage down from 800.

If anyone has used GPU encoding with best quality settings, while still maintaining decent compressions speed, please post GPU type and best settings used.

I know AMD tends to lag in the quality arena (but I am a loyalist) I don’t have a problem buying an Intel or Nvidia since I have a machine setup strictly for backing up and encoding, would prefer not to though.

So far I have experimented with Akira on both DVD and Blue ray using both CPU at various numbered cores (Hand brake and GPU encoding at various settings).

So far nothing has really given that (awesome) feeling. It is an OCD thing for me, however I am able to see the difference in the visual compression and hear the difference in audio compression from lossless.

Appreciate the input and opinions.

[homeserver78]

@Phoenix_Rising

Being an Audio and Videophile, I am wanting to keep the best quality possible of these backups during the encoding process, sticking with H264.

It’s possible to not re-encode at all but just remux the original into unencrypted Matroska (.mkv) container. That’s what I’ve been doing with my Blurays. I’ve done it manually though using a rather convoluted process to find the right features, audio tracks etc since I haven’t found any program that can do it for me, weirdly enough.

Let me know if you want details and I’ll try to write something up. It involves installing libaacs and then using mpv, ffprobe etc to list the features, and then ffmpeg to do the actual rip.

File sizes range from ~15 GiB to, I think the largest one I’ve done is >30 GiB (lots of film grain in that movie).

Edit: Maybe I’m misunderstanding something: you talk about backing up the content raw and then about encoding…?

[lcslllc]

Outside of Remuxing, as below, remember that you can go from X264 (bluray standard, even 4K) to x265 (well supported, much better compression and picture quality at compressed standard)

In general, my description is: CPU is absolute best but slow, Intel next barely, Nvidia, then AMD but AMD is getting better on this front.

I use VidCoder beta which I just prefer the interface, but it allows me to setup different frameworks after MakeMKV imports. Then I put my bluray on a shelf so that my animals don’t destroy any… had that happen to several once, very bad experience.

Lately, I’ve been cleaning out goodwill stores for Bluray and can come across real steals… and black Friday at Walmart, etc.

[tk11]

@Phoenix_Rising

If keeping them uncompressed gives me the highest 1 to 1 quality

Bluray and DVDs use lossy compression for both video and audio so everything will always be compressed in one form or another.

If you want an mkv that’s an exact copy of the original then you just want to remux. This is just copying as is from the disc straight to an mkv without any transcoding.

If you want to recompress video to save space then you’ll get better compression with HEVC(h.265) over sticking with AVC(h.264). Handbrake supports remuxing just the audio if you select “passthru” as the audio codec.

@Phoenix_Rising

If anyone has used GPU encoding with best quality settings, while still maintaining decent compressions speed, please post GPU type and best settings used.

Your 1650 super has a turing 7th gen encoder so if you want to try hardware encoding you’ll get best results using that with HEVC 10bit. The ideal quality setting depends on the content and personal preference so you’ll have to experiment to get a feel for the compromise between image quality and file size.

[Phoenix_Rising]

Thanks for the reply.

Yes I would be interested in what your process is.

When I was referreing to the encoding process: Using my Akira Anime movie as an example, MKV backs the disc up as an ISO at 8gigs. In order to for media aps to read the video it has to be in a MKV, H264 etc. format. So when comparing the compression tests I have done, the time to compress versus quality is actually a wide gap. So using MKV or Handbrake to remove all non essential add ons like extra audio language tracks, sub title tracks etc, it can bring the final backup down to a managable size.

I would rather spend money on storage, then time to compress and reduce file size, if the original backup is going to be better quality.

At just over 2400 total discs to backup, that is a huge amount of time just to backup let alone convert.

[Phoenix_Rising]

I have been doing the same as well, grabbing a lot of media, especially from the flea market. Some stuff is really hard to come by due to region license and such so I am try to grab as much as I can even if its used, even Laserdisc stuff that was never reprinted into DVD let alone blue ray.

Any process you can emphasize let me know.

Thanks

[Phoenix_Rising]

@tk11

Bluray and DVDs use lossy compression for both video and audio so everything will always be compressed in one form or another.

Yeah, I know. Digital Cinema Packages average about 250gig alone for theaters, then streaming averages about 5-7gig. I showed my wife how bad a streamed version of The Lord of the Rings was compared to a UHD 4k. She understood why I have been so bent on this project.

@tk11

If you want an mkv that’s an exact copy of the original then you just want to remux. This is just copying as is from the disc straight to an mkv without any transcoding.

MakeMKV didn’t give me that option when backing up, lest your are referreing to a different program like DVDfab. My saves by default are ISO for DVD saves, and the bdmv files and folders. Which is okay since I can just rework it in Handbrake.

@tk11

Your 1650 super has a turing 7th gen encoder so if you want to try hardware encoding you’ll get best results using that with HEVC 10bit. The ideal quality setting depends on the content and personal preference so you’ll have to experiment to get a feel for the compromise between image quality and file size.

I was watching a couple videos on GPU conversion using all three brands couple of days ago. I may break down and buy an RTX 3050/60. I messed around with using my dual Xeon 16c server. Surprisingly is really didn’t make that much of a difference as I would have thought.

[tk11]

@Phoenix_Rising

MakeMKV didn’t give me that option

MakeMKV doesn’t support transcoding so remuxing is mandatory not optional.

@Phoenix_Rising

Which is okay since I can just rework it in Handbrake.

Handbrake is the opposite in that it only supports transcoding so you’ll always lose some quality using it because all the supported video codecs are lossy.

@Phoenix_Rising

I may break down and buy an RTX 3050/60

The RTX 3050 and 3060 both use the same 7th gen encoder that your 1650 super has.

Video Encode and Decode GPU Support Matrix

[mihawk90]

@Phoenix_Rising

Being an Audio and Videophile, I am wanting to keep the best quality possible of these backups during the encoding process, sticking with H264.

If you’re wanting to re-encode/transcode anyway, sticking to H264 is pointless.

H264 doesn’t mean you “keep the best quality possible”. Every modern codec can get the same quality given the correct settings, what differs is the bitrate to achieve it and therefore filesize.

You can of course not re-encode and that will mean that for a good majority of Blu-rays you will end up with H.264. Blu-ray also allows MPEG2 and VC1 video though so check your playback devices what they support.

That being said if you are going to transcode, and your playback devices support it, I would skip H.265 entirely because it’s a pain in the ass and just go with AV1 right away. That’s what I’ve been doing with my Blu-ray rips of House lately and I get between 50% and 70% reduction in file size depending on the episode, while still being visually lossless.

@Phoenix_Rising

So far nothing has really given that (awesome) feeling. It is an OCD thing for me, however I am able to see the difference in the visual compression and hear the difference in audio compression from lossless.

Most Audio on Bluray is already compressed. Both DTS and Dolby are lossy formats, except DTS-HD MA and Dolby TrueHD which are lossless, but building on top of the lossy compression. There are Blu-rays that come with PCM audio but it’s a) rare and b) usually just for secondary stereo or commentary audio tracks, not for the main feature.

The video is also already technically lossy, but the bitrate is so high that there is practically no compression going on, i.e. it is visually lossless.

My point being:
If you can see the difference, you just haven’t used the right settings. Every modern codec has the ability to encode in a visually lossless manner at significantly reduced bitrates compared to the original. All it takes is the right settings and bitrate.

@homeserver78

It’s possible to not re-encode at all but just remux the original into unencrypted Matroska (.mkv) container. That’s what I’ve been doing with my Blurays. I’ve done it manually though using a rather convoluted process to find the right features, audio tracks etc since I haven’t found any program that can do it for me, weirdly enough.

That’s literally all MakeMKV does and it’s not exactly hard to find.

@lcslllc

you can go from X264 (bluray standard, even 4K) to x265 (well supported, much better compression and picture quality at compressed standard)

The codec is H.264 and H.265. x264 and x265 are specific implementations of it. And yes that is a fairly important distinction because there are more implementations then just the two.
Also 4K Blu-Rays are not H.264, they are always H.265 because you could not get the same level of quality out of the same bitrate at 4x the resolution (plus 10bit encoding).

@Phoenix_Rising

Using my Akira Anime movie as an example, MKV backs the disc up as an ISO at 8gigs.

First of all it’s MakeMKV, not just MKV. MKV is a container format, it doesn’t do anything on its own. MakeMKV is what makes an MKV (or ISO) out of the Blu-ray, but you don’t need to make an ISO. And it also takes care of only ripping the tracks you want if you configure it to do so. You don’t need to go through every rip 3 times to sort out tracks you don’t want, you can just not rip them in the first place.

@Phoenix_Rising

So when comparing the compression tests I have done, the time to compress versus quality is actually a wide gap.

As mentioned above that just means you haven’t used the right settings. I have not used HandBrake and I don’t plan to, but from what I know it has a plethora of presets out of the box and you can also set your own encoder parameters.

[zez77]

@mihawk90

That being said if you are going to transcode, and your playback devices support it, I would skip H.265 entirely because it’s a pain in the ass and just go with AV1 right away. That’s what I’ve been doing with my Blu-ray rips of House lately and I get between 50% and 70% reduction in file size depending on the episode, while still being visually lossless.

Have you/anyone converted H.264 to AV1? I’ve tried a few times and to H.265, but always lose quite a bit visually for minimal storage savings (especially with shadows and dark backgrounds). I think at this point all my devices support AV1 so I’ve been considering the switch especially for H.264 encoded 1080p Blu rays since they take up quite a bit of room. Can you suggest settings to use for Handbrake that worked for you?

[mihawk90]

@zez77

Have you/anyone converted H.264 to AV1?

@mihawk90

That’s what I’ve been doing with my Blu-ray rips of House lately and I get between 50% and 70% reduction in file size depending on the episode, while still being visually lossless.

Should add… this was actually an underestimation:

Episode 07 is an outlier because half the episode is intentionally extremely grainy and not easily compressed. I could crank up the film-grain but that could have a negative impact on the rest.

---

@zez77

Can you suggest settings to use for Handbrake that worked for you?

As I said I don’t use HandBrake. I use plain old ffmpeg, initially tested by hand and now via Tdarr. If you’re loosing quality with H.265 or AV1 at the same bitrate I honestly don’t know what settings you could be using to achieve that (because it’s impressive)…

But anyway I use these:

-c:v:0 libsvtav1 -crf 25 -preset 3 -g 120 -pix_fmt yuv420p10le -svtav1-params tune=0:film-grain=8:fast-decode=2

IIRC CRF 28 is considered visually lossless but I’m paranoid. film-grain parameter and its effectiveness very much depends on the footage, the rest is more or less boilerplate per the SVT docs.

Only downside is these settings take a hot minute to encode, but I let it run on a schedule during the night so I’m not really worried about it.

[Phoenix_Rising]

Really appreciating the feedback and discussion on this topic. Many valid points are being made, which is what I am looking for. Going back to my some of my original statements and questions:

All I am trying to do is retain the best 1 to 1 quality of my media library. If that means just copying the disc and removing the extra audio and subtitle content to reduce the final size , I’m okay with that. Storage is not a problem for me. Time is the problem with the size of the library I have.

This is what I have experimented with last night and this morning so far and I didn’t break it down well enough.

Using MakeMKV, DVDs get backed up as ISO, and Blue Ray gets backup as M2TS files.

Now I can go back into MakeMKV and deselect the non essential content and select make MKV file format.

Once that conversion is done I have all the separated chapters and content like the trailers.

At this point (My test DVD) the original Akira went from 8gig total size ISO, to just the movie at 7gig with all the extra content separated in other MKV files. This is the raw data from the media.

Now again I haven’t experimented enough with Handbreak or other alternatives to get results I am looking for with compression. I will probably do some more testing this afternoon. If I am going to see a noticeable difference compressing, like streaming compared to Blueray/4k, then I will keep the files as mkv.

[Dexter_Kane]

You may be using some weird setting in makeMKV. As the name implies makeMKV will rip the disc as an MKV file, not really sure why you’re getting ISOs.

[tk11]

@Phoenix_Rising

All I am trying to do is retain the best 1 to 1 quality of my media library

@Phoenix_Rising

Storage is not a problem for me.

If that’s the case then you’ll want to remux using MakeMKV instead of transcoding using Handbrake.

@Phoenix_Rising

If I am going to see a noticeable difference compressing, like streaming compared to Blueray/4k, then I will keep the files as mkv.

I sense some confusion here… Just to be clear mkv files can hold both remuxed and transcoded content. Remuxing to mkv using MakeMKV produces an exact copy, same as imaging a disc to ISO and same as going from ISO to mkv using MakeMKV. But using Handbrake to create an mkv will (possibly imperceptibly) always reduce video quality depending on encoding settings.

[tk11]

@Dexter_Kane

You may be using some weird setting in makeMKV. As the name implies makeMKV will rip the disc as an MKV file, not really sure why you’re getting ISOs.

I think they’re going the opposite direction and using MakeMKV to create mkv files from ISO files for the sake of player compatibility.

[Phoenix_Rising]

I just double checked it, yes you can either backup the disc as raw data (which is what I wanted to do), OR use the Make MKV format in the upper right hand of the program (which I may still do).

[Phoenix_Rising]

@tk11

I sense some confusion here… Just to be clear mkv files can hold both remuxed and transcoded content. Remuxing to mkv using MakeMKV produces an exact copy, same as imaging a disc to ISO and same as going from ISO to mkv using MakeMKV. But using Handbrake to create an mkv will (possibly imperceptibly) always reduce video quality depending on encoding settings.

You are correct. I had to double check Make MKV and by default it set to backup as raw data from the disc. You must select make MKV from the side instead of the backup choice. I was following the program steps every time a disc was loaded and it backs up the disc as raw data which is what confused me as it didn’t save as MKV. In the end, it worked out in a different way as I spoke with a former colleague that said if you backup as raw data, then you can run different tests better as opposed to going straight to MKV then Handbrake to encode. Handbreak will read the raw data and go from there based on what settings and compression you choose.

[tk11]

@Phoenix_Rising

I was following the program steps every time a disc was loaded and it backs up the disc as raw data which is what confused me as it didn’t save as MKV

That is confusing. You can still go back and open any ISO files you made with MakeMKV and then actually make MKVs if you need to.

@Phoenix_Rising

if you backup as raw data, then you can run different tests better as opposed to going straight to MKV then Handbrake to encode.

I’m not sure what “tests” you can run on an ISO vs an MKV. Either way the video and audio streams are identical.

@Phoenix_Rising

Handbreak will read the raw data and go from there based on what settings and compression you choose.

It doesn’t make any difference if Handbrake reads data from an ISO a remuxed MKV or the disc directly, you get exactly the same result. I’m still not sure what purpose Handbrake serves if you want to “retain the best 1 to 1 quality”. Handbrake is only useful if you want to sacrifice quality to reduce file size.

[mihawk90]

@Phoenix_Rising

Using MakeMKV, DVDs get backed up as ISO, and Blue Ray gets backup as M2TS files.

Now I can go back into MakeMKV and deselect the non essential content and select make MKV file format.

Once that conversion is done I have all the separated chapters and content like the trailers.

IDK what button you’re clicking but what you’re doing in the second step is exactly what MakeMKV does by default when reading a disc.

@Dexter_Kane

You may be using some weird setting in makeMKV. As the name implies makeMKV will rip the disc as an MKV file, not really sure why you’re getting ISOs.

There are 2 buttons, one is Backup for making an ISO/BDMV, the other is Save selected titles and incidentally the former is greyed out when loading an ISO or BDMV. So I’m assuming someone’s just clicking the wrong button.

@Phoenix_Rising

I had to double check Make MKV and by default it set to backup as raw data from the disc. You must select make MKV from the side instead of the backup choice. I was following the program steps every time a disc was loaded and it backs up the disc as raw data which is what confused me as it didn’t save as MKV.

Not sure what “steps” you’re referring to tho? MakeMKV has no steps. You load a disc, you click a button, done.
You can make alterations but it’s not necessary and the button is literally called “Make MKV”

@Phoenix_Rising

I spoke with a former colleague that said if you backup as raw data, then you can run different tests better as opposed to going straight to MKV then Handbrake to encode

Just as above, no idea what your colleague is talking about. Whether MakeMKV remuxes to MKV or HandBrake reads from an ISO directly, it’s the same bitstream. HandBrake has the exact same video to work on.
Arguably ISOs are worse in some scenarios, especially when you have some silent data corruption occurring somewhere, because chances are the ISO is toast. An MKV will still play and depending on where the corruption occurred it might not even make a difference.

The one advantage a full backup has is if you have a Disc that contains multiple versions of a movie (theatrical, extended, directors cut, whatever) because MakeMKV will rip these as separate files so you end up with double the size. A full backup can retain the separate versions because you can access the individual playlists, given you have a player that gives you that choice.
Maybe one day MakeMKV will support MKV Editions but I don’t see that happening anytime soon. The earliest posts I found about it on their forums are from like… 2010 or something.

@Phoenix_Rising

Handbreak will read the raw data and go from there based on what settings and compression you choose.

I think you have a massive misunderstanding of what MKV is and/or does. With a remux from MakeMKV you get the exact same “raw data”, just in a different fileformat.

[homeserver78]

@mihawk90

That’s literally all MakeMKV does and it’s not exactly hard to find.

I’m always rather sceptic against free closed-sourced software and since I don’t have a lot of discs I decided to just do things manually instead. (Maybe I should have said I haven’t found any open source program that can do it for me.)

@Phoenix_Rising

Yes I would be interested in what your process is.

As others have said you’re probably much better off using MakeMKV - the end result is the same. But maybe it will clear up some confusion if you see what’s going on behind the scenes.

The main point to understand is that the video on a Bluray disc is already (most commonly) in H.264 format. Remuxing to Matroska container (.mkv) does not in itself change the actual data, just the way it’s stored. As others have said its the same (“raw” as you put it) data as on the source disc. (Unless you transcode it using e.g. Handbrake, which will modify the content visually but can take a lot less space.)

Anyway, here’s what I’ve been doing:

• Install libaacs (required for decrypting Bluray discs).
• Download and install the device keys (DK), processing keys (PK) and host certificates (HC) required to view your bought Bluray discs (sigh). Details on this on Doom9 forums.
• Download cached disc-specific decryption keys (BD+ keys). Not necessary but speeds things up.
• Possible other things I don’t remember. Anyway, when all installation is done you should be able to mount (‘sudo mount -o ro /dev/sr0 /mnt/bluray’) and play (e.g. ‘mpv bd://longest --bluray-device=/mnt/bluray’) an inserted Bluray disc. (If I don’t pre-download cached BD+ keys this command “hangs” for like 3+ minutes the first time for some reason. Once the disc decryption key has been calculated and cached it’s instant, as it should be.)

Then for each disc:

• The mpv command above prints a list of playlists. You probably want the longest one. Example:

mpv bd://longest --bluray-device=/mnt/bluray
[bd] List of available titles:
[bd] idx:   0 duration: 00:00:14 (playlist: 00100.mpls)
[bd] idx:   1 duration: 00:00:06 (playlist: 00106.mpls)
[bd] idx:   2 duration: 00:00:17 (playlist: 00107.mpls)
[bd] idx:   3 duration: 00:00:06 (playlist: 00108.mpls)
[bd] idx:   4 duration: 00:00:06 (playlist: 00109.mpls)
[bd] idx:   5 duration: 00:00:06 (playlist: 00111.mpls)
[bd] idx:   6 duration: 00:00:06 (playlist: 00112.mpls)
[bd] idx:   7 duration: 00:00:06 (playlist: 00113.mpls)
[bd] idx:   8 duration: 00:00:06 (playlist: 00114.mpls)
[bd] idx:   9 duration: 00:00:06 (playlist: 00115.mpls)
[bd] idx:  10 duration: 02:41:28 (playlist: 00001.mpls)
[bd] idx:  11 duration: 00:00:06 (playlist: 00119.mpls)
[bd] idx:  12 duration: 00:00:17 (playlist: 00120.mpls)
[bd] idx:  13 duration: 00:00:06 (playlist: 00121.mpls)
[bd] idx:  14 duration: 00:00:06 (playlist: 00122.mpls)
[bd] idx:  15 duration: 00:00:06 (playlist: 00124.mpls)
[bd] idx:  16 duration: 00:00:06 (playlist: 00125.mpls)
[bd] idx:  17 duration: 00:00:06 (playlist: 00126.mpls)
[bd] idx:  18 duration: 00:00:06 (playlist: 00127.mpls)
[bd] idx:  19 duration: 00:00:06 (playlist: 00128.mpls)
[bd] idx:  20 duration: 00:00:06 (playlist: 00130.mpls)
[bd] idx:  21 duration: 00:00:06 (playlist: 00131.mpls)
[bd] idx:  22 duration: 00:00:06 (playlist: 00132.mpls)
[bd] idx:  23 duration: 00:00:06 (playlist: 00133.mpls)
[bd] idx:  24 duration: 00:00:06 (playlist: 00134.mpls)
[bd] idx:  25 duration: 00:00:06 (playlist: 00135.mpls)
[bd] idx:  26 duration: 00:00:06 (playlist: 00136.mpls)
[bd] idx:  27 duration: 00:00:06 (playlist: 00137.mpls)
[bd] idx:  28 duration: 00:00:06 (playlist: 00138.mpls)
[bd] idx:  29 duration: 00:00:06 (playlist: 00139.mpls)
[bd] idx:  30 duration: 00:00:06 (playlist: 00140.mpls)
/.../

Here the playlist you want is number 00001.
mpv also prints the available formats in the chosen (“longest”) playlist:

 (+) Video --vid=1 (h264 1920x1080 23.976fps)
 (+) Audio --aid=1 --alang=eng (dts 6ch 48000Hz)
     Audio --aid=2 --alang=ita (dts 6ch 48000Hz)
     Audio --aid=3 --alang=spa (dts 6ch 48000Hz)
     Subs  --sid=1 --slang=eng (hdmv_pgs_subtitle)
     Subs  --sid=2 --slang=eng (hdmv_pgs_subtitle)
     Subs  --sid=3 --slang=ita (hdmv_pgs_subtitle)
     Subs  --sid=4 --slang=spa (hdmv_pgs_subtitle)
     Subs  --sid=5 --slang=dan (hdmv_pgs_subtitle)
     Subs  --sid=6 --slang=fin (hdmv_pgs_subtitle)
     Subs  --sid=7 --slang=hin (hdmv_pgs_subtitle)
     Subs  --sid=8 --slang=nor (hdmv_pgs_subtitle)
     Subs  --sid=9 --slang=por (hdmv_pgs_subtitle)
     Subs  --sid=10 --slang=swe (hdmv_pgs_subtitle)

Make sure sure the chosen playlist contains the format you want, e.g. sometimes there are two equally long features one of which is 720p. For me, mpv has always picked the best one so far!

I then use ffmpeg to rip/remux. Here I want the english audio and subs and also the swedish subs:

$ ffmpeg -analyzeduration 1G -probesize 1G -fix_sub_duration -playlist 1 -i bluray:/mnt/bluray \
-map 0:v:0 -codec copy -metadata title="<Fill in movie title here>" \
-map 0:a:0 -metadata:s:a:0 language=eng -metadata:s:a:0 title="English 5.1 DTS-HD MA" \
-map 0:s:0 -metadata:s:s:0 language=eng -metadata:s:s:0 title="English 1 (Bluray PGS)" \
-map 0:s:1 -metadata:s:s:1 language=eng -metadata:s:s:1 title="English 2 (Bluray PGS)" \
-map 0:s:9 -metadata:s:s:2 language=swe -metadata:s:s:2 title="Svenska (Bluray PGS)" \
movie-file.mkv

The -map options are what actually selects the bitstreams. The -metadata options fills in some basic metadata so that the player program can select the right streams on playback later. (I have not been able to figure out how to rip chapter info though.)

The rip then takes perhaps 20 minutes (with my Bluray drive).