Video Encoding to AV1 Guide (WIP)

The official docker is pretty easy to use. I use it for truenas since I cant really maintain all this stuff on a system that doesnt let you do more than what it is intended for.

Av1an Docker

Why you unlink? Link the docker lol. Its probably their preferred way of packaging

I hate formatting posts. I was going to fix the link.

1 Like

As Far as I can tell only in niche cases do people run the docker. If they cant get it up and running with compiling from source. I am in their discord and that seems to be their go to. Bare metal.

1 Like

Hmm thank you for the link imma run over to talk to my friend about how I can push this into a kubernetes cluster and distribute the load over way more cores than one ayyymd cpu

1 Like

Bare metal will ALWAYS be faster yes and thats probably why

There are arguments against docker that are legitimate .. It does duplicate your libraries on your system with different versions of said libraries. It is bloaty. It has issues but it has some strengths too

1 Like

Which makes sense. Proxmox generally says you lose ~5% of raw performance with a given containerized application, that is LXC of course which I do run a av1an instance in lxc.

In total I have 6 computers running encodes, 2 x truenas, htpc, guest pc, proxmox, my primary desktop and maybe for giggles my mini pc I bought and still waiting on. it is 8 cores of zen4 goodness.

1 Like

There is also a benefit to compressing audio too. Opus is hard to discern a difference at 300kbps vs DTS source at 3-4mbps. It cuts the overall file size down to less than 1GB in many cases. An example is an episode of Person of Interest averages 1.0 to 1.2GB with both audio and video compression. I am seeing 4.1Mbps for audio and video combined. That is significantly less than original bluray source it is from. Throw in nvidia’s video super resolution, you have fidelity nearing 4k at 1.2gb file sizes.

How do you map the surround sound properly?

It can be buggy with av1an doing it. What I do is transcode the audio in a separate process with ffmpeg.

It is still a wip. But I use this in a for loop after converting the video…

ffmpeg -i "$x" -map_chapters 1  -map 0:v:0 -map 0:a:0 -c:v copy -mapping_family 1 -ac 6 -c:a:0 libopus -metadata:s:a:0 title='Opus 5.1' -metadata:s:a:0 language='eng' -map 1:s -c:s copy "${x%.*}.final.mkv"

-mapping_family 1 signifies more than stereo and -ac 6 signifies 6 channels. It has done well from what I can tell.

I only recently started compressing audio, like a month ago. Up until then all I compressed was the video.

@Argone
I just noticed that all your used software is properly packaged in FreeBSD, Vapoursynth is a bit dated but that’s fixable. The framework also flexible so you can optimize all you want =)

As far as sound it’s bit more to it than just transcoding. If you’re not going to keep the original audio track you might as well downmix it do 2ch instead. DRC is a thing that don’t translate into Opus, you also have the issue of not being able to bitstream which many receivers benefits from compared to sending (L)PCM. The being said, I would say that AAC/xHE-AAC is the way to go on sound simply because of support both in software and hardware especially if you want to rip it out for whatever reason later on. As far as sound quality goes it’s pretty much a dead race if you use decent encoders (fhg-aac or Exhale).

2 Likes

I may delve into FreeBSD later on. But for now. I am sticking with linux, windows and mac.

1 Like

What you also might want to consider if you’re concerned about performance is looking into LLVM/Clang is it’s quite common that it produces faster binaries however there are instances where GCC is faster.
See LLVM Clang 16 vs. GCC 13 Compiler Performance On AMD 4th Gen EPYC "Genoa" - Phoronix for example

1 Like

If you are interested. This is my exact bash file i run and it loops my source file directory and fixes audio bugs that may come up. It is a one step does all bash file.

The bellow bash file is my way of automating it mostly. You have an input directory and a output. The input is for source file that is encoded to av1. The output contains the av1 encoded file, ripped or transcoded audio from ffmpeg and original source file. and a final file.

Directory Pathes
transcode/input/source.mkv
transcode/output/av1an/av1file.mkv
transcode/output/ffmpeg/mkaudio (original or transcoded).mka # From input directory
transcode/output/final/final file with correct audio.mkv
#!/bin/bash

# replace with path to your input directory
cd /home/argone/00-Local/work2/transcode/input/

for x in *.mkv

do

# extracts time codes for original audio (needed for audio sync correction
# input.vpy is generated bellow
touch input.vpy
cat <<EOT >> input.vpy
from vapoursynth import core
clip = core.ffms2.Source(source='$x', threads=32)
clip.set_output()
EOT
vspipe input.vpy -t "$x.dat" .

# av1 encoding tool that chunks and encodes video
av1an -v "--cq-level=22 --end-usage=q --cpu-used=3 \
--enable-fwd-kf=1 --aq-mode=1 --lag-in-frames=48 \
--bit-depth=10 --kf-max-dist=240 --kf-min-dist=12 \
--enable-qm=1 --sb-size=64 --enable-keyframe-filtering=2 \
--arnr-strength=1 --arnr-maxframes=3 \
--deltaq-mode=0 --sharpness=1 --enable-dnl-denoising=0 \
--denoise-noise-level=5 --tile-columns=1 --threads=2" \
--photon-noise 10 -e aom -m lsmash -w 16 \
--concat mkvmerge --resume --verbose \
-i "$x" -o "../output/av1an/$x"

# compresses audio to opus.
ffmpeg -i "$x" -map 0:a -mapping_family 1 -ac 6 \
-c:a:0 libopus -metadata:s:a:0 title='Opus 5.1' \
-metadata:s:a:0 language='eng' \
"../output/ffmpeg/${x%.*}.mka"

# extract audio, use only if you want original audio untouched
ffmpeg -i "$x" -map 0:a -c:a:0 copy \
"../output/ffmpeg/${x%.*}.mka"

# muxes audio + video stream, audio timestamps, fixes metadata
mkvmerge --output "../output/final/$x" \
--no-video --no-subtitles --no-chapters "../output/ffmpeg/${x%.*}.mka" \
--no-audio --timestamps 0:"$x.dat" "../output/av1an/$x"

rm input.vpy

done
1 Like

Thank you im going to bookmark this

1 Like

Something cool if you use browser of course and have nvidia you can upscale to damn near 2160p. I have found it also works with amd gpus or any gpu that is powerful enough. But jellyfin mpv shim supports dlss and fidelity fx super resolution. So a lot you can work with and looks damn good. I am happy getting av1 down to 1gb plus or minus a few hundred megabytes. And look clean and crispy on my 4k tv and projector. I am not sure if i can take a photo and do it justice for how good it looks.

I am trying to get a windows equivalent to the bash i shared with you, the problem is I cant get

from vapoursynth import core
clip = core.ffms2.Source(source='$x', threads=32)
clip.set_output()

to echo or cat to input.vpy

any ideas?

1 Like

I got it working in windows kind of…

@echo off

F:

cd F:\share\input

for %%x in (*.mkv) do (

echo from vapoursynth import core> line1
echo clip = core.ffms2.Source(source='%%x', threads=32^)> line2
echo clip.set_output(^)> line3
type line1 line2 line3 > input.vpy
vspipe input.vpy -t "%%x.dat" .

av1an -v "--cq-level=28 --end-usage=q --cpu-used=3 --enable-fwd-kf=1 --aq-mode=1 --lag-in-frames=48 --bit-depth=10 --kf-max-dist=240 --kf-min-dist=12 --enable-qm=1 --sb-size=64 --enable-keyframe-filtering=2 --arnr-strength=1 --arnr-maxframes=3 --deltaq-mode=0 --sharpness=1 --enable-dnl-denoising=0 --denoise-noise-level=5 --tile-columns=1 --threads=2" --photon-noise 10 -e aom -m lsmash -w 16 --set-thread-affinity 2 --concat mkvmerge --resume --verbose -i "%%x" -o "F:\share\output\%%~nx.mkv"

ffmpeg -i "%%x" -map 0:a:0 -mapping_family 1 -ac 6 -c:a:0 libopus -metadata:s:a:0 language='eng' "F:\share\output\ffmpeg\%%~nx.mkv.mka"

mkvmerge --output "../output/final/%%x" --no-video --no-subtitles --no-chapters "../output/ffmpeg/%%~nx.mkv.mka" --no-audio --timestamps 0:"%%x.dat" "../output/av1an/%%x"

)

pause

It keeps skipping the first file episode 1, and goes on to 2 and 2 is correct but 1 wont do the vspipe part. I dont get it.

Oh it works when you do vspipe before anything. Since it uses source file it can happen before av1an and ffmpeg part.

Batch wont do this. Look into powershell or python

I did end up getting it.

It kept leaving out a parenthese until i found out that a carrot is needed.