I decided to post this as I am sure I am not the only person wondering about trying a server with multiple GPU’s for transcoding. From my experimentation, each GPU used must have its own Emby Server instance/VM.
Over the last 3 months I have been experimenting with Linux, Emby and GPU passthrough. I have tried many distros and configurations. I started with Arch linux to cut my teeth on GPU passthrough as the documentation is excellent. My adventures have proven very educational. It does seem like every BLOG and post out there is usually missing at least one thing if you are doing a ‘from scratch’ set up. Here is what I have currently settled on as my ‘production’ system (yes, using Ubuntu server). This can be done using Arch, Fedora, CentOS … however, I found this to be stable (for my use-case). I have also tried Proxmox and ESXi server. I was able to set this system up with Proxmox… EXCEPT, I ran into the AMD GPU reset issue in the VM that is pretty universal across ALL disto’s and configurations (from my experience). Therefore, I settled on the Base VM system also being an Emby server running on Ubuntu 18.04 and using the AMD GPU for video AMD GPU acceleration in Emby Transcoding. This works great as in the event you need to reset the AMD-based transcoding server … you won’t run into the GPU hanging issue since the entire server will be rebooted. Below is my hardware I am using. I began this experiment with an I7 4790K, 16GB DDR3 and ASUS Z97-A motherboard. That configuration worked, but when I decided I wanted to run ALL of the GPU’s in ONE system, I found that I was running out of processors to dedicate to VMs, hence I went with the 8700K and NOT the 9700K (12 threads vs 8 threads) – another bonus to the processor upgrade is the addition of hardware HEVC enconding/decoding that the older 4790K didn’t have.
By heart I am a ‘bang-for-the-buck’ kinda person, so EVERY other computer/server I have are AMD-based. But as a technologist and realist, I realized, if I wanted a Multi-method transcoding server in one-box… I needed to go with Intel (at least until they release their GPU in the coming year(s).). I chose the ATX motherboard platform due to the number of full-size PCIe slots (even though not all of them are 16x slots. The motherboard I selected has 3 full-size PCIe slots, and I configured them as 8x/4x/4x. I have the AMD RX 570 in the 8x slot, the MSI GTX 1050Ti in a 4x slot and an Intel PRO/1000 quad port card in the final 4x PCIe slot. In my VMs I assign each VM its own NIC port on the Intel quad-port card.
You may be wondering; “Where are the media files stored?” … well, I have a FreeNAS server (ZFS RaidZ2) that holds all the media and the Emby Server configuration backup files. Once I figured out how to use the Backup Configuration plugin … my experimentation became so much easier. Purchasing my Lifetime Emby Premiere License has been invaluable (No, I am not affiliated with Emby – other than being a user).
Multi-method transcoding Server
Intel I7 8700K (6core/12thread) w/UHD 630 graphics
Gigabyte Z390 UD motherboard
G.Skill Ripjaws V 16GB (2x8GB) DDR4-3200
Intel PRO/1000 PTQuadPort (PCIe 4x)
MSI GTX 1050Ti Gaming X 4GB
MSI RX 570 Armor 4GB OC
Inland Profressional 120GB SSD (SATA) - Boot/OS
WD 1TB HDD (SATA) - VM & Storage
Base OS : Ubuntu Server 18.04.3
Add-ons : Libvirt, QEMU, OVMF, Cockpit Web Management, Emby-Media server (AMD-based transcoding)
VM1 (Intel-based Transcoding)
OS : Ubuntu Server 18.04.3
CPU : 2
Memory : 2GB
VM2 (NVIDIA-based Transcoding)
OS : Ubuntu Server 18.04.3
CPU : 2
Memory : 2GB
VM3 (Software Transcoding)
OS : Ubuntu Server 18.04.3
CPU : 4
Memory : 4GB
You will notice that the VMs that use hardware accelerated transcoding only have two (2) CPU’s assigned… as the GPU does all the heavy lifting and you don’t need a bunch of CPU power. I have tested this using using my cell phone to ‘force’ the GPU to transcode - I have noticed on occasion if the source is HEVC it defaults to software transcoding for some unexplicable reason.
I have been able to re-build this system numerous times with full Transcoding functionality. FYI, I am using the Ubuntu drivers for ALL GPUS (Intel i915, NVIDIA and AMD)… if you are interested, I can make another post with links to the sources I used.
I will also be testing AMD-based Hardware accelerated transcoding using the AMD Ryzen 5-based APU (R5-3400G).
I have an AMD-based test-server that I will be using to do the same as above, but without the Intel-based transcoding option).
For those interested… my File(‘Media’) server is:
AMD Ryzen 5 3400G (4core/8thread) w/Vega 11 graphics
ASRock Fatal1ty B450 Gaming-ITX/ac motherboard
G.Skill Aegis 16GB (2x8GB) DDR4-3000
LSI SAS9211-8i 8-port 6GB SATA+SAS PCIe2.0
2x Mini-SAS to SATA Cable (SFF-8087 to SATA Forward Breakout) - SAS to 4 SATA connectors
6x Hitachi HGST Ultrastar 7k4000 3TB 7200RPM SATA III
*NOTE: Storage is ZFS Raid-Z2 (2 drive fault) - 10TB available storage.
I hope this is beneficial information.
HiTekAgPilot