Hey Guys,
Looking for some advice/sanity-checking here.
Ive setup a proxmox server for 2 reasons, host media and act as a google photo alternative
For my os and containers I have a 1TB nvme.
For movies and shows I have a 4TB drive zfs
Then for the photo drive I have 2x 4tb in mirror-0 zfs.
(I also have another 1TB nvme if there is any use for it.)
My main question is how I should set up the storage for these applications. For media I’m planning to use plex&Jellyfin and for the Photos I’m going to use photo prism. I originally planned to have the photoprism folder on the container and the same with my jellyfin/plex server but now I don’t know if its better to have share in proxmox itself or setup a samaba container?
As a side note, I set up an lxc with OpenVPN running PIA with some iptable rules to make sure nothing goes out to the internet unless it’s on the VPN but now I’m wondering is it a better option to use openwrt with PIA on its own linux bridge and route my contains through this.
I know its a lot but Im hoping someone has good plan of attack for me! Thanks in advance!
A little surprised no one replied to this…
Firstly, you are going to wreck that 1TB NVMe, so add the spare as a ZFS mirror.
I would lean towards running your apps as docker containers within a VM, use disk images for containers with small storage requiments. I tried LXCs, but mostly run apps in docker containers. LXCs have their uses, juat not as application containers.
For larger storage for media look into NFS, 9pfs or virtio-fs shared to your docker VMs.
Oh and, I beleive in keeping VPNs on the Firewall/gateway using pfsense or opnsense.
1 Like
Raw Storage:
Proxmox is a poor file hosting solution. I personally try to avoid adding more services to the hypervisor. I would use Proxmox to manage the raw ZFS storage.
Container:
I don’t know why the dislike for LCX container for apps. They are lightweight and might be better than adding just another layer of VM + docker images. Accessing smb shares might be a bit more involved with certain LCX containers. (like jellyfin which has no option to mount smb inside its own interface).
Media Storage:
The easiest solution would be to use a VM, add a virtual disks as storage and share it via smb. You can also select the location of the virtual disks (if you need ssd speed for certain files). For a data grave this is fine enough.
I use debian for this on my workstation. You can also add cockpit to get a Webinterface for this Linux distro + plugin to manage samba and other Linux services.
Media storage connection:
I use a second network bridge without physical ports to link the container / VMs for file sharing. This means should something happen to the external network I can still access the data within the same server / workstation. If you are not physically accessing the same machine this might not be necesarry.
Advanced solution:
On my Media-Server I use Proxmox + TrueNAS + Passed through SATA controllers. With TrueNAS Scale you can also run most of the file hosting solutions (jellyfin etc.) as native applications in one package. I share data / storage back to Proxmox via SMB and iscsi.
1 Like
Hi @kingveder, I am not a fan of containers. The reason is people never check if their containers have security holes or if the dependencies of the containers have security holes. Also, I never understood the benefits of containers over virtual machines.
Never use a Docker container without doing this first! (And don’t create one either!
One thing that I’m surprised no one has pointed out yet, is that your storage has no redundancy. Each to their own but running ZFS without using mirrors or RAIDz?.. I just hope the striped mirror and the 4TB disk vdevs are at least in separate pools.
As for how to share data to your LXC containers. With my personal setup, I use LXC Bind Mount Points. If you’re at all familiar with TrueNAS, then LXC Bind Mounts are broadly similar to Jail storage mounts (whatever they’re called… if TrueNAS still even uses Jails!). They can be used to add storage to an LXC container (even an unprivileged one), without the overhead and added complexity of something like SAMBA, NFS, or some people even pass disks through to a VM, to then share the data to their containers.
Specifically in regards to the question of whether the sharing should be done form Proxmox or a Container though, if you’re set on the SAMBA or NFS route, then I’d recommend a Container as I just think it’s best practice to leave the Proxmox install as clean as possible. Others may disagree though.
As with anything, there are pros an cons to using Bind Mounts. Many of which are shared with the Containers vs. VMs debate but for my personal use case, they work just fine.
Bind Mounts can be created through the Proxmox WebGUI but as I always do it manually, I can’t tell you how to do it the WebGUI way. The manual method is a little complex but easy enough once you understand it… it’s basically editing the Container’s .conf file to include the mount point. Then the rest of the setup is permissions related. Let me know if you want more info.
1 Like