Help with Peertube setup on a Linux box

Operating Systems & Open Source Linux
1

Hi all I have been spending this past week trying to find an answer to a question about setting up a Peertube instance and have not discovered any good starting points. This is why I am here hoping to at least get some help or a nudge in the right direction before I go and create this server to be self hosted.

So before I start, yes I know things about Linux, I would call myself under the immediate level when it comes to the process. I am running an Unraid server for NAS items and have an Ubuntu box to host a test bench version of ArchivesSpace. Since this installation is going to be slightly out of the box and that I am not 100% sure what the correct terms are it is making this scenario hard to search for. Any help would be great.

What I am trying to do: create an instance of PeerTube so that I can embed videos to my website. (https://www.theunofficialconventionarchive.com), currently I am using both the Internet Archive and LBRY for this purpose, however due to some recent information about LBRY I am looking for a more concrete option. My idea was to take my old computer, throw Linux on it, have it dedicated to peertube and ngnix, and tada problem solved. However while I am planning this I am coming across a few concerns.

  1. I have almost 1900 videos spanning about 13 TB that I need to house, this is not including future development of creating these videos. My idea was to let PeerTube to look at the server, get the file locally, and then be able to serve it up on the instance. From all the documentation this does not appear to be an option. So the first question would be if anyone who has worked with PeerTube was able to basically use a NAS for storage and have the videos working on the instance, and if so point me in the direction to figure it out.

  2. If I need to store it locally on the instance, fine. I will need to get a massive drive to store it on the instance. Currently have an M.2 and a small SSD in the machine. In the windows world I would map a drive just for the files and install the OS and I am off to the races. In reading the instructions it does not appear I can do that. Am I missing something obvious that would allow me to do this?

  3. I would like to be forward thinking and be able to eventually move this instance to another location if grants were awarded. Meaning I would like the root and boot to be on the M.2, the application on the SSD, and the Data on a large HDD. Is there a way I can set this up, or a tool I can use to tell Linux where things go and have it play nice with one another?

  4. Since I am dealing with videos I know I am going to be trans-coding a TON (this is why I could not go with Linode for this, $1000/month minimum is way too much for my budget right now) I know that PeerTube does allow transcoding once you upload a file to the instance however I have been unable to figure out how it does it. I can throw a NVIDIA card in there and have the Cuda to help speed the process but it appears it is just CPU bounded. Does anyone have the information on the process that PeerTube is using so that I can understand a bit more.

  1. If I were to simply apply handbrake or ffmpeg to these videos anyone know the render formats to make them appear in the correct formats?

Sorry for the lengthy post hoping to get started with this project in the new year and want to make sure all my ducks are in a row before creating Frankenstein’s monster. Any help or tips or places I need to go to learn more would be grateful. Thank you for taking the time to read this I hope you have a great day.

2

I don’t know much about peer tube , but YouTube transcodes and stores videos ahead of time into several most popular formats.

Turns out if your originals are in 4k, then 1080p+ most other lower resolutions typically only double your storage requirements, and you need no extra CPU.

Linux doesn’t care about applications vs data vs os. They’re all just files and you as the admin get to choose what filesystem you want mounted where.

You also get to pick and choose if you want e.g. your ssd to act as your read/write cache in front of your HDD… you get a funny looking block device with some interesting performance properties for your filesystem to love on.

You can also marry multiple block devices into various raid like setups in various ways. e.g. on a block level like traditional raid, or using a filesystem like btrfs and zfs, or just by pooling storage using mergerfs+snapraid… these have varying degrees of flexibility… there’s so many options, it can get really confusing.

I don’t know much about peer tube, but I thought the idea was you could attach multiple physical storage servers as backends to boost your bandwidth and availability and the light weight frontends would guide the viewers through redirects to these backends.

Are you planning on putting all your videos in one place / on one server?

1 Like
3

From reading the technical documentation the files need to somehow live on the instance server. I am still unable to get an answer if I could just map my NAS to this device and allow it to do the heavy lifting so I managed to get a Boxing day deal on an 18 TB HD (which is only 16.75 TB which I am annoyed it but will roll with it going forward).

The idea is I will basically copy the file to the instance via an “upload” and it will live there while the NAS is going to be the storage device for long term storage. Sadly having the same file over multiple machines is going to cause some headaches for me so going to rework some metadata server configuration so that I can keep track of all their locations.

4

Yes!

One way or another for sure.

Form glancing around the “production guide”, the docs don’t seem super clear on it, but it’s Linux so you can mount whatever network filesystem to wherever/whatever path on your server.

There’s some guides for how to run it in docker using docker-compose out there, that would probably be the direction I’d be looking in.

Usually with docker install methods, the dependencies (incl storage and persistence) are described more clearly.

5

I was going down that route until Docker on my one box decided to implode (I am still trying to fix that issue -_-) thankfully I have an old spare computer that I am going to be using for this. One bummer is that in order to both have the material on the LBRY and Peertube I would need to double my data since one wants HLA sharing while the other is just bit torrent. I may just see if I can “host” the LBRY connection on the NAS and take care of it that way but that is a project for another time. I can only do so much at once.

6

An Update: it took a few days but I got it installed. Made sure Duck DNS is running, managed to get certbot to issue me a certificate, made sure that the DNS records were pointing to the correct address and…still having some issues.

It appears I have a default nginx page on display (meaning all the DNS records and port forwarding is working as intended). However I was hoping to see a peertube page and not the default. Anyone with some nginx experience point me in the right direction where to look to make sure I got everything right?

If you are reading this thread in the future two great sources I used to get to this point are the following:

https://docs.joinpeertube.org/install-any-os

Update2: Config is now working, however site is still not loading checking the journalctl for peertube and noticing I am getting a fatal code 28P01 error. Google tells that that is postgres based where the password is incorrect. I checked the config file again and it is matching, just to be on the safe side went into psql and altered the password for the user to make sure it is matching. Started the service back up again…same error. Anyone have any ideas?

7

I’m kinda curious what you mean by this. LBRY shouldn’t be going anywhere, as long as you (or others) are hosting and seeding the files. But I do agree PeerTube is a better option in many regards.

I never used PeerTube, but I have experience in system administrating a lot of linux applications.

For problem 1., you said you have a NAS running Unraid. Easiest option would be to use NFS and connect your PeerTube instance to the NAS. But depending on the amount of traffic you expect to see, it’s probably going to be network intensive. Judging by the number and size of your video collection, I would guess you have a pretty big following, so if you haven’t already, you should start looking to invest in 10G network gear.

Unlike in the windows world, on Linux the mounting is transparent to any programs you are running, so while you may have some delay if access speed is an issue, the program itself, in this case PeerTube, shouldn’t care if you are storing your files on a single drive, a RAID, an LVM, or in a network storage location.

For 2., you don’t need to store things locally, just “local enough” on your network for the videos to be almost instantly accessible by your PeerTube instance. I’m not sure what you are using your Unraid NAS for at this moment, but if you are using SMB / samba to mount a network path on windows machines, you can do the same thing with Linux. By using a samba client, you can mount the SMB location on your PeerTube instance. But I would recommend you run NFS instead, even if it means you’d be running SMB and NFS in parallel. NFS, from experience, pretty much always performs better than SMB. If your requirements weren’t this big, I would have suggested sshfs, but it can be quite slow compared to even SMB. So NFS would be your best bet IMO.

I do not understand what you mean by 3. If you mean having the OS install on one partition (/ and /boot) on the m.2, the PeerTube instance on a SATA SSD and the data on a HDD, I think you need to look a bit on how the Linux File System Hierarchy works. If you install PeerTube outside of your package manager, you need to set /opt on your SATA SSD and install PeerTube there. Otherwise, if you use your package manager, like say adding a PPA or a repo, you need your /usr/bin partition on the SATA SSD. But in either case, just moving the partition around means nothing if you don’t keep your OS with it. So your program is basically bound to the OS you are using, if you reinstall the OS, but opt to keep using your old /usr/bin, you might be in for a big surprise of your program not working, mostly due to things like variables and libraries.

So the best alternative is to keep a backup of your instance’s data files (user credentials db, file uploads, descriptions etc.).

I have no answer on 4. I am unsure if PeerTube uses ffmpeg (most likely), so you can use either CPU or GPU to transcode. What you can use Linode for is to host your PeerTube instance itself, then connect your NAS to it. Basically having your files in your local infrastructure for cheap storage, but using Linode as an internet facing server. But if your connection from home to the PeerTube / Linode instance is flaky, your users could have a really bad experience.

No idea about what format PeerTube uses, so no answer for 5. either.

You can trick PeerTube or any other application to think a NAS is its local storage. All you have to do is find the location where peertube videos are uploaded to and mound a network path there. It could be somewhere like /var/lib/peertube or similar. Stop peertube from running, copy the files from its upload location to your NAS in whatever path you want, then mount that NAS server path into the location peertube wants its videos and you should be good to go.

But at the size of this infrastructure, I do hope you have a valid backup of your data. And RAID is not a backup, so your NAS alone is insufficient to keep your data safe.

Regarding the db, is the postgres user set up correctly in the peertube config? No real idea what could go wrong in this case.

As for nginx, in the nginx server conf, you have to point your vhost (I forget what it’s called in nginx) of your main domain to the peertube port, instead of the nginx default page. Should be pretty easy, the config is in /etc.

8

Can you log in to with peertubes credentials?

Are you using localhost, a Unix socket, or one of the non localhost IPs to connect?

9

I agree LBRY is not going anywhere but the majority of the users I know on the platform are either CryptoBros who DGAF about what they are actually doing and are holding the coin until they cash out, or people like me who actually use the platform to upload videos to. With the Odysee chaos and the impending split of rewards no longer being issued out to those who use the platform it is going to start to slow down in general use.

I have enough spared up to last me a while (around 200 LBC or 2000 video uploads) however in projecting where this project is going I either need to jump through several hoops and somehow work the crypto market to get more LBC, find people who are interested in what I am doing and have them jump through those same hoops, or start working on a more controlled platform. Hence me trying to take an old PC I have and throw PeerTube on it.

1 Like
10

Right now I am trying to access it via the domain on a separate computer (kinda like I would do in the real world) I have not checked locally yet since the machine is a bit of away from my main battle station. Might give that a go when I get back from my small tour of duty.

I am thinking that it is something with the user’s password not being set correctly due to a typo I did initially, going to play around with that and hope to see if that isn’t the case, otherwise I can always create another user right laughs nervously

11

While rare, also keep in mind some software do not like certain special characters, like \ or $. I don’t know if that’s the case for PeerTube, but be warned anyway. If you generate passwords (which you should), just try to use mostly alphanumeric characters and just a few special characters that usually don’t cause issues, like dots and commas (and a few others).

12

The good news is that the password is not anything super special so no need to worry about the special characters, what I think could be the cause is when I created the user I entered in their information without a password (had a brain fart) then created the database and associated it with them.

I realized my mistake and added the password in the config and the user thinking that should take care of itself. In my limited knowledge this SHOULD be good but the error is telling me otherwise from a file called auth.c. From a quick search it does not appear that this file actually exists so I am trying to scramble using the tutorials mentioned above to see if I was missing anything and so far have not found anything.