I have seen the adverts for Linode in the latest videos and its looking quite useful. I still feel like I don’t fully comprehend all the possibilities of what I can do with this?
Is it possible to use Linode as a metaphorical caching server? Can I download linux isos and steam titles to the linode server and then push them directly to my PC? I seem to struggle getting above 40-60 megabytes per second downloads on steam and origin at home even though I’m running a gigabit connection. Didn’t know if this was possible or not really ideal for this service.
Aside from this I am data hoarding about 40 terabytes of data at home currently and I have an emby server on my main desktop. I have a second desktop which is running an i7-3570k 16gb ddr3 and a gtx 1080. I was wondering what is the best way to implement a NAS and get a second backup of my storage utilizing more Local HDDs or the best way to make all my data be stored on my backup PC instead of my main PC.
I have a 5 gigabit LAN connection router, but do i need to get a network switch, is 10 gigabit LAN worth it? What’s necessary for 4k streaming to WAN or LAN? Do i need ECC ram and a used Xeon from Ebay? What’s the best way to setup this NAS for outside access and the ability to communicate with my other devices in a secure and safe way?
I’ve looked around and seen Synology but I REALLY want to DIY this thing myself. I know a lot about hardware but the software and setting up of this NAS is whats intimidating me. I also want to know if its possible to SSD cache or RAM Cache this machine in a way that would allow me to physically download large 80gb files in a 60 second to 100 second time frame. What can I do to speed up the nightly backups or etc?
P.S. I’ve heard of Pfsense and Wendell is always talking about the benefits of linux. Should my NAS run linux and should i make a 3rd box manually to be a Pfsense firewall and can I incorporate Pi Hole into that 3rd box as well to make a network wide adblocker to protect my families risky click habits?
Certainly possible, although not ideal for a number of reasons. It is much better to do this with a machine on your local network.
You will need 10gpbs networking at a minimum for this. You will also need NVME, a ramdisk, or raid sata ssds for the disks.
Is the downloading from the internet, or downloading from the NAS?
It depends on what you mean exactly by access. Nextcloud is good for some needs. Syncthing is great if you just want sync. Nginx+Letsencrypt can make a simple HTTPs server with basic directory listing of files.
SMB, NFS, etc are all not meant to go over the internet.
SSH with keypairs then port forward? Or do you mean RDP type access?
Might be a good way to stay in budget, since used server hardware gets cheap if you go 4ish+ gens back.
You don’t have to set everything up all at once. Remote access, caching, etc can wait till the basic NAS functionality is done and working.
I doubt you will get much value from a caching server that is outside your network, especially if you are not caching for at least 10 computers locally. There are bandwidth costs for sending and receiving data in a datacenter.
You should probably do an offsite backup of important files. Maybe run something like a seafile, syncthing or nextcloud.
That pfsense box should be running on your local network. I dont have enough experience to suggest other stuff for offsite/cloud.
Then I will not do this. I thought it might be a little far fetched and not necessarily what a linode server would be for.
I am looking for downloading from the NAS in this use case. I know you can optane cache, ssd cache, raid, and etc… I was wondering if DDR3 or DDR4 was getting cheap enough that making a 256gb or 512gb ram cache would be worth it. Or utilizing a bunch of 256gb Samsung 860s I have lying around from work. Which leads me to my next question. If I have a 40tb server of documents, photos, videos, music and other media I wish to have offsite access to (obviously at much slower speeds) but LAN access to at 10gbps what’s the best way? If necessary I could settle for just 10gbps physical transfer speeds via usb 3.2 gen 2 or 3.2 gen 2 2x2. At the moment my current problem is my spinning rust only goes at 200 mbps and i haven’t raid’d anything. My budget is 1000$ starting and up to 2000$ as time allows. Obviously I am anticipating a new board/cpu being needed and can go higher if its ideal/recommended.
Well what’s the best way for me to get access to my documents, music, and photos remotely? Like if I need a copy of a jury summons or a UPS return label off my desktop or server. Or forgot to grab my new CD before leaving the house? Situations like that and if necessary starting up Emby server remotely. RDP at this time would be easy stuff like that and maybe more serious once I get more into coding and crunching simulations as this system ages.
Both if possible? Currently I am using Chrome’s remote desktop program and I am not sure how bad or security vulnerable it is but if its a threat to privacy and security id love to stop using it.
Is there server hardware that isn’t quite so loud? I would unfortunately not have a closet to hide this thing in for several years until I move into a new house/apartment. It’s gonna be located in my room or the living room for some time. So I can’t have 10 blowimatrons going or be pulling 1200 watts constantly. That’s really my only concern with older less power efficient xeons and server fans/racks. To elaborate more on this point is ECC that necessary?
The most important thing at this time is setting up access to my documents and being an emby server with some 4k videos and mostly 1080p.
Well the caching I should’ve elaborated more on but essentially I want to run at least 6 local devices and have the ability to transcode or push data to 3 offsite locations. Since I want to build for the worst case scenario of watching my emby server on 3 tvs, a computer, and 2 cell phones/ ipads. I figured that would set me up for good future proofing as my family expands. Aside from this I also don’t want me to spend 10 days straight of my HDDs spinning when I initiate a new backup or want to do a fresh backup from scratch for all 40tbs and more as I expand.
Enh, maybe old registered ECC ddr3 is cheap enough.
For fast LAN access, setup 10gbps lan, then make samba(SMB) or NFS shares.
Then for internet access of stuff. Probably Nextcloud for documents and either nextcloud or pigallery2 for pictures. Plex/Emby/Jellyfin for video, or possibly just download what you want to watch. Music, I just sync my entire library with syncthing, but if that takes too much space then you may have to find something else. For other generic files, you may just want a basic directory listing that you can download things over HTTPs, which Nginx or Apache can provide.
For the HTTP based services, you will want to setup Nginx as a reverse proxy, and combine it with letsencrypt for HTTPs access over the internet. That way, on your lan you can see them over HTTP.
SSH is generally all you need for a Linux system. For remote desktop, Teamviewer is probably the best, as long as they don’t “detect” commercial usage and start kicking your sessions after 5 seconds. I use Anydesk, and would be using Apache guacamole if I had better internet speed because RDP is a bit bandwidth intensive.
No
Get server hardware that can fit in a normal case, so a (e-)ATX motherboard. Then replace the default CPU heatsink with a desktop one, and you are more or less good to go. Sometimes motherboard chipset/vrm heatsinks are lacking, so a extra fan may be needed, but that does not have to be loud. The case, PSU, GPU, fans, etc all can be quiet desktop ones.
Not for 40tb. That is WAY WAY WAY underpowered for what OP wants to do.
I only mentioned this in terms of an additional small backup. Of course, it is not suitable for 40TB or as a super-efficient server, it is rather obvious …
I have no idea what odroid or HC2-HC1 are? I definitely want a second full backup of all my 40tb of data locally. Then If I need to backup mission critical files I can do that via the cloud? What’s the least expensive cloud based method?
I guess the main point here is would SSD’s or RAM be more performance useful? The price is semi-negligible. If I need to spend 600$ on RAM to make my build virtually immediately responsive and super fast at transfers then I am okay with that. I can always add more RAM or SSD cache later right?
Should I get a board with 10gigabit LAN built in like Aquantia or Intel? Or should I get a PCIE card? What’s the best way to do this? What are some Network Switch options?
I have Emby/Jellyfin setup already and I just need to work on making them WAN accessible. I need better security in place and the PFsense box may help with that.
As far as music goes I just put it all on my phone. I have 512gb internal and 512gb sd card. I perfer FLAC music so its very space intensive but this is also the reason I’d like 10gbps physical transfers because if I move 50gb of music over I’d like it to take a couple minutes at most instead of it taking hours like it does currently. USB 3.1 Gen 2 2x2 can fix that obviously but didn’t know if there would be more benefits to other types of devices in the future going with a 10gbps LAN as well.
Most RDP stuff I do is merely emailing myself useful documents, turning my emby server on, checking my security/cameras at my house, or getting important notes/lists off my desktop. Nothing super intensive but who knows maybe I’ll get more creative as time goes on.
Okay, I have lots of Noctua’s lying around and didn’t know it was that simple. I will probably go that route. I have a Corsair Carbide 500R for a spare case but would you recommend something bigger, more bays, and etc? Any case recommendations?
I am not knowledgeable enough about this to give an answer.
So the way to do this is to
Setup a DDNS service, there are free options that will give you a third level domain name.
Setup a Nginx reverse proxy with letsencrypt certificates, goes from your internal ip on port 8096 to HTTPs on port 443.
Forward port 443 to your reverse proxy
[quote=“Sirwestofash, post:8, topic:152349”]
Corsair Carbide 500R for a spare case but would you recommend something bigger, more bays, and etc?[/quote]
There are not as many ATX server motherboards, so you may need something bigger.
I personally really like my Fractal Define R5.
There may be some server motherboards that have a non-disableable warning if not all fans are plugged in or perhaps a custom connector that goes to breakout board in the chassis. Supermicro is generally pretty good about being ok to use in desktop case, but others should be googled to check. Then there are workstation boards with support for Xeons and ECC memory that may be a good choice.
I’m probably gonna wait for the next generation but what kind of Workstation boards should i focus towards? Would a server board be over kill?
I should get an E-ATX board though right?
I wish Wendell, Krista, and Ryan could make a video on this because I know exactly nothing about this other than port forwarding lol. I can probably figure it out but it’d take way longer then watching one of their well made videos.
These are small ARM SBCs for one disk. In your case, rather not useful since you have 40TB. Theoretically, they could be used in the number of 4 pieces and large hdd’s then split those 40TB or make Glusterfs. But this is too much fussing probably for you.
Maybe just buy a larger NAS from qnap / synology. As for the online method, it can cost a little, since we’re talking about 40TB here. In addition, the matter of how fast your connection is and whether in the event of a disaster you will be able to recover these 40TB in a reasonable time.
Is this a hot backup or a cold backup? Maybe a tape drive would be better for a cold backup if you already have an active system with these 40TB?
Some time ago I think there was also a discussion about backup in the cloud / online.
Depends on what you find. The the define R6 has e-atx support AFAIK.
Ah, I thought you were think of buying an older board. New server hardware is expensive, so prosumer level is probably what you want if buying current gen new.
I’d really like to make/build it myself and I have a lot of spare parts laying around to make it cheaper. Such as the GTX1080 for transcoding video and remote gaming/emulation.
It would be a cold backup. I don’t need to backup all 40tb in the cloud. The cloud can be more essential/critical files. I can even zip them to minimize the size right or does that increase the risk of data loss? I like Synology but its pricey and i wanted to gain some knowledge of setting these systems up myself since it may one day be useful for a job/part time hobby.
I’ll look into that, didn’t know if there were any guaranteed perfect server/HTPC cases.
I can buy older hardware, but I want a certain type of content/performance. Really trying to make it 4k HDR capable out of the gate if possible for Movies and Gaming emulation. If I go server hardware route instead of workstation then i’d buy older. If I go workstation I may shoot for new or 2 gens old hardware.
