Need some help regarding network storage and long term archival solutions

Hardware Hub Networking
1

Hello there!

I need your help with solving a network storage dilemma I have since the year began.

All the storage I own (aside from SSD boot drives) consists of 1x12TB, 1x4TB, and 2x1TB 3.5" HDD drives used to be plugged in my old desktop computer. They contain raw video and photo files, music, and mainly many many device backups I have been creating over the years (before phone changes, camera SD dumps after vacations, etc).

Since the structure of them could be considered chaotic (there might be duplicate files, but each backup captures the memories of what I used to do back then, so putting everything uncategorized in the same photo library would ruin the point) I wanted to create a couple of external backups of all the media combined that could be reasonably accessible and long lasting, using optical media.

Then, once that’s done, I would like to set up a easily accesible storage for my home network with the remaining drives so I could access it from both my laptop and other desktops computer. I do not know how secure a NAS would be, but I can imagine the access to it could be restricted to certain visitors foreign devices that say, use the home wi-fi connection from time to time.

The HDDs have been replaced overtime, so at most, the higher capacity ones that hold 90% of the important files are 1.5/2 years old now. When it comes to redundancy, I’m not sure how to set up RAID, every single time I see a video about a real life storage scenario there are problems and when HDDs go wrong it really looks like no fun at all.

All the sensitive files have been put into a secure encrypted container using Veracrypt and a password manager.

So long story short, here are the things I need to solve:

1- When it comes to the optical archive of the full files, do I go for Blu ray storage, M-DISK, tape, or propietary solutions like Sony’s cartridges (which are stacked high capacity disks after all)?

2- Is there any way to easily reorganize the file structure by years while mantaining the timeline? Lets say if I used device A from 2016 to mid 2016 and device B from mid 2016 to early 2017, could some kind of tool sort that out automatically YYYY/mm like Adobe Lightroom does? ( i wont use lightroom to do it since there are more than video photo and audio files in the backups)

3- What solution/ NAS do I go for? I’m overwhelmed with options. The network is 1Gbps or 125MB/s so quick access to the files and backing up should be a problem. Awaiting on 10Gbps upgrade in the next few years.

4- Once the files have been archived safely, I might start cleaning up the NAS disks or deleting the clutter I dont need access to on a daily basis. What are some of the features you like the most about your setup if you use something like this? I’ve seen some video editors from Youtube channels pull some videos directly from servers or network storage to their software timeline, avoiding the hassle of physically connecting external SSDs. Could I do this? If that is true, I guess that could help me run a small form factor PC instead of a full tower right?

5- How secure can I make it? I definitely dont want my storage accessed by non authorized people, I guess it would only be used by the home devices . After I set up my first local network storage, I might look into selfhosting as well, but that is not a priority for me.

Thanks for your help and patience, I hope I didn’t bore you too much

2

  1. Looking at optical media (CD/DVD) in the past I’d say about 10-20 years if you store the discs appropriately and use quality media before they go bad (disc rot) however M-Disc is supposedly better in that regard. While it’s not the best way to for term storage I think it’s the best way to go if you want long term that’s reasonably priced.

  2. I’m sure you’ll get many answers regarding this, my advice is that you don’t need to over-engineer it. If physical space isn’t an issue just grab a cheap tower server (mainly because of proper ECC memory support) from Fujitsu, Lenovo, Dell etc, run FreeNAS Core and call it a day. If that’s out of your budget look at a prebuilt business PC (outlet or refurbished) and run FreeNAS Core on it. You can build your own NAS device but it’s usually not worth it in the end due to not being time and cost effective. Most servers and prebuilt NAS devices will still only come with Gbit but you can easily add a PCIe 10Gbe NIC once needed. Just make sure to get a CPU does that AES-NI (hardware crypto) as that will be crucial later on at least looking at Samba/SMB. If you want 10Gbit speeds or anything near that you basically need to look at NVMe as far as storage goes. If you get a pre-built NAS you’ll most likely get something relatively underpowered for 10Gbit but “small” in terms of physical size. Asustor seems pretty reasonable looking at what you get for the money so to say if you want a NAS device.

  3. As long as you have the storage (and backend) you can use it as either a shared drive or as a local using iSCSI depending on your needs.

  4. The more complexity you add also increases things that can go wrong so you have to decide for how complex (read potentially fragile) you want your setup to be. Under normal home user circumstances a decent password and possibly access locked by IP address is fine but you can go further but you can also use iSCSI and use something like Bitlocker if you want more security. That however doesn’t necessarily stop people accessing files from your PC but there’s no remedy for that except disconnecting it and common sense.

Edit: Clarify #2

3

Thanks a lot diizzy, i will keep looking into it and try to follow your instructions, physical space is not an issue so I will end up going for a tower server most likely. I will also try to minimize the amount of data and definitely use NVMe drives.

I really appreciate your help!

1 Like