RAID 1 or better ideas

Hi, first of all, please don’t explain how raid works :slight_smile:

So I’m a video editor and graphic designer and I’m out of storage… so I bought 2x 8TB IronWolf hard drives and now I’m thinking what to do… The first idea was to config them to RAID1 so I get ‘‘safe’’ drives, so i can keep my long term documentary footage, project files, renders there and time by time work with it - but my question is: after all thous years windows still are not supporting something like dual read? So I don’t get Read Speed benefits?

Second question: My motherboard is Asus Crosshair VI Hero (x370) and I can’t get info about hardware RAID… is it supporting some kind of dual read From RAID 1 setup? or I’m still don’t get any benefits except ‘‘safety’’.

In short, im don’t know what to do with both drives … right now I have 2x2TB external Storage on RAID 0 with important footage on it and I want ASAP to get a copy of it.

I’m here because maybe you have better ideas how to organize all better, any ideas?

I have

1TB ssd

250Gb SSD

2TB - HDD

1TB - HDD

2x2tb - RAID 0

3TB - Archive

(and brand new in packages 8TBx2)

  • in the end, I’m interested in ‘‘Safe’’ as possible storage with fast read speeds (for complicated 4k editing projects), don’t care about write speeds…

With that amount of drives i’d be tempted to build a FreeNAS box and attach to it via a pair of 10 Gb NICs, but that is additional investment.

Windows is not great for storage options that are both safe and performant IMHO.

I’d stick the 1TB SSD in your editing rig (for local editing/scratch space), stick all of the drives into the FreeNAS box and use the 250 GB SSD as a read cache on it.

Again though, that is significant additional spend.

But… having an all SSD workstation with no drive noises is nice (the drives can be in the ZFS box, elsewhere away from your now-silent workspace), and ZFS is going to be a lot more reliable for your archive/hard drive storage than Windows will. You can also set up periodic snapshots on it so that if you do get cryptolocked, accidentally delete shit or whatever, just roll back to snapshot.

If all your storage is on windows, it is at the mercy of whatever windows bug/malware/accident/etc. happens inside windows.

Not without three drives… But you didn’t want a lecture, so I’ll leave it at that :wink:

1 Like

If you have extra cash to spare, why don’t go for RAID 10? Your scenario seems to suit this kind of setup.

There are only two reasons to run RAID on a desktop.

  1. Speed - You need the write speed for e.g. video editing. Raid 0 offers nearly twice the read- and write performance.

  2. Reliability - In the case of a drive failure, you still need to access data.

New advancements however have made both pretty useless:

  1. m2 PCI Express drives have anywhere between 125% to 650% write speeds of regular SATA drives - Where SATA has around 600MB/s, m2 starts at 800MB/s can go up to 3.9GB/s. This means you would need to buy a 7-drive SATA RAID0 disk just to keep up.

  2. Modern drives have a reliability between 97% to 98.5%. This means, after 3 years you have between 1.5-3% chance of drive failure. In a raid setup, this increases. With RAID1, the availability increase with 0.97^n, where n is the number of drives. This means availability rates decrease with the number of drives: 0.97, 0.94, 0.91, 0.89, 0.86, 0.83, 0.81 at seven drives. This means at seven drives, there is roughly a 19% chance your RAID will die, compared to 3% for a single drive.

Thus, if you truly need the fast storage, go ahead with a RAID0 setup on a desktop, but I would probably instead buy a large m2 drive and invest in a NAS for older projects. This way, I can do my working copy on a 2TB m2 drive, and have both on- and offsite backups available to cover my ass. It’s a bit more hassle to move content over to my “primary” drive, but a lot less than a 7-disk RAID0 would provide. :slight_smile:

Thanks, but i have worst memories with RAID 10 - some time ago I bought 4 Hard drives, spend a fortune and guess what, something was wrong with controller and i lost all data. Then with specific software we try to recover and some thing we did recover with 14Kb/s

And from other side, i don’t need big Speed boost, so just working RAID1 with Theoretical speed boost would be awesome.

RAID1 (mirroring) will not give you any speed boosts at all, not even in theory. Though I guess you could get better read performance from it for larger files.

I always recommend backups over RAID1 for desktops, the chance of a hardware failure is simply much lower than the chance of getting hacked, power failure or other nasty risks.

How often should you do a backup? That depends on how much work you are willing to part with. Hourly incremental backups to an on-site NAS and then full daily off-site backups to an off-site server somewhere is my recommendation in general, that way in case of a fire, flood or ransomware attack you will only lose one day at most, and 1 hour in the case of hardware failure. It’s managable.

Thank you! :slight_smile:

Some ideas, for example:

Ok so what about I split hard drives in half make 4+4 Mirror, from rest 4 I make a passive copy of (2x2tb - RAID 0) and in other 4Tb I make passive copy (3TB - Archive) & (1TB ssd) - in this case, i get backUp of of my old hard drives and get 4TB of ‘‘safe’’ storage’’ for futher projects, or once again this is stupid idea?

Another one: There is some cheap controller what I can actually buy in Europe and all my problems will be solved.

So nobody knows how well this will perform?

In Long, long terms I will have no choice because every year cameras produce larger and larger files and I just will drown in data :slight_smile:
Right now I searching options for minimal or none investments, but thanks for long term ideas.

Some of the video files are 200GB+ so there is no clear answer will I get read benefits or not. Unfortunately, i dont want to experiment right now, but looks like I will need to :slight_smile:

So in general what I understood that RAID is dying and everything is bad so I start to regret i bought 2x 8Tb…

From experience can say, this is believable…

So The best option is To use one 8Tb as Single drive, and put other in USB box and make back up once per day from 8Tb?

In theory all sound good: ‘‘Do that’’ , ‘‘Do this’’ - but in reality sometimes i have deadline and all my data become one big mess, all my plans for back Up don’t work because I just don’t have space anymore - and time, I Sleep 1h and in that moments i just cant think about some backUps - that why in long term RAID 1 sounds so compelling.

Also couple years ago one drive just randomly died in a moment when I was making backUp and I Just wish there was some RAID 1 setup…

I don’t know how recently you got the 8tb drives, or if you could return them for a refund, but as mentioned, an M.2 drive would be much better to work from, and you might be able to set up a Windows Storage Space for a pair of smaller spinning HDD’s to hold the projects not being worked on?

Otherwise I’d say just look at using Storage Spaces to mirror the two 8TB drives

1 Like

My latest idea is to return One drive and with one make BackUp for all other drives I Have - with some software, that will be my next question which software to use for windows in this scenario.

M.2. - Overall will be too small, some big projects are around 3Tb - that why i bought 2x 8TB - so still don’t know what would be right solution in my case. For smaler projects, I’m fine with 1TB SSD and RAID 0 HDD.

Of course in theory everything is easy, but you should really strive for hourly backups and daily offsites, it will save your bacon more than once. :slight_smile:

As for buying an NVMe drive, I have seen 2 TB NVMe drives out there for around $600, and some motherboards support two slots for NVMe drives, meaning your total amount of “fast” harddrive space could be 4TB. Not the most cost efficient option of course, but at least it’s a decent start.

You can also have internal backup drives mounted directly to your PC, though it’s better to have a NAS for that. Of course you can also have a three-tiered approach with local backup, NAS and Off-site backups. Your PC, your choice! :slight_smile:

I was in similar situation recently, several different drives and sizes, tho running in one big btrfs volume. Since I was running low on space and had to upgrade, so I decided to get a reasonable file server and setup some reasonably safe storage.

I spent 10 minutes a day for the last three months or so, to find out what is available and to get a feel of what the market was like. I managed to get a Dell T320, 80 gigs of RAM, Intel Xeon 2470v2 for €450 total. Yesterday I ordered 2x8TB SAS drives for it, so I can start moving data from old server.

My plan is to use hw raid controller to mirror the two 8TB drives, move 2x4TB drives from current server over mirror them and run a btrfs volume across the mirrors. Reason for doing the btrfs volume, is that it’s easy to expand, and drive sizes are irrelevant. The NIC’s in the server are teamed for higher transfer.

I don’t need the speed that you might want for video editing. If I was in your situation, I’d grab something like this add two or more NVME drives (probably Intel Optane) and use this as a work drive in either Raid 0 or Raid 1. Wendell has a video or two testing these PCI cards, worth checking out if you haven’t watched them.

I’d get two 10Gb NICs off of ebay, one for server, one for workstation.
The 1TB SSD I’d keep in workstation for less requiring things.
The 250Gb SSD i’d use in server for OS and whatever else I chose to run on there.
The 2x2TB and 2x1TB I’d move to server in RAID 1 and add to the btrfs volume.
The rest of the disks I’d just add in a standalone btrfs volume for data I can live without.
Over time I’d exchange the smaller/older disks and acquire new ones, a set at a time, and add them to the btrfs volume. I’d also start looking for a tape drive on ebay, and be ready to grab it if the price is right, the durability of those tapes is top notch, and it’s fairly inexpensive.

I use my server for just about everything that wont be done while I’m sitting at my workstation. Longer running tasks, I find a way to run on server as a job, most often in a docker container. I recently had to re-encode 140ish h265 files to h264, instead of having my workstation do this, i did it in a docker container on server.

Edit: Mistakenly wrote Raid 0 when It was supposed to be Raid 1

1 Like

Everything would be easy if RAID 1 would give read speed benefits - then everything would be much easier :slight_smile:

Right now I’m not ready to invest so much, I think for next PC definitely that will be the way to go with some external server for BackUp. Meanwhile, I can start to plan that. Off course if money is not a problem I would not ask here for some help :slight_smile:

Yes, in long term i need to think something about this, because lately 4k and 8k video is in enormous size, especially RAW - unfortunately its not like I can finish one project and then archive it and then start doing next, in reality, im working seamlessly on 5 or more in same time and just can’t normally manage my data flow. The only solution is like really some ‘‘big’’ server with extendable storage. Even now with new 8TB drive, I feal it will not pass by the end of the year, it will be full very soon.

So I figure out the smartest thing (without additional investment) for now what I will do is.
1TB SSD - For OS - etc
250Gb SSD - AF cash drive
2TB - HDD - Media Drive with movies, music etc
1TB - HDD - Out Of system with Long term BackUp data
2x2tb - RAID 0 (For Actual Video Footage for projects)
3TB - Back Up For ‘’ 2x2tb RAID 0’’ (because it will not be full with important data, but I Need to find software with synchronization feature, what would be best for this case, any suggestions?)
8x8TB in RAID1 - For an archive, stock and project files and most important media files.

Caple external HDD for random usage.

For some time I suppose this is a more effective and safe way to go, or it’s totally irrational?

p.s. I will start to delete all projects what is finished long time ago and hopefully I have nothing to do with that in future, and if the client will ask for it then I will not be a world saver that I kept that footage, but anyway its not my problem, it becomes not rentable to archive some old projects.

EDIT- Words…

I’m searching in google for many questions, and some of them already is outdated can you please help me with actual questions?

  1. I created Storage Pool in Windows UI (NTFS etc) Two-Way Mirror - speed in not exciting but… ok. Is this right way to do it or I need to use Power Shell?

  2. If I decide to Opt out of this Storage Pool - is there a way to not lose all data on it?

  3. If I Decide to add additional drives in future, let’s say 2x 8TB - how that will work? How easy will be opt-out of storage pool? will speed change?

  4. If one drive will go down I can access my files on other PC? on That drive?

  5. What is ‘‘Optimizefdrive usage’’ ?

  6. Any other tips about this topic?

I just can’t decide to Leave this storage space or make it solo drive with one drive outside what will make Synchronized backUp…

I’m not an expert, especially on Windows, so go with a more experienced comment, but my two cents would be:

The solo option with manual syncronisation will allow for more compatibility with other systems/ moving physical disks, but obviously requires more manual effort. This is the simplest way to do it, and lets you be in control. Realistically, when something goes wrong, you will kinda remember how out of date your manual backup is, but when you stop cursing yourself, you will rest assured you are in control

For research, I had a quick look over the following, especially the last section on managing storage spaces, and it seems to answer most questions in a simple overview

(The first link I found on DDG, which you may have already read:)

but basically 1- yes;
2- no, you’d need to run the pool, manuall move the stuff, then destroy;
3- storage spaces look pretty easy to expand (and if enough space, contract/remove devices) or transfer data elsewhere before removing;
4- If one drive goes down, I simply don’t know if the remaing drive fails over to read only, or if it is read/write until the faulted drive is replaced;
5- it’s moving data after device add/remove, so it works better (re-arranging the data after a drive added so all drives are equal;)
6- I only suggest storage spaces as an option, because I presume you are on Windows, and it should be able to import if you change your motherboard, whereas a “Hardware” raid on the motherboard might not.