Hi, I’m new to level one, I posted on the sysadmin subreddit (and can’t link to it when posting here) but a friend suggested that I might get more helpful advice here, so here goes.
I’ve been using Storage Spaces with ReFS in 2-way-mirror mode for a year or two now and it basically doesn’t work whenever IO happens - the virtual drive shows as maxed out at 100% usage with 0 reads and 0 writes, Windows locks up, Explorer greys out for tens of seconds at a time.
I have researched and found out about logical/physical sector sizes, in fact here’s some data:
The pool is made up of 4x4TB WD drives, and the guidance I read was that the logical sector size should equal the largest physical sector size (perhaps this is bad advice?) - turns out it already did, I didn’t need to change that setting.
What bothers me most is that I run a Linux machine with a BTRFS pool on it, ith a variety of differently sized drives, similarly configured for a single level of redundancy in parity, and it works perfectly. So this should work.
Am I doing something wrong? I’m at the point now where I’m thinking of either
Dropping parity - as the name suggests it just has games and stuff on it anyway, I was merely being careful using parity
If that doesn’t work, I would have to migrate to some other pooled filesystem - maybe the Windows ports of BTRFS or ZFS - but that seems drastic as they don’t seem that well used yet
Any tips or advice would be very welcome, and if I need to provide more information do let me know what to execute to do this!
I’m pretty hardheaded when it comes to dipping toes into Linux and my knowledge of networking is as limited as my Linux knowledge, so Take from this what you will
I use freenas with ZFS, I can pick what level of parity I want from 0-3 drive parity and it’s pretty easy
Nitpicks-you can’t add a drive to an existing pool later down the road, or change parity level after adding a drive
This might be exclusive to freenas, which I admit is that the best solution out there, it’s just my first and Current experience with pooled storage
I believe you can make more pools if you do add more drives, so that’s a sort of a workaround
I’ve tried using Windows software raid, and honestly it’s the worst piece of hot garbage I’ve ever seen, even when it was working I don’t think it was even close to intelligent as zfs or btrfs
If you don’t use encryption or compression, freenass will run on a potato
I use level 9 compression on 5 Sata Hdds on a fx6300 16GB of ram and I’m still just under gigabit speeds
TLRD use a NAS Linux distro, if I can do it, it must be idiot proof
Samba or iscsi over network; zfs, btrfs (or even lvm on spiny drives, btrfs is easiest for adding/removing individual drives changing raid levels on the go; ZFS has good balance of features and stability). I’m not sold on iscsi to be honest, I’ve used it, but there’s pretty much nothing you can do with iscsi you can’t do with images on a file system… maybe it’s a better fit for small data centers or cloud providers-big cloud uses it as a cruch usually
Samba, the hardest parts for me are little things because I’m dumb and can never remember if it’s //192.168.0.X or \192.168.0.X
I was even able to set it up to be password protected
I’ve also set it to be a network drive so I don’t have to remember if it’s the / or the \
Slightly off topic, he first and only Android file manager app that can view samba shares AND have thumbnails for pictures and video that I’ve found is an app called X-plore
For Samba/Windows file shares, You can use hostnames or DNS names (short or long, whatever resolves to an IP) to access files on shares. All windows has to know is an IP.
(Also, even though I know SMB only needs port 445 to run, to be honest, I’ve never tried using windows with Samba on anything other than port 445, not sure if there’s any point to that, or if that’s even possible)
The slashes are interpreted by the software on the OS you’re on, DOS/Windows likes backslashes as path separators, internet urls use forward slashes, same as Linux/Unix os-es.
For Android, you can use Total Commander with Samba (as well as with FTP or SFTP or however you want to deal with your files). But, whether or not you’re running Samba you could also use SFTP - that’s a little more battle tested for use over the internet when you’re out and about.
If you want to pool disks in Windows, Stablebit Drivepool ($30) works well enough and allows you to designate data that you want to be duplicated within multiple drives in the pool. If you do a trial of it and decide you like it, message me prior to purchasing.
However, if Linux is an option at all, I strongly recommend mergerfs for pooling and snapraid for parity. It’s highly flexible and allows you to add storage or parity drives as you go with minimal fuss.
Here’s the latest Perfect Media Server guide. It recommends first reading the 2016 and 2017 versions. Do that. There’s a lot of other software that it goes over, but mergerfs and snapraid are really all you need.
Thanks for your ideas - I have a few things installed on this (virtual) drive, and I worry that if I use a network share that I won’t be able to use them, as some things refuse to install on network shares.
I do actually have a Linux box with a load of storage on it, running BTRFS, which is part of what makes me think the Windows situation is bad.
ISCSI intrigued me, people have described exposing an ISCSI target on a network drive that’s actually a “ZVol” - dalaris_dot_com/how-to-install-zfs-and-present-a-zvol-through-iscsi-in-ubuntu-14-04/ (whatever that is) - so this might be a decent solution for me if I host a Linux system in HyperV and have a “networked” ISCSI drive that’s really all on the host machine for speed?
Drivepool sounds interesting, although it scares me a bit that I’ve never heard of it; I was preferring to put my trust in tried-and-true-by-the-masses solutions so I didn’t have to think too much about my data. Just to be clear: $30 is perfectly reasonable - it’s the hours of potential data recovery I don’t want
Linux is certainly an option, tbh I’m more comfortable on there than Windows. I would note that I’m using this drive predominantly in Windows for Steam gaming and that I have Steam installed on this (virtual) drive. A lot of software doesn’t allow you to install it on a network share so I’m not convinced that that is a valid route for me.
I can run a copy of Linux in HyperV and share a ZFS or BTRFS volume locally on the virtual network adaptor, but I’d need some way to hide this from Windows programs, potentially with iSCSI? I don’t know.
I hear good things from UnkyJoesPlayhouse about drive bender, but not sure if it is server only. Worth looking into.
It must be pretty easy for him to stick with it
I have bee using DrivePool and Clouddrive (both products from the same company) without issues for over a year.
Unfortunately, I don’t use it for mission critical storage, just for spanning camera recordings over 3x4TB drives, but it has worked for me perfectly during that time. I run their included Scanner product to monitor the health of all the drives in that system because some are really old SSD drives.
Unfortunately, the most important aspect for you is recovering from a drive loss, which I just haven’t had. Check their forums and reddit for some experiences of others, but I have heard nothing but bad things about Windows Storage spaces.
I’m going to test Drivepool to see how that goes - the reason I’m probably not going to go for a Linux solution (even though in many senses I prefer it) is that most of my data is Windows Games and it just seems a whole lot easier to be running a native filesystem.
