I have a small server (Laptop with USB dual 3.5" HDD Bay) With 3, 1TB hard drives; 1 internal, 2 external and running Debian. Is there a way to easily share the storage dynamically between them? i.e When one drive is full, start storing files to the second drive and when that one is full, start storing files to the third. I have heard that you can do this with RAID but since I'm using a laptop, I'm not sure if that's possible.
Is what I want to do possible?
Yes, you can use AUFS for this. It presents the data on the three drives on a single mount point without you having to reformat the drives and you can still use the disks individually or stop using it without breaking anything.
Alright, I'll look into that.
Alright, install aufs-tools and mounted the drives to a folder and did a df -h and only showed 382GB of free space under the mounted folder when there is actually over 1.7TB of free space. Is this supposed to happen?
Edit: Submitting the post with the mounted directory in text in the post resulted in a '403 Connection refused'. Odd.
You need to use the preformatted text button if you want to post stuff like that, otherwise it treats it as html or whatever.
add the sum option to the mount command, that will give you a sum of the total free space rather than (presumably) the free space from the first disk.
for example this is my aufs mount command:
mount -t aufs -o br:/mnt/data1=rw:/mnt/data2=rw:/mnt/data3=rw:/mnt/data4=rw:/mnt/data5=rw:/mnt/data6=rw:/mnt/data7=rw:/mnt/data8:/mnt/hyron/data1=rw:/mnt/hyron/data2=rw,sum,udba=reval,create=mfs none /mnt/pool
But either way while it is only showing you the free space from one disk all that space is still there. The sum option will fix it so you can see it but you can still use it as is and it will still work.
kane@HELIOS:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 7.8G 0 7.8G 0% /dev
tmpfs 1.6G 64M 1.6G 4% /run
/dev/sdi1 95G 24G 66G 27% /
tmpfs 7.9G 84K 7.9G 1% /dev/shm
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup
tmpfs 512M 156K 512M 1% /tmp
tmpfs 1.6G 36K 1.6G 1% /run/user/1000
//vps/keys 8.0G 4.7G 3.4G 59% /home/kane/keys
/dev/mapper/data1 1.8T 1.6T 178G 90% /mnt/data1
/dev/mapper/data2 1.8T 1.6T 179G 90% /mnt/data2
/dev/mapper/downloads 917G 554G 317G 64% /mnt/Downloads
/dev/mapper/data8 1.8T 1.6T 177G 90% /mnt/data8
/dev/mapper/parity2 1.8T 1.7T 69G 97% /mnt/parity2
10.10.1.220:/mnt/data1 3.6T 2.0T 1.5T 58% /mnt/hyron/data1
10.10.1.220:/mnt/data2 3.6T 2.0T 1.5T 58% /mnt/hyron/data2
/dev/mapper/backups1 1.9T 561G 1.3T 31% /mnt/backups
/dev/mapper/data3 1.8T 1.6T 194G 89% /mnt/data3
/dev/mapper/data4 1.8T 1.6T 182G 90% /mnt/data4
/dev/mapper/data5 1.8T 1.6T 183G 90% /mnt/data5
/dev/mapper/data6 1.8T 1.6T 182G 90% /mnt/data6
/dev/mapper/data7 1.8T 1.6T 190G 90% /mnt/data7
/dev/mapper/parity1 1.8T 1.7T 69G 97% /mnt/parity1
none 22T 17T 4.4T 79% /mnt/poolAlright, working now.
thomas@server:~$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 9.1G 781M 7.8G 9% /
udev 10M 0 10M 0% /dev
tmpfs 1.2G 8.6M 1.2G 1% /run
tmpfs 2.9G 0 2.9G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 2.9G 0 2.9G 0% /sys/fs/cgroup
/dev/sdc1 917G 72M 871G 1% /backups
/dev/sdb1 917G 426G 445G 49% /share
/dev/sda6 915G 568G 348G 63% /var
none 2.7T 993G 1.7T 38% / var / www / html / ShareWould I be correct in thinking that, to add this to my fstab, it would look something like this?
none / var / www / html / Share aufs br: / var / www / stuff=rw:/backups=rw:/share=rw,sum,udba=reval,create=mfs 0 0
I put spaces in the path since otherwise I get 403 forbidden even with preformatted text.
That should work, although for me I have it run as a cron job on reboot. I can't remember why I did it like that but I must have had issues with it in fstab.
That's weird about not being able to write the path without spaces, I'm getting the same thing. How odd.
You may also want to use create=tdp rather than mfs depending on how you want files to be written to the disks. This is from the man page:
Policies for Creating
create=tdp | top-down-parent
Selects the highest writable branch where the parent dir exists. If the parent dir does not exist on a writable branch, then the internal copyup will happen. The policy for this copyup is
always ‘bottom-up.’ This is the default policy.
create=rr | round-robin
Selects a writable branch in round robin. When you have two writable branches and creates 10 new files, 5 files will be created for each branch. mkdir(2) systemcall is an exception. When you
create 10 new directories, all are created on the same branch.
create=mfs[:second] | most-free-space[:second]
Selects a writable branch which has most free space. In order to keep the performance, you can specify the duration (‘second’) which makes aufs hold the index of last selected writable branch
until the specified seconds expires. The seconds is upto 3600 seconds. The first time you create something in aufs after the specified seconds expired, aufs checks the amount of free space
of all writable branches by internal statfs call and the held branch index will be updated. The default value is 30 seconds.
create=mfsrr:low[:second]
Selects a writable branch in most-free-space mode first, and then round-robin mode. If the selected branch has less free space than the specified value ‘low’ in bytes, then aufs re-tries in
round-robin mode. Try an arithmetic expansion of shell which is defined by POSIX. For example, $((10 * 1024 * 1024)) for 10M. You can also specify the duration (‘second’) which is equiva‐
lent to the ‘mfs’ mode.
create=pmfs[:second]
Selects a writable branch where the parent dir exists, such as tdp mode. When the parent dir exists on multiple writable branches, aufs selects the one which has most free space, such as mfs
mode.Well, only one drive is connected via SATA with the other 2 on USB 2.0 so I probably would want to fill the SATA connected one up first before going to the others and suffering with 30MB/s max read/write speed of USB 2.0. No USB 3 on the laptop :(
Thanks for the help! I'll mark it as solved.