Filesystems: What makes sense in the modern world?

Ohh. This reminds me an idea for a video I had a while back.

What would make sense for a modern filesystem organization?

7 Likes

Soā€¦ if you havenā€™t run Linux with a separate /home partition you havenā€™t lived.

Makes upgrades, distrohops, etc way easier.

Problem is Iā€™d like that type of separation of concerns with things like mysql data and other ā€œuser dataā€ that lives outside home.

The Mac thing of packaging an app in a folder is ā€¦ Interesting. Itā€™s one way to sorta-dodge sane package management.

Iā€™ve done the whole /opt thing where I put stuff under /opt

/bin and /usr/bin make sense to me kinda as levels of trust. Sometimes PATH is only /sbin / /bin

But surely be donā€™t have to go all gobolinux for something that more clearly dileneates users changed stuff vs stock stuff.

4 Likes

There was a file system that was set up to mimic a pile of papers sitting on a desk. Canā€™t remember what it was called.

OpenSUSE setups /home as separate partition by default :slight_smile:

2 Likes

What Iā€™d do in my own OS (soonTM) is something along the lines of

  • /system - system configurations
  • /programs - programs live here, where each program is
    • program_name/program_bin
    • program_name/program_name.cfg
  • /users - user file directories

idk if Iā€™ll have devices as files, yet, if so, separate high level dir

2 Likes

But surely be donā€™t have to go all gobolinux

I was going to laugh about that, glad Iā€™m not the only one who feels that way about GoboLinux.

Have you ever taken a serious look at the Nix package manager? Not NixOS, just the package manager portion.


I make one non-standard place on my filesystem: /storage

I treat almost everything else as ephemeral data. /storage is important.

Having everything placed under a /programs folder would be nice (especially for user-specific programs, instead of hiding it in %appdata%, on Windows). It would keep things a lot more organized then ā€œProgram Data,ā€ ā€œProgram Files (x86),ā€ ā€œProgram Files,ā€ ā€œAppData,ā€ and wherever else program-specific information is hidden (ex. the registry, FOR WHATEVER REASON)

1 Like

This is opt for me. /Var should be that butā€¦ I donā€™t want lib

I think the structuring of GoboLinux and NixOS is a great idea. Not using global directories such as /bin allows for package multiversioning. The exception seems to be /bin/sh for bash in Nix though

yup, been doing that since the 90sā€¦ as above, it makes upgrades, etc. painless outside of user fuckups with the partition manager during upgrade/distro-hop. which imho used to be a lot less confusing with cfdisk than the stuff they use today.

Most of that stuff is/used to be under /var

My partition scheme used to be (now, i sync stuff to my NAS, so doesnā€™t matter what my layout is for user data on my workstation).

/
/boot
/var
/home

But distro hopsā€¦ may have different layout for /var so youā€™re still kinda boned.

Iā€™m a fan. Disk these days is insanely cheap (even without de-duplication).

I think eventually Linux will be dragged kicking and screaming into the 90s and end up doing similar things, with symlinks or filesystem level dedupe (even scheduled runs, not necessarily real time like ZFS) the ā€œcostā€ is minimal, and it makes package managers no longer really required for 90% of stuff.

I love mac application management for this.

Want to run Pixelmator or whatever from a USB drive (for whatever reason)? Drag the icon there, boomā€¦

Linux even more so would benefit from this than macOS - as macOS is standardized - there are no different distros. Application management on Linux is a bit of a nightmare at the moment to be honest. Anyone who has tried running commercial games or software on anything other than Ubuntu knows what Iā€™m talking about. Package linked to version X of library Y that either isnā€™t included by default, is not in the default repo, or whatever.

I would argue that there is no ā€œsaneā€ package mangement on linux. Some are better than others, but none of them have 100% market penetration, so which ever flavour you choose you run into issues from time to time.

I actually ran slackware for a while in the late 90s/early 00s in order to avoid package manager dependency hell. I think it may have been around the time where i got super pissed at all the breakage in Redhat 5.0 (not RHEL, redhat linux) with the glibc switchover. Just compiled everything from source, and if the package isnā€™t installed the feature doesnā€™t get turned on :smiley:

edit:
scratch that symlink ideaā€¦

1 Like

As long as the filesystem is hierarchical then branching has to be based on some sort of (arbitrary) feature (be it security, portability, encapsulation, whatever). Everyone is going to have a different preference, so one size will never suit all.

For a one-size-suits-all solution, I think you need to eliminate the hierarchy entirely.

Think of your photo manager software (e.g. Shotwell on Ubuntu 18.04). Individual photos are copied into your library but you really donā€™t care where they are stored. You ā€œtagā€ each photo, which modifies the id3 metadata for that photo, and updates the index for the tag. Finding stuff after that is trivial and fast.

Now consider the situation where ls /bin/ might actually translate to ls -bin /stuff/ behind the scenes ā€” all files are in the stuff directory and the -bin switch is actually acting as a tag selector.

Of course, since ā€˜everythingā€™ is stored in /stuff/ it should be the default behaviour, so commands reduce to something like ls -bin and .myApp. The actual physical location of the files is irrelevant as far as usage is concerned ā€” they are all just ā€˜pluckedā€™ from storage using one or more tags.

Note: Iā€™m not suggesting that all files literally be stored in one directory. What Iā€™m saying is that all files reside under the /stuff/ directory. The actual sub-structure of that directory, however, should be of no concern to the user. It could be organised like a squid cache, a key-value locker, or whatever ā€” doesnā€™t matter. How files are stored in there is unimportant. Users only care about how to access files in there.

Access control can be trivially implemented at the tag level.

For portability, the filesystem could be instructed to save -usr or -db tagged files on a different partition/drive.

For performance, certain tags could be saved to a RAID or other v.fast media.

Apart from a few default tags (e.g. -sys, -swap, -device, -vm) the rest could be user-defined. The policies regarding storage could also be user-defined.

FUSE could be used to prototype such a tag-based filesystem, perhaps?

So, for the ultimate in flexibility, I think a tag-based filesystem with user-definable storage policies is the way to go. It would pretty-much solve every problem I can think of.

There are some already in existence. In general these are called semantic file systems.

1 Like

I regularly struggle with giving users the ability to run and update their own apps on windows WITHOUT local admin. Frequently when I ask vendors what files / registry keys the software writes to I get back ā€œdisable UACā€. Whenever looking at new software im stuck running procmon and watching what folders / regkeys it writes. Giving permissions over a single folder so users could update and run the software would be a big sanity upgrade instead of C:<software>, C:\ProgramFiles\Software, C:\ProgramFiles\InstallShield\Software, C:\ProgramData\Software, C:\ProgramData\Download\Software, C:\Users\Public\Documents\Softwareā€¦ it never ends.

1 Like

@wendell - could do this with ZFS and data-sets.

Different distro you hop to moves the location for your SQL data, you simply mount the dataset under a different point in the tree? Or just change the configuration in the distroā€™s xSQL install i guess. Requires ZFS support i guessā€¦ which is still far clunkier and painful in Linux than it is in FreeBSD landā€¦

I havenā€™t DONE this, but should be feasible iā€™d think? ZFS mount point is just a property on the data-set (from memory).

Either way, iā€™m a big fan of splitting the OS and data onto different partitions, datasets or disks.

Wait, are we talking about Filesystems or Partioning your harddrive?
I might be off here, but where on a drive your data is doesnā€™t have a whole lot to do with the chosen filesystem, does it?
All the ideas so far wouldnā€™t care wether you use ext4, ntfs, fat32 or anything else.

Very true. I think my next re-install I might try F2FS since im OS NVME /shrug.

Its like 4 years ago now an FS wreaked me on linux. I made my /home xfs and some steam games just broke and crashed.

I donā€™t know. Iā€™m just sticking with ext4. Iā€™d certainly want to try other things, but in the end, i have a single disk. I just donā€™t seem to find any reason to switch to another fs.

As a programmer, I like having my own personal ~/Src and ~/Bin directories for personal stuff. It would be interesting to see a package manager that allows users to do local installs when they do install and system installs when they do sudo install

I like installing / on an NVME drive, /home on an HDD, and /tmp in RAM.

As a Gentoo user (Gentuser?) I like having portage build directories in RAM too, but thatā€™s a different story

1 Like

Both.

Filesystems and utilisation of.

Those who donā€™t understand Unix are condemned to reinvent it, poorly.

ā€“ Henry Spencer

Yes, UNIX (and Linux) is an old system with a lot of warts. Yes, these days we have different needs. And at the same time, we for many things really donā€™t.

To understand the current file system hierarchy, consider this matrix.

      | user   system
------+---------------
 read | usr    etc
write | home   var

The UNIX filesystem in the beginning followed this simple principle, and it was designed in this fashion. As a user, you are essentially only concerned about where your work and config files (home) and programs (usr) were stored, and the system is only concerned about config files and logging. That said, things could be updated to a more modern layout for sure.

So, what has changed from the olden days? Letā€™s do an ls -d /*/ of my ubuntu root:

/bin    /dev   /lib         /media  /proc  /sbin  /sys  /var
/boot   /etc   /lib64       /mnt    /root  /snap  /tmp
/cdrom  /home  /lost+found  /opt    /run   /srv   /usr

Of these, /cdrom, /lib64, /mnt and /lost+found are most definitely legacy cruft. Other ones, like /bin and /sbin, could be moved to different folders.

What would I change? Mostly a cleanup. Move all OS-dependent and kernel related dirs, such as /proc, /bin and /dev to their own folder (e.g. /core), move snaps to usr, and so on. Perhaps with a bit of symlinks remaining for backward compatibility. I would also make core extremely hard to alter without any trace whatsoever, since that would be what makes the system run.

Storage would all go to /media as would temporary file systems such as CD ROMs, SD cards et cetera as well as remote file system mounts (network drives).

Apart from that I think there isnā€™t much else Iā€™d change actually.

1 Like