Filesystems: What makes sense in the modern world?

Root and /boot on SSD, /home is on a 2TB HDD drive formatted to EXT4.

Don’t really have much to add, but it would be great if the config files wern’t all over the place. Having a skeleton, global, local and we put it here also because this other distro decided to put it here so we just toss them all over the place and let the end user guess what one is in use. Empty configs will also drive me up the wall, there is nothing in it but the program will not run without it…

Microsoft adopted c:\programData for anything machine specific. I kind of like this for the new /var … Move /var/lib to /usr/lib

This is where things like steam games belong and acls that make sense for that top level folder.

Backups would be easier …

For me it’s EFI, Swap, Root, Home, Storage…all of them on separate partitions, except for storage, which has 2TB SSD Raid partition on it’s own. My steam/game library goes in a separate folder on storage partition. And everything using EXT4 (other than EFI and swap).

This is basically how it’s been during the 12 years I’ve been using linux(last 6-7 years it has linux exclusively on all my machines at home) . Heck… I’ve used pretty much the same partition regime the few times I have been using window in the past.

One thing that has always annoyed me is that different programs pollute the /home folder. While more and more uses ~/.config, you still end up with a bunch of hidden files and folders in ~. Some of this is configuration, some is autogenerated configuration, and the rest is caches and other random stuff like wine prefixes, etc… I don’t like the idea of mixing all of that with my personal files, they have nothing to do with my Word documents, cat pictures, music, etc. and should not be anywhere near each other.

Another wish I have is that the configuration is more separated than now. There is the /etc folder for the system configuration, but 99 % of that is default configuration files, but what you are really interested in is that 1 % of files where perhaps a few lines have been changed. I have no way to manage that.

Bunch of brainstorming below:
I have been speculating if it is possible to virtualize the entire package management system. In short, keep packages as single files and allow the system to access them as if they had been extracted and merged to the file system. So sort of a mix between the Mac approach but still separating the reusable parts out.
Each process would then live in its own filesystem, only given access to just what it needs to run. (Partially to ensure the dependencies are correctly defined.) The biggest issue with Linux package managers is that they don’t support versioning, but here you could easily have several versions and specify which you need in the package. (And no more drivers which fails to load after an update because they have been overwritten by a newer version than the currently loaded kernel.)

This causes the entirety of /etc to become virtual/read-only so you can’t edit it directly. Modification would need to be made using some form for patch based system. While some tooling would need to exist to make this convenient, you would end up with something like a /etc-patches folder which you could easily browse to see exactly which changes have been made. The package manager could also during an update do a git like workflow and check for conflicts.
I think this is really important for a sane environment that it is easy to go back and revise your configuration. Maybe you applied a workaround 2 years ago which is no longer necessary. A default value changed for a line you modified in a configuration file, you probably want to check out why and if this affects your choice of setting. Or you are on a server where multiple people have changed stuff and you need to understand what is going on.

I’m still not sure how to protect the /home folder. I think the best is just to abandon the /home folder completely for personal stuff, but this is still were all programs defaults for the save dialog so that is kinda annoying. (Also an annoyance for me, all GUI toolkits have their own open/save dialogs with no way of tweaking them to your liking and certainly no way to get them consistent. Allow us to make our own in some way, even if we have to reimplement it in each toolkit!)
One sure-fire way would be to do something like I believe it works on iOS, make all programs read/save to their own unique part of the file system. Literately no way to affect the global file system, which is an issue by itself. So while I don’t think that is a sensible approach, I think it might be interesting from a security point of view to be able to control which processes can read/write what. (Android like permissions: ‘Allow access to webcam?’)

These are just ideas without not considering real world complexity, but I fell like desktop computing in many regards is stuck to old ideas and we desperately NEED to move forward. I didn’t know about GoboLinux before reading this thread and while I don’t know if it is a good approach I’m glad to see someone trying to do something unconventional here.

As for the /home partition, I definitely think this needs to use a semantic file system and have thought so for the last 10 years. (Automatic version control as well would be a good addition.) I just don’t know of any good solutions. (I’m seeing some new have popped up, will have to check those out.) We are seeing so many solutions specifically for organizing pictures, music, ebooks, movies, etc. which I think fundamentally does much of the same so we shouldn’t need to do this on top of the file system. But I think reconsidering the GUI on top of that is just as important as the file system itself.

There are so many new concepts popping up on the web and mobile, while the desktop experience is pretty much the same as it was 20 years ago. Now we are loosing everything to the cloud, into the hands of big corporations…

6 Likes

Where else should user-specific configuration live? Your home folder is where your account has unrestricted read/write access. You’d prefer to move that “clutter” to somewhere global with every other user’s clutter?

I LIKE stuff being stored under .whatever, as far as my (non configuration) personal data is concerned it lives in /home in folders without a preceding dot (e.g., Documents/*).

User specific configuration is still valid user state you may want to back up. Stuff like SSH known hosts, identities, shell profiles, email autocompletes ,etc.

Just because you didn’t explicitly save it there, doesn’t mean you won’t notice if it wasn’t backed up and restored. Any of your “personal data” should probably live in a sub-folder (or more) of /home anyway; just relocate your personal stuff to ~/Documents and ignore /home if the clutter bothers you?

Everyone is pulling back to the cloud and “owning” you that way. Linux will need to stride forward as over the next decade. MS and Mac OS will slowly rain in everything to the cloud.

Linux apps may not even have Windows ports as windows has a linux kernel already. Windows apps will be like phone apps hosted on the store only.

1 Like

I believe (s)he meant that the config stuff should move to ~/.config/ period. Of course, there could also be a /etc/user/… configuration folder, for instance. Both viable options to accomplish the same thing.

I think exploring past the folder/filing cabinet design metaphor we’ve been stuck with for decades might be productive.

But, you need legacy support for it to work with anything else in modern computing so it’s not really practical outside of a thought experiment.

I find systems like collaborative pipeline file managers and content databases developed for creative work to be much more intuitive than file managers, and apple implements some of that, but I don’t think it goes far enough.

A folder, in practical terms, is basically a “tag” that filters files out right?

Maybe we’re limiting ourselves by treating it as a separate, heirarchical thing in the first place.

As a practical example, I work with Resolve Studio day to day.

I can search all the files in the folders i’ve made it aware of by details as granular as color temperature and time of day it was shot, have bins that autopopulate with clips containing a certain person’s face, or filter out still images that don’t have normalized motion vector data.

Where the file is doesn’t matter, just what it is. Most modern operating systems have a rudimentary version of this to make finding your files easier, but why not just focus on the utility of that and have a flat storage structure rather than separating the two concepts?

I could see a system where you could integrate that type of file management tightly instead of relying on a separate sql db and paid software, and I think it would be very nice to use.

You could even make package management easier by having binaries tag the stuff they depend on and not have to worry about what hidden folder it’s nested in. Stuff you don’t need to see day to day isn’t tagged as an image or a program or whatever, so there’s a litter free file browsing experience.

You could install programs that rely on different versions of the same library in a snap by having version of the lib be metadata, and having package management check for less than/greater than before tagging. No installing in /opt/, no altering environment variables, no dependency hell. just plop in the other version of the lib and tag the misbehaving program as part of the installer, or manually as the end user. Obseletes stuff like which and the reliance on commands like --version for scripting too.

Not just for power users either. Window pops up, single click image, hit related tags, click png, hit advanced tags, and FR metadata shows a grid of faces. click the face of the person you want and they all show up without having to dig through dozens of folders labelled “DCIM_Canon_201X” because you didn’t painstakingly rename all of them.

And you can used a “saved search” to keep that specific set of filters for quick recall, or even set up a “bucket” that scans for new things matching metadata and automatically adds them. If the system is designed to be “metadata aware” at the base level, past mimetypes and file tagging and stuff, right down to the core of it as a replacement for folders and links, you can do a lot more without 3rd party software.

You could even have a uuid system that lets identical filenames or files coexist and lets the user delete duplicates when they want more easily by searching the file names as metadata.

Doesn’t completely rule out the benefits of volume management either, you could have a tag that indicates drive or partition location and honors that at a block level.

2 Likes

I love the discussion here. Wish there was someway to get a collective thought forward and make something stick in the Linux world. Hell windows too.

Fair enough.

But its the same amount of clutter, just another inode wasted to put it under another subfolder.

The only dotfiles i have in ~ are config/app state related, just turn off auto display of dotfiles and no problem?

MacOS has done tags for years and they generally work pretty well if you remember to use them.

But its a brain-shift and if you’re multi-platform (and 99% of people are to some degree) its a mental adjustment you need to make.

Microsoft’s WInFS (From Longhorn) was meant to be a database filesystem that was talked about and promised to be coming soon way back in the mid-late 90s.

It never really gained traction.

People’s brains work with hierarchical filesystems pretty easily (as it relates closely to real objects, e.g., the sock is in the drawer in my dresser in my bedroom), other more exotic and probably better concepts are perhaps too abstract for them.

1 Like

I’m aware, which is why I said apple didn’t go far enough with it.

It’s also why I mentioned a system like this would have to ignore all prior art to function well,

Ahh, i elaborated a bit more above. I agree it is a good idea and wish it went further. but both apple and ms have tried and outside of fringe adoption it hasn’t really flown…

Google also tried it with gmail and people still find the gmail tagging stuff wierd.

1 Like

yep, I don’t see it taking off, specifically because of legacy support being a requirement for something like filesystems rather than lack of merit.

maybe you could have files that act as fake heirarchical filesystems like zvols on zfs, and incentivise people to dump what’s on them to the main system because the contents aren’t indexable otherwise.

ZFS is actually what renewed my interest in a “db filesystem” paradigm because it opens up so many things by integrating the whole merkle tree transaction/metadata management thing into the fs itself.

1 Like

I think there’s also some resistance because the current generation(s) of computer users simply don’t trust computers.

Hierarchical structure still works if the OS breaks and you boot from something else, etc. And “it’s exactly where i put it”.

Database-like tags? Kinda abstracts you away from the implementation of where things are a lot and its a bit of a leap for most people to let go and let the computer handle it.

It’s a leap of faith that a lot of people (myself included) feel slightly uncomfortable with.

1 Like

folders are an abstraction too, It’s just a less flexible linked list. files aren’t physically organized on the disk in the way represented by folders either.

Relationally it only makes sense if you get rid of one or the other entirely.

Historically I’d agree with you but people have been primed by mobile operating systems, search engines, and creative software for long enough at this point that i think there’s already some understanding of both required anyway.

also, assuming the OS breaks in this world where we have a db filesystem, it will mount and work on any other OS that supports that format. Not really a concern as long as there’s a single other computer that can read it.

Of course, but i feel that the human brain understands (or rather, accepts - whether the understanding is correct or not) the logical structure a little easier.

People like to split things into individual groups to classify them in a hierarchy.

Tags enable things to be in 2 or more groups and i feel the brain is less comfortable with that. Its a silly mental restriction but i do think it’s a factor and “letting go” to use something like tags i feel is difficult. For me at least anyway, and i even think its a good idea.

1 Like

Folders are what people are used to, and it was a useful metaphor in the 80s when people were transitioning from literal files and folders for productivity.

People were also used to crapping in a bowl under their beds for almost a thousand years but we still transitioned to modern sanitation.

There are adults alive today that don’t know what “hard copy” means, or what a vhs or a floppy disk is. Most kids learn to interface with a mobile operating system first and foremost, and those hide the folder structure as much as possible by default.

It might be time for a switch if there ever is one.

Don’t get me wrong, i agree it’s a good idea to press ahead.

I just think it’s going to be a difficult shift, that’s all.

Cloud storage may help actually because file sync to all devices is another “this isn’t in only one place” or “i copied it to devices X and Y so that’s where it is” mental shift…

1 Like

Yeah. I get that. But I also have similar opinions about gesture based touch interfaces (they’re good but apple doesn’t go nearly far enough in the places they need to) so my biases tend to lean that way in the first place

The difference is I can buy a phone with a full-on gesture UI (sailfish,) and I can’t use any type of full db filesystem, so I’m much more curious about the latter.

1 Like