Hello all, This is a call for help. I believe Linux uptake is increasing, and with that, this trend of sharing installation scripts via curl into bash is becoming more common as shown below:
I used to do this in my tutorials and was "told off" by the internet by people I respect, because of the insecurities. Essentially anything could be pumping into your bash terminal. I wanted to address this so that people could still share BASH setup scripts easily, but user's could be sure that the version they are downloading and running on their machine, is exactly the version that the user who posted the script intended.
This is why I created prog-exec (explanation below), but I'm pretty sure I'm the only person using it, which makes it pointless.
I am not asking for everyone to "spread the word" for everyone to use my tool. This is a band-aid on a wound and could be greatly improved, and I don't want all of the internet hitting my little VPS and being the single-point-of-failure for people.
What I would like to do is the following:
This idea becomes a community driven tool under the Level1tech name to give it some "credit" more than my "programster" brand which is almost non-existent (I only get roughly 15k hits per month on my blog).
I transfer the code for prog-exec and scripts.programster.org under an open source license to an official level1tech github area, organization.
rename the tools as part of the rebrand (e.g. no longer called "prog-exec").
Preferably we move the hosting to a level1tech server that is happy to take a heavier load.
We publish packages (deb/rpm/etc) to make installation of the client tool (currently the prog-exec BASH script) to make it much easier for users to install and give it more trust.
We add other features, like authors with PGP keys that sign the scripts. This way users can add public PGP keys of authors they trust and only their scripts execute (similar to how PPAs work)
If it becomes big enough, we look into changing the architecture to become more distributed, so anyone can deploy a scripts server which acts like online key servers, and scripts spread across the net, thus if one script server falls over, the client looks for another.
Why Not Just Use PPAs and Packages?
One could argue this is what signed packages and PPAs are for, but have you created and published a package recently? Its a pain in the butt and this makes it so much easier for people to share scripts. If sharing packages was as easy, people wouldn't be sharing scripts via curl commands. This is basically a step before a package, and should work cross-distributions.
Because this is a very, very bad security practice, and WILL NOT be repeatable across different distributions without the complexity that is already inherent in most packaging systems.
What you're proposing seems to be some sort of jury-rigged makefile system on top of a tool that was never meant for that functionality. I'm not trying to be too harsh, but there are good reasons people aren't doing this. Have you read up on arch PKGBUILDs or how the ports system works on BSDs? This may come off as condescending, but they may be drop-in solutions for your issues with full packaging.
Because this is a very, very bad security practice
Please elaborate. I believe once we got OpenPGP signing and authorship in, I believe it would be on the same level as a PPA, you just put your trust in the author or anyone else who also signed the script. It is up to the user to add the public keys of the people they trust to review/write scripts.
NOT be repeatable across different distributions
Scripts will not be repeatable across all distributions (I'm referring to Linux distros here, not BSD), but a lot of them will work across a lot. E.g. the hello world example. One just makes sure to say something like "Ubuntu users can install using this command..."
This really is meant to be a step up from people just pumping curl into bash. Unless an alternative is easy, people will keep doing this and using it. Docker did this a lot until they finally got the docker-engine package in the repos and a PPA.
In terms of making the scripts reproduceable, there's a reason people use things like puppet or ansible instead of raw bash to set up systems with the same distro on similar equipment, and a reason that even the simplest package managment systems differ somewhat from that spec or adhere to a standard.
The more complex you have to make a shared bash script, the more headaches you run into, and the more you'll want to construct a standard for them, or a framework to add more functionality, secure or not. At that point you just have a ports tree or PKGBUILDs with a less robust feature set built around them.
I get what you're going for, but what you're proposing is literally just another of many simple package management systems without as much functionality, and the possibility of breaking users' systems if they mix and match with the native one, or happen to use a distro that manages things even a little bit differently.
I don't want to shoot down your idea on principle, but there are important design decisions that went into making modern package management the way it is: some good, some bad. However, making packaging more complex than curling remote scripts in order to make things reproduceable and less likely to damage things on installation was not a bad one.
If people didn't need to build containerization, flatpak, snaps, to solve the many cross-disribution package management problems that exist, (more or less,) then we'd all be using signed bash build scripts right now.
I think the fundamental point that I think people get stuck on here is that I am not trying to replace a package management system, or even make something that is fool-proof. I agree that the ultimate solution is for people to create packages for the system(s) they are trying to target. All I am trying to provide something that is somewhat safer than pumping curl into bash yet is extremely easy.
A basic example could be a as simple as running the BASH commands provided by docker for installing docker on ubuntu. Yes that script has external resources that the system cannot automatically check, such as adding a key for docker for the docker PPA, but that is why the script should ideally be signed by the docker private key (private keys are not kept on the server, one uploads the signature for the script) for people to trust before the script executes. This isn't a case of simply pulling a script from scripts.programster.org, but having a client that does that and perform the security checks like checking signatures, and hashes in case the script was manipulated either in transit or on the site itself.
At the end of the day, one can take a bash script, turn it into a deb, and list that on their site for people to download and run. They often provide the hash on the site for you to check the downloaded deb against. This basically makes it easier to essentially do the same thing.
Could someone publish a script on there that would **** up your system or cause packages to clash, absolutely. This is why one should only scripts that are signed by authors you trust (such as perhaps the ubuntu team, or webupd8 team) etc and not just trust everyone.
I'm not sure I understand the value of what you're describing in practice at all.
In theory, it's useful for certain niche use cases, sure.
In reality, people tell their users to curl a .sh for one of a few reasons:
They're lazy, in which case they won't bother with the extra effort of the system you're proposing for the same reason they're not bothering with a PPA, the AUR, or whatever other flavor of community repo system they use.
They don't know how to package things properly themselves, and this is categorically not an actual solution for that problem.
Their software is malicious.
Their audience/users are on a system without proper package management
There's also extant, simple solutions to this problem in the form of things like PPAs, The AUR (which closely mirrors what you're describing if you want to use it that way,) building from git, gists, etc. At best you're reinventing a wheel for use on a vehicle that has too many already.
The best way to remedy this bad practice is to educate people about it, and on using alternatives, not enabling its use via signing and a frontend.
That said, I wish you the best, in current and future endeavors.
If there is already a solution that achieves what I am doing (ignoring teach people to build packages, which is good), I will gladly use that instead. It is well possible that someone has already invented what I am trying to achieve, but I just haven't found it.
Can you link "the AUR", I don't think I've seen it yet or maybe I just dont realize what the acronym stands for. - nvm discovered its just Arch's package management system.
My point was that nothing's stopping someone from providing the hashes on their site, and telling people to download, chmod +x and run the script only after reviewing it manually. The entire premise of this thing is just enabling bad behavior, unless it adds value past that, in which case it becomes a package manager.
Am I crazy in thinking that curl | not having built in hash checking isn't the problem here?
Again, there's every possibility that I'm just retarded. Bear with me. My aim is constructive criticism.
Exactly. Digital Ocean (and many, many others) should either provide build instructions or hashes and a manual download for their .sh. Very similar effort required and much less terrible.
That is exactly what this does right now, then I want to go further and add signing. The prog-exec client downloads the script, generates a sha256 hash of the script and checks it against the one provided by digitalocean in the prog-exec command and asks the user to review it before pressing y to proceed.
Sorry maybe my post was confusing, this isn't attempting to make something that is cross-distro. Its just that BASH scripts tend to work a lot cross-distro. Yes, create github scripts for each distro, then create a prog-exec command for each script.
