CI orchestrator - A simpler Jenkins

I’ve worked with Jenkins for a few years now, and I have really come to dislike it. One of the main reasons is its fragile plugin ecosystem. After a while it becomes very difficult to update/upgrade without breaking things, and they are continually finding vulnerabilities and stuff in its core.

The other main pain point I have is that you essentially need to learn a lot of Jenkins-specific stuff, which doesn’t carry over to anywhere else. Also, some features are really limited to what the Jenkins/plugin developers envisioned and can be quite restricting when you need to do something that’s a bit off the beaten path.

The project I have in mind to work on this Devember (never participated before) is a simpler job orchestration system to replace Jenkins in my workflow. I would like it to be focused on scripting (of any sort) in a flexible way, so that all you need to use it is scripting knowledge (in any language like Bash, Python, Perl, or anything else) that would be useful for many other things.

I wouldn’t want its core to have too many features, and I’ll try to add on the ones that make sense, but things should be as scriptable as possible (like spawning agents for the jobs). The main “missing” Jenkins feature I wasn’t sure how to replace was the credentials management, but for that purpose, I wrote a small shared credentials manager called Vaultfile (https://github.com/danielthegray/vaultfile), so I will, as a next step, try to get a bare bones job orchestrator running.

I will write it in Go, to get a better feel for that language (which I’ve been wanting to learn for a while), since it would be concurrence-heavy and probably won’t be very CPU-intensive. I made an initial attempt to do it in Rust, but I very quickly started fighting with the borrow checker, and while I would normally embrace the opportunity to improve my Rust, I kinda want to get this over with faster and I feel that Go would be the more productive/useful choice for this project.

I started writing some bits on & off the last couple months, but I felt that doing this Devember thing would be a good way to push it forward (hopefully) faster than I would otherwise. I’d also be open to discussing ideas with people or things that I haven’t thought of or missed. I’ll be pushing the code to https://github.com/danielthegray/formica-ci

Here’s to a productive Devember!

2 Likes

Hi, sorry for the SUPER late reply, and I know it’s way to wait to “win”, but I wasn’t doing this to wait, anyway. I did make decent progress on the core throughout December, with a somewhat-expected slowdown at the end of the year. I have continued to work on it here and there (my work schedule this month has been a bit tight) but I am still pushing to make this a reality!

I have implemented branches and automatic “new versions” detection (to run jobs when a new code commit is pushed, for example) but am still lacking some other features before getting something working. I won’t be posting updates on here, but you can follow the Github repo to see the progress on the project and to be notified when I release an initial version for testing.