Sysadmin Mega Thread

ZoL has not released a kABI for el8_2 so you will have to use DKMS to get it to work with RHEL/CentOS 8.2 for the time being.

Not testing it on 1 or 2 first??

I only have ZoL on my main server. My nas runs FreeNAS.

All my other VM’s updated to el8.2 just fine but this was the issue when I updated the underlying system because all the VM’s are stored as .qcow2 on a SSD zpool.

1 Like

oic. :slight_smile: My dad is still testing it on 2 vms to make sure. :slight_smile:

Yeah unfortunately I don’t have any secondary server’s to do a proper DR testing but what I had to do was uninstall ZoL and reinstall from the 8.1 repo and then enable DKMS and then it rebuilt the package and my pool was imported.

The version did not change it is still 0.8.4-1 but due to the way the kernel modules load this was necessary.

Everything else works without issue.

1 Like

Now, CentOS can be a valid type of hypervisor. :smiley:

1 Like

There is a pretty major issue on latest CentOS 7 and 8. Looks like it’s on skylake xeons, not sure if it’s broader in scope, but it’s definitely breaking servers.

1 Like

I had an update to that microcode package but everything installed successfully (AMD Epyc). My whole environment is updated now.

I do see however, that the update to 8.2 re-enabled kdump (I disable it for VM’s), and that the MCElog now fails saying my CPU is not supported.

Everything else works fine.

1 Like

That’s interesting. STIG says to leave it off because of a security concern, I forget what exactly. I wonder if that’s going to inadvertently throw some people out of compliance.

Yeah, so MCELog has a problem with Zen architectures for some reason. I remember running into this when I was on Fedora. I just yeeted the daemon. This was back when I was on a 1700x.

1 Like

Actually, its disabled. But I get more annoying errors about it being turned off.


1 Like

what does the MCELog even do?

What does Machine Check Exception Logging Daemon even do?


Most of the time, you’ll get ECC or memory problem reports here, but it’s really any hardware error the kernel can detect will be noted in that log.

Perfect so I can ignore it. Any hardware issues I use my iLO for.

1 Like

Yep, basically.

My hardware error detector is segfault -> memcheck -> stability test

So my boss is working on hiring a company to do a security audit of us and I can’t wait. We have some really bad security practices here for the sake of convenience.

1 Like

Here’s a fun one.

GitLab released a package build for el8 but has not released a complimentary package for the gitlab-runner because in el8 RedHat dropped support for Docker in favor of Podman.

Now podman is great but there was a lot of infrastructure build up around that.

I managed to get gitlab-runner installed via the rpm script by telling it to install the el7 version of the package and it seems to work just fine. The issue is that I cannot use the docker executor when registering a runner because the way the runners are built they need to communicate with the docker daemon, which podman doesn’t have.

So I am currently experimenting with using the shell /ssh executors.

tldr; if you still want to use GitLab runners with docker then you need to stay on el7 or switch to Debian.

1 Like

Does the runner need to be on the same machine as gitlab?

It does not.

This sounds like a PSA that Gitlab should be aware of.