Sysadmin Mega Thread

The default behavior is automatic failover but this is not possible on DenseIO because of the pass-through direct hardware.

What they do in these scenarios is that OCI will send out a notification with an intent to evacuate the virtual machine. Basically, all we have to do is make sure our data is backed up and then reboot the vm manually.

The issue was that the email went to the head of security for some reason instead of OPS and they did not let us know of such a thing so we got caught with out pants down.

Vm failover, sure but can’t the actual database fail over?

No. We have Master->Slave replication.

What we do is we manually promote the slave and then re-seed.

The issue was how the hung VM fucked us up because we had to verify the slave’s data wasn’t borked.

We were concerned the master sent corrupted binlogs.

on prem exchange and skype for business, how i hate thee

also lol at ms suggested vs. real world CPU physical to virtual core ratios supported by exchange :smiley:

also cry at doing cumulative updates during an outage window when every other fucking thing on the cluster is getting backed up and the SAN is getting absolutely hammered

1 Like

Anyone played with Stratis?

https://stratis-storage.github.io/

FYI : Let's Encrypt is switching his root cert, will break andoid <7.1

We will have unhappy Samsung TV :frowning:

2 Likes

Still works as a TV though.

Connect an RPi to it for home theatre.

I can see the disappointment because its a smart TV but the burden is with the manufacturer abandoning support for the older OS. Don’t be mad at the security experts.

1 Like

FUCK auditd :fu:

Holy shit what hot trash.

2 Likes

no, the issue is that we have over 10K TV fetching url to show movie poster and other movie theater signage… around europe

We won’t afford the replacement cost and already had to throw some of them away because they didn’t took the full chain we were sending, and one of the intermediary on there catalogue was expired.

Happily there is a way to bypass that for now.

Then you need to ride Samsung’s (of which ever vendor) ass about getting updates or the person who had the bright idea to use a device that becomes deprecated after only a few years due to security should have a tongue lashing because due diligence of security was not thought of.

If you did purchase devices without a service contract – how did that ever fly by management? Gross oversight. Because now that shit is hitting the fan you are left holding the bag. I feel for you.

2 Likes

Partition fill up?

all the immutable shit man

adds so much extra boilerplate to make a change via automation tools

1 Like

After using GCP for two months.

WTF I LOVE GOOGLE CLOUD NOW

It fucking rolls up and smokes AWS and Azure.

2 Likes

What’s immutable? The auditd config is once it’s started but other than that it just logs stuff (everything), right?

once auditctl -e 2 is turned on its impossible to kill the daemon unless you kill the pid as root. So you have to change this, update the rules, reload the daemon, turn back on.

This is a few for loops and if conditionals I have to do with out tools which is a pain since its different for both ol6 and ol7.

1 Like

Protip, OCI is great but don’t build anything that is not ephemeral on the DenseIO shapes.

They crash like fucking crazy lately.

Yeah. I think if selinux is on, reconfiguring auditd is a mandatory reboot in Fedora so look forward to that.

I think the idea is that auditd config almost never changes after onboarding the server. What are you changing about it?

We are shifting from the use of Auditd to Tanium.

Unfortunately, its a progressive cut over so I had had to bake in additional logic to allow both of them to live together in the environment. :crazy_face:

Auditd likes to be the single source of truth so its been a royal pain in the ass.

1 Like

I have no idea to be honest, i don’t manage those (happily) but it usually goes a something like

Hey your server suck the client no-longer have the webpage on the TV

Queue the 2 day of mail back and forward to get them to fix there shit …

Rip my VPS uptime

2 Likes