oO.o's Neverending Tech Blog

mutation666 · June 12, 2023, 11:56am

That looks clean

oO.o · June 22, 2023, 5:54am

Somewhat reluctantly sticking with oVirt on CentOS Stream 8, but going with a vanilla OS install instead of their appliance iso to avoid LVM thin provisioning and generally reduce the footprint somewhat.

Here is my kickstart. Installs onto a Supermicro SataDOM, brings up the first onboard NIC and deploys my SSH key. I believe it should work with any RHEL distro.

clearpart --drives=disk/by-id/ata-SuperMicro_SSD* --all

zerombr

ignoredisk --only-use=disk/by-id/ata-SuperMicro_SSD*

bootloader --location=mbr

#autopart --type=lvm --fstype=ext4
part /boot --fstype=ext4 --asprimary --size=4000
part pv.01 --asprimary --size=100000 --grow

volgroup vg-001 pv.01

logvol / --vgname=vg-001 --name=root --size=32000 --fstype=ext4
logvol swap --recommended --vgname=vg-001 --name=swap
logvol /tmp --vgname=vg-001 --name=tmp --size=8000 --fstype=ext4
logvol /var --vgname=vg-001 --name=var --size=8000 --fstype=ext4 --grow
logvol /var/tmp --vgname=vg-001 --name=var-tmp --size=10000 --fstype=ext4
logvol /var/log --vgname=vg-001 --name=var-log --size=8000 --fstype=ext4
logvol /var/log/audit --vgname=vg-001 --name=var-log-audit --size=4000 --fstype=ext4
logvol /home --vgname=vg-001 --name=home --size=1000 --fstype=ext4

selinux --enforcing

rootpw --lock

network --bootproto=dhcp --device=eno1

keyboard us

lang en_US

timezone --utc America/New_York

%packages --excludeWeakdeps
@^Minimal Install
openssh-server
%end

sshkey --username=root "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPoWEqeFgAsri67X5zNkT51frC
ZLzvfTAufpCqFaQdu7CuEmtI69t/3SlqzmZVokbH0q6OL1XQzrCFzKdgmQbdylW7AoLSEThT44BOrh+nkTO8sp89
kT3HTdRt7zmhMgN0tCrIZn5Y3Tz+Vb5vBkhFWSborIiJb9EkKx6ERDY2SR8TWGeyefoTJhx2V+Q1ULTaCEQX3I9C
0YGRybJ5x9jPGvhHlaACRBAGecBPi3ortOICZnbyMpe239gq/UIILHCET7b6kVW5FnYUa6ljSl0nrdUC7ho4DPbb
zAr88mSU9y1q/JrnYvjr6W5D2WIeZNXmxXloZD+4MxufQNUNB2BmsaL5+FhPC2tnTazWWqWPlaO3us6kG1bvbR1W
lcIxChJ4xwLOBuyKTyCFak0+hG+2CXpaJ9X37LXcYgNljfA4895Dj2HKTH/vTch8+bdpN60TDd3htWLQuipecf9x
1pM563Llsql9y66i6MV0cSLhfA2AsMTup0VYpqQb/D3HAag6rdxOJUqSA6wSmOmTrdWYHKiyJkjcBOHsQ4vFcwHk
WXpJdOmh4fV+zTWpkc2vjLUMUZy91M4/zAAIIb6fB6jo2t3sSc9frv0ScefcGbOSlaUEyBhRP8DKOO/Twl9Eq1Q9
g+vtmFvMJvlXaVjackxd7bvGjHLRrliAHmWom/woIw=="

services --enabled=sshd

reboot --eject

oO.o · July 3, 2023, 2:11pm

Screenshot 2023-07-03 at 10.10.50 AM

oO.o · July 4, 2023, 12:28am

@PhaseLockedLoop do you know the syntax for a /23 classless PTR zone? I usually stick with /24s but I need a big boy and I remember being thoroughly frustrated by classless PTR syntax.

I think I just do 2 class c reverse zones. It’s the smaller subnets that get weird.

PhaseLockedLoop · July 4, 2023, 2:09am

I havent written that one before but this should help

As should RFC2317

Hope that helps

oO.o · July 5, 2023, 2:54pm

Finally implementing SSHFP. Feels good.

oO.o · July 20, 2023, 5:47am

I’m done. No more RHEL.

https://www.reddit.com/r/AlmaLinux/comments/1544w8b/red_hat_refuses_almas_cve_patches_to_centos/

I think I can break completely from their ecosystem with the exception of Ansible.

Suse and NixOS are new areas of interest for me. Relying heavily on Debian currently.

oO.o · July 21, 2023, 6:14am

I mostly have relayd working as a reverse proxy with a wildcard Letsencrypt certificate. I have 3 lingering issues:

Miktrotik admin interface refuses to work. It throws TLS handshake error: handshake failed: error:1400442E:SSL routines:CONNECT_CR_SRVR_HELLO:tlsv1 alert protocol version, GET: Undefined error: 0 which appears to be an error relating to incompatibility between the browser and TLS but this is working fine on other nearly identical forwarding instances, so I’m confused… In this case, relayd is relaying to HTTP, so there is no TLS between the proxy and server at all and TLS is resolving fine in every other instance between browser and proxy.
noVNC doesn’t work in Proxmox while using the reverse proxy.
Various issues with checks in relayd. For proxmox, HTTPS code 200 checks do not work and might be an issue of checks using HTTP 1.0 vs HTTP 1.1. However, TLS check works fine for Proxmox. I could not get HTTPS or TLS checks to work with IPMI HTTPS interfaces despite no issues while using curl.

Also of note, relayd does not support ecdsa which is disappointing in 2023. That said, I’m still using RSA for SSH so whatever…

ThatGuyB · July 21, 2023, 9:30pm

It’s cool that you’re using OpenBASED, but are you sure you don’t want to use HAProxy and use relayd instead? I have a feeling you might have a better time with the former.

I think the noVNC console fails because it needs to use some additional ports that aren’t redirected (and probably because of the http version as well, but I can’t tell). HAProxy might be able to get around that easier.

oO.o · July 24, 2023, 5:29am

We’ll see! I am not so ideological about using OpenBSD that I will avoid using a clearly superior solution but I will put some effort into making the openbsd solution work. In this case, I have not yet put in the requisite effort to write off relayd entirely, but I might fall back on haproxy if relayd is not suitable.

oO.o · July 25, 2023, 8:39pm

Yeah I don’t know enough about HTTP and websocket to figure out how to make relayd work here unfortunately. Might come back to it, but indeed haproxy works so long as you specify balance source so that the novnc session is not load balanced to a different host.

Nope, relayd is working!

This was the problem:

match request header set "Connection" value "close"

Otherwise you need http websockets and mode source-hash and it’s all good.

oO.o · July 26, 2023, 5:45am

This caused some frustration this evening: Certificate validation fails when the last certificate of the chain is signed by an expired CA (DST Root CA X3) · Issue #606 · ipxe/ipxe · GitHub

Also, OpenBSD PSA:

zenbleed fix requires additional update procedure. I don’t know if this is necessary if you aren’t running an affected processor, but better safe than sorry since the consequences are a system that does not boot…

#!/usr/bin/env sh

set -eux

{       [ $(whoami) = 'root' ]
        syspatch
        pkg_add -u
        fw_update
        installboot -v sd0 #change to your boot drive
        echo SUCCESS
} || {
        echo FAIL
        exit 1
}

#Reboot at your discretion

Maybe not a bad idea to do this on all updates tbh

oO.o · July 26, 2023, 4:04pm

So I have the gateways serving some files over http to facilitate iPXE installs, and that creating some headache with relayd. relayd has one very frustrating behavior if you’re forwarding to multiple hosts via SNI on the same interface, some of the forwarding arguments appear to be applied universally. For instance, I cannot forward to a combination of HTTP and HTTPS backends on the same interface. Transparency also was all or nothing. Checks were not though? It’s weird and I wonder if it’s a bug or I’m doing something wrong.

Anyway, for one relay, I want to essentially pass it through to httpd running on the localhost. But since relayd is bound to that interface, it has to act as the intermediary. So my instinct was to run httpd on 127.0.0.1 and relay to that. But I need a certificate because as mentioned, it’s all HTTP or HTTPS and all the other SNI-defined backends for that interface are HTTPS (self-signed). I could not figure out how to generate a self-signed certificate that httpd would accept for the localhost interface. I’m sure there’s a way to do it, but I could not figure it out. Instead, I set a fqdn in /etc/hosts ending in my Let’s Encrypt wildcard domain for 127.0.0.1 which tricked httpd into loading the Let’s Encrypt certificate. Now this works but only if transparency is disabled, so I’m living with that for now.

Anyway, config looks like this:

interval 3
log state changes
log connection errors
prefork 3

table <localhost> { localhost.my.letsencrypt.domain } #127.0.0.1 (defined in /etc/hosts on the relayd host)
table <oobm1> { 192.168.1.11 }
table <oobm2> { 192.168.1.12 }
table <oobm3> { 192.168.1.13 }
table <proxmox> { 192.168.2.11 192.168.2.12 192.168.2.13 }

http protocol "https" {

        return error

        match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
        match request header append "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT"
        #match request header set "Connection" value "close" #Breaks novnc

        match response header set "Strict-Transport-Security" value "max-age=15552000; includeSubDomains; preload"
        match response header set "X-Content-Type-Options" value "nosniff"
        match response header set "X-Frame-Options" value "SAMEORIGIN"
        match response header set "X-XSS-Protection" value "1; mode=block"

        tcp sack
        tcp nodelay

        http websockets

        tls keypair "my.letsencrypt.domain"
        tls ciphers HIGH
        #tls no tlsv1.2
        #tls tlsv1

        match request header "Host" value "www.my.letsencrypt.domain" forward to <localhost>
        match request header "Host" value "oobm1.my.letsencrypt.domain" forward to <oobm1>
        match request header "Host" value "oobm2.my.letsencrypt.domain" forward to <oobm2>
        match request header "Host" value "oobm3.my.letsencrypt.domain" forward to <oobm3>
        match request header "Host" value "proxmox.my.letsencrypt.domain" forward to <proxmox>
}
relay "lan" {
        listen on 192.168.1.1 port https tls
        protocol "https"
        forward with tls to <localhost> port https check https "/" code 200
        forward with tls to <oobm1> port https check tcp #IPMI doesn't like relayd's https or tls checks
        forward with tls to <oobm2> port https check tcp
        forward with tls to <oobm3> port https check tcp
}
relay "proxmox" {
        listen on 192.168.2.1 port https tls
        protocol "https"
        transparent forward with tls to <proxmox> port https mode source-hash check tls #proxmox also doesn't like relayd's https check
}

DagobahPrime · July 27, 2023, 11:01am

yikes
I barely touched code but it sounds like poor scope handling when declaring some object or function.
Source code?

oO.o · July 27, 2023, 2:38pm

Yeah I don’t know. OpenBSD devs make design choices very deliberately so I’m sure there’s some reason but it definitely will be a deal breaker for a lot of people coming from haproxy. Luckily in my case, everything I am using has or can easily be equipped with self-signed certificates, and it’s always nice to keep things encrypted anyway, even behind the proxy.

oO.o · July 27, 2023, 5:25pm

ThatGuyB · July 27, 2023, 11:42pm

Lol, I had to deal with that at work, although not with ipxe. Funny how the certificates were valid, so no alerts triggered, but the CA expired and everything went to . Who thought it was a good idea to make a CA, but not set alerts when it expires?..

oO.o · August 18, 2023, 4:59pm

I finally saved my random word 1-liner to a script instead of searching for it in history.

sed -n "$(( $RANDOM$RANDOM % $(wc -l /usr/share/dict/words | awk '{print $1}' ) ))p" /usr/share/dict/words

I just realized I had actually already done under a different name and forgot…

Dynamic_Gravity · August 18, 2023, 5:12pm

I’m stealing this

DagobahPrime · August 18, 2023, 5:14pm

Sir? wtf is this?