Token's lvl1 blog- edit -- Token's rantings

[Token]

@oO.o

Did you not like Proxmox? I feel like you considered it but I don’t remember the specifics.

I ran proxmox for about a day a few years ago when I really REALLY didn’t know jack so I should run it again to give it a fair shake. How do you feel about a nested instance just to get a taste instead of bare metal?

I tried to spin up a CentOS7 minimal ISO I’ve used many times. Found some interesting results.

Basically I have to use a tag-team combo of XCP-ng Center and Xen Orchestra to get the CPU and storage settings right.

• edit, interesting, xcp-ng always puts a chunk of hypervisor local memory into the VM, I have not been able to delete it during setup. Xen Orchestra gets it right.

I want to rock opensource, but I think I need something more mature- I say this but I’m not gingerly walking over and rebooting my box with the ESXi disks- I want Xen to work, I’m liking a lot about it.

I will have to do some kind of tag-team setup between both GUIs just to make a simple VM lol, but still, what a day to be alive, this is free.

For fun I did an OBS screen capture of installing the same ISO on VMware workstation pro (i7 desktop, Samsung SSD) and on my type 1 (R710) with NFS as the storage. They were pretty much neck and neck. Workstation beat the type1, but narrowly. Pretty happy with that. I was really afraid of NFS but I guess its a concern only for legit datacenters. For home-labbers a twin 1G nic is probably enough to slum it, a quad better. I’ve been watching netdata on FreeNAS and there are small moments where it spikes to ~.60 to 1.27 gigabits but its very brief.

For those that want to get into this with spare parts, heck even a 1G connection should be fine, look at all the valley space vs. peaks. I’ll try to get another shot when I have the VM actually doing some work. I think something that graph won’t show is lag when I start to query from a DB on that VM. I’m going to have to feel it with the butt dyno as gear heads say- how the new build feels vs. when I had a similar build on SSDs on ESXi (local storage).

I couldn’t help it, this came to mind

[oO.o]

@Token

How do you feel about a nested instance just to get a taste instead of bare metal?

Idk, probably fine…

ESXi is still probably the easiest no matter what… Proxmox is a close second though. I’ve stayed away from xen just because I didn’t want to learn a whole new thing.

[Token]

For anyone interested in Xen, XCP-ng, this post does a great job of listing the projects out, little bit of history, URLs- short and sweet, really well done.

What is what: XCP-ng, XCP-ng Center, XenOrchestra, XOA, Vates

[Token]

So I saw a squirrel and am sidetracked.

Hunting down why the TA-pfsense add-on for Splunk only sourcetypes (from memory) I think 3 sources correctly, the others get skipped.

The regex and following stanzas in the transforms to ‘tell’ the app how to assign a sourcetype is:

REGEX = \w{3}\s+\d{1,2}\s\d{2}:\d{2}:\d{2}\s(?:[\w.]+\s)?(\w+)
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::pfsense:$1

If I’m reading into this correctly, it takes from the pfSense log output (example):

Apr 14 17:44:22 pfSense filterlog: blah blah

It gets through the date time, see’s pfSense and takes the following word after the dead space and passes that along to be appended to the sourcetype Splunk assings- this is then useful for the props.conf stanzas- so that the right regex for field extraction is used for the right type of log (filterlog, openVPN etc).

Here is the issue, all of the logs when viewed directly from pfsense (putty) have that standard date time format. But in Splunk a lot of these logs other than filterlog and two others seem to come over the wire without that date time, so it breaks the transforms and the logs get labeled simply “sourcetype=pfsense”. Subsequently no field extraction.

Looking at the raw logs in Splunk show no date/time in the raw:

VS an example of a good log (has date/time in the raw portion and therefor transformed and props applied)

So to the admin gurus out there, is there a method short of standing up a syslog server to view the logs as they come in from pfsense? Some netcat command (this is one a CentOS7 box- to remove Splunk for the equation I can re-direct syslog to a vanilla CentOS7 VM I have) to temporarily take in these logs? I want to see if pfsense is sending them over that way, or if Splunk is somehow stripping this off before getting to that transforms.conf.

I also triple checked pfSense’s log settings to see if there was any setting that stood out about outputting date/time from the logs- I didn’t find anything but it could be right under my nose and I missed it…

[oO.o]

You should ask this in sysadmin mega

[Token]

Good idea

[nx2l]

Setting up rsyslog to receive on centos is simple.

Or were you saying you didn’t want that… Only had an hour if sleep so far

[Token]

I’m thinking that is what will have to be done, set it up on my other Cent VM, point pfSense syslog output to it, make sure the firewall is good with the port, and the tail the log.

I was just thinking there must be a slicker way of doing this, something like netcat. Teh googles is a little dry on this, I’m probably using horrible search terms.

[Token]

And speaking of Splunk, their COVID dashboard for all to view:

https://covid-19.splunkforgood.com/coronavirus__covid_19_

[oO.o]

Wish they would include a per capita graph.

[Token]

Also, I wish they could get some kind of integration, API I guess with google’s dataset here:

COVID-19 Community Mobility Report

They could make some really interesting correlations with people’s movement activity.

[Token]

@oO.o

Yeah, sync sucks (ha) when it comes to performance. It’s probably more critical for a database than a vm but is still recommended for VMs…

This was super informative:

I really really had it out to run iscsi, but I was seeing a lot of ‘disclaimer’ ‘read-here’ fine print with running iscsi on FreeNAS that is over my head. I got the impression you really have to have your ducks in a row to run iscsi on FreeNAS. I would probably tank my whole NAS because I derp these things, so I went with NFS.

That said I’m really curious on your opinion with this- Splunk swears up and down do not use NFS, data will get corrupted. Would that be the case when the NFS storage is on top of a file format like ext4 or NTFS? Would data inherently be safer when NFS is being served on-top of ZFS or is this a concern inherently simply to the nature of NFS and not in relation to the format it sits on?

[Token]

Holly cr*p. Took me all of today (and a bit of last night re-imaging the Pi) to setup rysnc in a less than usual orientation- From FreeNAS to a Raspi with 5 TB HDD.

I wish I documented it better because even though I just finished doing it, I don’t even really know what I did LOL. Its all a blur now.

Still learned a lot by reading countless posts and really informative sections of the FreeNAS guide.

In typical Pi fashion, it has so many changes over the months, tutorials even less than a year old will still be wrong. I had to do so many things to the Pi to get this working, but surprisingly not install or start rsync. Raspian Lite right out of the box has rsync installed and running (reading for a PULL).

My key takeways:

Understand system sending data is the client and referred to as PUSH

System receiving data is server and therefore PULL

Then use the FreeNAS guide to understand you do not need to enable rsync in services. That in tasks you setup an rsync task, and in the drop down select PUSH.

Then, follow the FreeNAS documentation/guide (not the forums, so full of outdated info) to generate a user (root in this case, but I would like to change this to a specific new user for this task) public/private key pair.

Follow Raspberry tutorials to:
allow ssh
allow root ssh (until I change this)
Copy over public key from FreeNAS (not just paste)
Copy from Pi host key to FreeNAS
Edit Raspi ssh config to allow no-password key based SSH

And I’m sure I’m missing a lot of stuff.

Was so much learning, including using very specific key word searches because again, the vast majority of stuff out there is about rysnc from FreeNAS to FreeNAS (in which case IMO use ZFS replication) or from linux or other box to storage in FreeNAS. Its less common to rysnc stuff off of FreeNAS.

Now crossing my fingers that as the data uncompressed onto the ext4 format that my drive is big enough, I’m afraid its not.

Anyone know a trick to enable and tail a log of the rsync progress? For now I just check in the FreeNAS GUI that the rsync task is “running” in status, and on the Pi I just ls’ed the path the data is going to (and its filling up so thats a good sign).

[Token]

Trying to find a battery with similar fitment as a 6.5v BA-5372/U in case whatever this gets used for later needs a battery source.

12v A23 is the winner so far, but it’s pretty small and loose in there.

Have connector parts:

Now to find an actual use case as these boxes are too cool to just get tossed to the side.

[mutation666]

You want the battery cover on? Cuz you could just make a custom adapter that fits in and allows you to use w/e battery outside the case.

[Token]

A little foam on the battery cover inside and I think it will hold down the a23 pretty well. Or maybe no use case for a battery at all. Just figured I would plug away a bit and see what fits if I end up finding a project that could use it. Would be neat to reuse that part as it’s intended.

Any projects come to mind that can use a water proof case, 6 lead or less harness and the box has an easy access door to the inside?

[mutation666]

Maybe password strips figure out what the output from the pins looks like then have a device on the other side do a translation

[Token]

Oups should have given context, these are gutted and stripped!!! NSA look away, no crypto here!!!

I got my hands on a couple empty shell cases and you would think there are 101 diy projects that could use this box but I’m coming up blank.

[mutation666]

Geo Cache?

[Token]

I want to keep it though