See, the problem with ZFS implementation of dedup is that every write (and read, for that matter) needs to do a dedup table search, and it needs to compare the csum of the block to write with all existing blocks. It then either adds the block to the table or references an existing block.
BTRFS has a much more sane solution for dedup, where it scans and dereferences blocks on a schedule. This allows fast writes, fast reads and no huge memory requirements.
Reliability:
block checksums
parity
Copy on Write
ZFS Intent Log (ZIL)
As far as the features you really want, you’ll probably get good use out of snapshots, which are deduplicated even without the dedup feature enabled. That’s the nature of Copy on Write. Additionally, the ability to zfs send and zfs recv data to and from pools, over UNIX pipes is really handy. Means you can zfs send rpool/dataset | ssh remote.zfs.system | zfs recv rpool/dataset and send an entire dataset to another server.
The other features I found to be nice is transparent compression and native encryption. You can tell zfs to compress the entire dataset with gzip, lzma, bzip or zstd (as of the most recent version, IIRC), and it’s 100% transparent to the user. Encryption works similarly. There’s no need to cryptsetup luksopen /dev/sdc, it’s just zfs load-key rpool/dataset and then zfs mount -a.
I don’t know what the sweet spot is, but the limitations are boundless. Wendell has a 192TB zfs pool for Level1, and that’s honestly probably not anywhere near the biggest pool he’s worked with.
Linus uses ZFS on his “petabyte project” pools I think, but I’m not 100% sure.
The maximum pool size is 256 quadrillion zebibytes (2128 bytes), so any pool you can reasonably construct, ZFS won’t even sneeze at.
I will remember this. Im assuming there is some fine tuning to this?
Awesomesauce. I would put that to good use incase anything dumb happened with my data or I made a mistake administering something
Sweet… Thats pretty convenient. Yeah I might go ZFS
This is pretty much a must for me. You know me. I even encrypt stuff that makes no sense to encrypt.
IDK how I feel about facebook code but its open so it cant hurt. They have done some interesting math with this
So theres a high likely hood it may just be one single array data set (Z2). I will assign space through LXC of my stuff such as nextcloud and collabora. Otherwise generally store data on it for Jellyfin. (see above… moving away from plex and emby)
Sometimes I love his insanity
What I meant was price to space ratio max gigabytes for money spent economically
Parity is just if you’re running raidz, and the only tuning is if it’s 1 2 or 3 drive loss tolerances.
CoW is on or off.
Checksums, I don’t think you can turn off.
ZIL is definitely something you can tune, but you can also set up a SLOG device (separate log) for this, to improve your overall performance.
Same here. I hate facebook, but you can’t deny that their compression algorithm is the best out there. Arch compresses all their packages with zstd now.
datasets are like thin-provisioned filesystems.
Think of it as a folder, but it has tunable filesystem properties and ZFS can administer it. You should have a dataset for each distinct category of data. I can give you more details if you want.
Used to be 4TB, but it might be going up now. I’d check /r/datahoarder for best results.
It’s worth noting that BTRFS is a worthy contender if you use raid10 instead of parity. (their parity implementation is broken and the write hole still exists, IIRC)
BTRFS has the benefit of being flexible, but ZFS is working on flexibility soon, I think, with device removal or raidz resizing.
Numbers 0-3 are Ports 4 to 1 as labeled on the unit, 5 is the internal connection to the router itself. Don’t be fooled: Port 1 on the unit is number 3 when configuring VLANs. vlan0 = eth0.0, vlan1 = eth0.1 and so on.
At the end of the day its math how do you store data in less bytes via manipulation of the binary data its hard stuff. Huffman coding is useful to index where the literal sections of the math start
In a true me fashion. Every device extends from the domain named after the Viking tree of life
Firewall=heimdallr.yggdrasil
GovLaptopVLAN=Loki.yggdrasil
Mylaptop(soon)=bragi.yggdrasil
Server=Odin.yggdrasil
RaspberryPi4 smarthome and DNS adblocker hub=baldr.yggdrasil
Switch+wireless=Himinbjörg.yggdrasil
5GHz net=Asgard.yggdrasil
2.4GHz net = Nidavellir.yggdrasil
The bridging protocol holding it altogether=bi_frost.yggdrasil
AuthenticationServer=Freya.yggdrasil
Got a few more things to setup and a few more things to acquire and ill have my own encrypted cloud going. Fuck google. Excuse my french
Wow impressive work, especially with the network setup and phone. From doing it myself I can tell you that running nextcloud will be well worth it, even if it can be a bit of a pain to get it to play with other services. Maybe this will be of interest to you, a security focused nextcloud install(on freenas though).
Two questions: Are you going to have it face to the web for remote access? And is there a landline in the picture? A lot of them seem to rely on the ISPs router.
[eric@odin ~]$ sudo ausearch -c 'rtkit-daemon' --raw | audit2allow -M my-rtkitdaemon # semodule -X 300 -i my-rtkitdaemon.pp
[sudo] password for eric:
******************** IMPORTANT ***********************
To make this policy package active, execute:
semodule -i my-rtkitdaemon.pp
[eric@odin ~]$ sudo semodule -i my-rtkitdaemon.pp
[eric@odin ~]$
SELinux has never been easier and I love it
Security and Convenience? Why not both?
For those unfamiliar
What is SELinux:
Perhaps the most useful description is thinking of it as a framework for managing access control to files and other resources in the Linux system that go far beyond standard file permissions or aces control lists. As if chown and chmod werent already a headache
Is there a Presentation I can watch?
Yes, RHEL has one. The video goes into its difficulties but also why its good.
Whats my issue?
My issue before was inconvenience. I never had a testing and production system. You want an air gapped testing system so that you dont have to remove protections from your production machine to fix the issue. Cockpit mostly solves that including for stuff that doesnt like SELinux (like PiHole)
Interesting 2 antenna 1s a 2 and a 3. This may shed light on how the code the mu-mimo and how the array transmits and recieves. Ill dig into that later after a before and after testing of stock vs professional antennas
Alright guys heres the AP setup for signal strength testing. You will notice I am going to keep the power the same and keep DTIM and beacon interval the same so to not skew the tests as well as disabled disassociate on low ACK… When I run my tests you will see the difference between true professional antennas and the basic stocks.
Set Delivery Transaction Indication Message Interval to 1
Channel Power (Max Regulatory 30 dBm-1Watt)
Please note I will change the TX power to accurate reflect EIRP to be within legal bounds later. Im sure a test or two wont upset HAM folk
Reason for testing is for science
Notes about high gain antennas; If you change the following parameters via antennas you can no longer guarantee you are operating within FCC Spec and proper EIRP:
Gain
Output S parameters (output VSWR etc)
Q factor
Impedance
Beamwidth
Its important to realize if you do get higher gain antennas the polite thing to do is to turn down the power and try to stay in spec. Its not nice to operators or your neighbors. Most consumer and even prosumer radios do not have the abilit yot check the matching and all the parameters above, only high end Amatuer radios. They are programmed with whatever they got from the factory!