Hypothetical: ZFS, ARC and non-ECC memory

Operating Systems & Open Source
1

I have a question about ZFS and ARC that I’m hoping someone might know the answer to. Is the data stored in the RAM verified for integrity with a checksum like the data on disc, or do the checksums only apply to the data that’s on disc.

Example:
Lets say I have a Truenas server at home storing my data and the non-ECC memory is full up with ARC. If a cosmic ray happens to flip a bit in my RAM, and I then happen to request that data that was subject to said bit flip, what happens? Will ZFS read the data in the arc as having an integrity issue and reload the data from disc, or will it give me the data with a bit flip?

2

ZFS trusts the RAM is good, it defaults to not checking but you can turn it on.

Load zfs module with modprobe zfs zfs_flags=16, or boot with zfs.zfs_flags=16.

man 4 zfs:

zfs_flags=0 (int)
     Set additional debugging flags.  The following flags may be bitwise-ored together:
     ...
     │  16  ZFS_DEBUG_MODIFY  Check for illegally modified ARC buffers.  │
     ...

Verify with cat /sys/module/zfs/parameters/zfs_flags.

Source reference: zfs/arc.c at zfs-2.1-release · openzfs/zfs · GitHub

2 Likes
3

ZFS actually trusts something???

Thanks for the info, I’ve been curious about this for a while. I don’t want to mess around with debug flags to try and make my regular memory into ECC. I just see a bit flip in the cache over time as the most likely issue with non-ECC memory.

4

Yeh, suprising huh? :slight_smile: You’d think it’d do every operation at least twice just to make sure the CPU is working correctly :slight_smile:

Due to the effectiveness of the ARC algorithm above contemporary caching strategies, I think the age of data in the ARC was a concern that born the “ECC required” myth, but considering the number of problems I’ve had with memory over the years (and the hard evidence in the form of oscilloscope traces), I’m keeping my ECC :slight_smile:

5

My servers are old desktops, so they often aren’t ECC. My current desktop is ECC, so the next generation of server will be. But that’s like 5-10 years from now.

1 Like