Need guidance around setting up drives with ability to handle "bit rot" without needing a parity drive if possible

Dude, I think you’re wrong again. No?

EDIT:

Dude, I conceded that one failed VDEV will destroy the pool.

However, this deviated very much from your original argument that a single pool of multiple single-drive VDEV is not possible. You have yet to acknowledge you were wrong.

Now whether to proceed with ZFS without parity disks is up for debate which I was and am not interested in at all.

In what way? How would you expect such a pool to work?

It’s just a RAID0. Stripe vdev+stripe vdev+stripe vdev+etc.

So, I was not wrong “again” then.

I said no such thing, which you will see if you read my original post again.

Also, please stop calling me “Dude”. It’s disrespectful.

1 Like

You’re moving the goal post to win an argument.

If you were born 1978, I’m older than you. Respect is mutual. See? I think Americans consider ‘dude’ neutral.

I’m currently looking into btrfs and the issues I’m facing with it. This post on reddit really got me a bit paranoid about going with zfs https://www.reddit.com/r/zfs/comments/ynjwit/comment/iv9kc2g/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

I already spend too much time trying to sort out my nas issues. Once I get more comfortable with it I will definitely look into it.

For now my rsync issue seems to be constant.

@homeserver78

The “critical target error” seems to be btrfs-specific? I only get a single hit doing a web search. Not much to go on, unfortunately.

Could it be the checksum option blake that I selected that could be the cause? For the time being I’m going to attach both the drives to my more powerful local machine and directly to sata ports. Will test out the same rsync command and see if I’m still facing the same issue. The good thing is at least for the time being the older drive still seems to be able to read and write. I have a feeling it is mostly the checksum issue but I could be wrong.

I tried a new usb type c cable for the mediasonic enclosure but faced the same issue again. I just hope both drives are healthy. I’m liking btrfs for the time being… Being able to check the condition of the drive and it’s data is quite handy

The old drive on fedora is showing an error like the earlier 14TB “DISK IS LIKELY TO FAIL SOON”

I ran the smartctl command and I’m guessing it’s because of these two properties having a number 0. Do I need to start the return process for this drive too? Is there a way for me to check for bad sectors in linux? (ignore… found a command called badblocks which I think will do it for me

197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       1
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       1

Full output is below

smartctl 7.5 2025-04-30 r5714 [x86_64-linux-6.14.11-300.fc42.x86_64] (local build)
Copyright (C) 2002-25, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Exos X18
Device Model:     ST18000NM000J-2TV103
Serial Number:    ---
LU WWN Device Id: 5 000c50 0f19c2374
Firmware Version: SN04
User Capacity:    18,000,207,937,536 bytes [18.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database 7.5/5706
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sun Jul  6 18:55:42 2025 BST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82)	Offline data collection activity
					was completed without error.
					Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0)	The previous self-test routine completed
					without error or no self-test has ever 
					been run.
Total time to complete Offline 
data collection: 		(  567) seconds.
Offline data collection
capabilities: 			 (0x7b) SMART execute Offline immediate.
					Auto Offline data collection on/off support.
					Suspend Offline collection upon new
					command.
					Offline surface scan supported.
					Self-test supported.
					Conveyance Self-test supported.
					Selective Self-test supported.
SMART capabilities:            (0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:        (0x01)	Error logging supported.
					General Purpose Logging supported.
Short self-test routine 
recommended polling time: 	 (   1) minutes.
Extended self-test routine
recommended polling time: 	 (1583) minutes.
Conveyance self-test routine
recommended polling time: 	 (   2) minutes.
SCT capabilities: 	       (0x70bd)	SCT Status supported.
					SCT Error Recovery Control supported.
					SCT Feature Control supported.
					SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   084   064   044    Pre-fail  Always       -       0/225401043
  3 Spin_Up_Time            0x0003   093   092   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       27
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   080   060   045    Pre-fail  Always       -       0/93711426
  9 Power_On_Hours          0x0032   094   094   000    Old_age   Always       -       5911
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       22
 18 Head_Health             0x000b   100   100   050    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   098   000    Old_age   Always       -       126 126 129
190 Airflow_Temperature_Cel 0x0022   057   049   000    Old_age   Always       -       43 (Min/Max 39/43)
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       11
193 Load_Cycle_Count        0x0032   092   092   000    Old_age   Always       -       17666
194 Temperature_Celsius     0x0022   043   051   000    Old_age   Always       -       43 (0 23 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       1
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       1
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Pressure_Limit          0x0023   100   100   001    Pre-fail  Always       -       0
240 Head_Flying_Hours       0x0000   100   100   000    Old_age   Offline      -       2920h+07m+30.638s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       46780073512
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       60974574937

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%         0         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more

I know almost nothing about btrfs so I can’t comment on that, really. I don’t see why a checksum option would cause that error though. I would certainly suspect any USB controllers first, that’s for sure. So it’s great that you’re trying with a direct SATA connection!

When it comes to ZFS, I can mostly agree with the points in the post you linked to. ZFS was made for the data center, and it shows. Many points are not very important to me though (I don’t need the absolute latest kernel for instance; in fact I like to use a “longterm” kernel on my server). Some points are “mostly” wrong (“Can’t recompress or uncompress”, “Can’t move data between datasets”), and at least one point has changed since that was written (“No reflink copies” - although there were some nasty bugs related to this one when it was first released. I have no real use for it so haven’t tried reflinks yet.)

But yeah, ZFS is kinda inflexible and I’m not impressed with the code quality: I’ve ran into bugs relating to parsing of command line options (including the --dry-run option doing nothing, not even parsing the options! :facepalm: ), and I don’t like ZFS’ habit of panicking the kernel if something’s wrong rather than aborting the mount/whatever and emitting a kernel Oops.

ZFS’s features are amazing though and worth the tradeoff to me.

Just remember that the typical advice given if something is wrong with your pool is to recreate it and restore from backups. Think about that beforehand and keep it in mind while planning your pool(s) and backup strategy! (You do need backups anyway and ZFS makes backups relatively easy!)

I personally like multiple smaller pools rather than to have all drives in a single pool partly because I don’t fully trust ZFS and partly because of its inflexibility: if something goes wrong it’s quicker/easier to restore just part of the data, and if I want to redo a pool layout it’s very nice to have other pools that I can migrate the data to temporarily. So multiple pools, to me, is a way to relieve ZFS’s inflexibility.

1 Like

So the format of the SMART data is a bit weird. VALUE starts off at 100 (most values) and ticks down when something’s not good. If it reaches the THRESH value it triggers a SMART fail.

I don’t see anything wrong with that drive, although I’d be a little bit vigilant of the Current_Pending_Sector and Offline_Uncorrectable RAW values. I’d like them to be 0 – but it’s possible it’s a remnant from the factory testing and that everything is perfectly fine. I don’t see a reason to return the drive.

Also, rather than badbocks I’d run an long offline smart test. (smartctl -t long <dev> + wait many hours and then check the “SMART Self-test log” with smartctl – where it shows a “Short offline” test in your output above.)

Edit: I have no idea why Fedora would think the drive is about to fail. There’s nothing in the smart data that would entail such a warning, unless I’m missing something. Maybe it’s reacting to controller issues?

One thing you might want to look at once everything else is working fine is the Load Cycle Count. Its VALUE is lower than the Power On Hours VALUE which means you risk wearing out the head ramps before the drive has reached its usable lifetime. Nothing urgent but worth looking at down the line, perhaps.

Perhaps the one pending sector (which AFIAK is a detected bad sector that is pending to be remapped). I had a drive too that had a single pending sector but it was obviously failing (getting IO error 5’s then getting the data off of it, etc.). I also suspect it’s not Fedora .

Is this the empty btrfs drive? If not, My first step would be to copy all data off of the drive if you can, perhaps using ddrescue. After that you can reevaluate if it’s still useable. I say this because running badblocks or a long test on a failing drive might just push it over the edge (especially a large one)

If it’s the empty drive, you could do the long smart test and hope the drive remaps the bad sector and is useable again. I would not be comfortable to use this drive without redundancy or backups again though.

BTW the IO error 5 is from the kernel indicating an error on the device level, it’s nothing to do with btrfs. The same would/could happen if you would dd from/to the device on the block level (I’ve seen that happen before).

2 Likes

@quilt: Any idea how Seagate weighs the Current_Pending_Sector VALUE? I find it weird that the value would be unaffected by a current pending sector. But I can’t say I know the details here. Or that I trust Seagate to do the right thing.

I belive many places in the kernel could issue an IO Error (-5). (I’ve never worked with filesystem code but I did work on other kernel code for several years so I say this out of that experience.) So this error might not mean the same as the one you got (unless it was in the exact same context). That said, I fully agree that the first step should be ensuring working backups of any data OP wants to keep.

This is the older 18TB x18 drive. That had ext4 on it.
The newer 18TB x20 drive is the one that has btrfs on it.
I’m trying to copy all the data from the ext4 onto btrfs and that’s why I’m running the rsync command. All these failures seem to be during the copy and most of them seem to be with write. I had run a full scan on the newer x20 drive and there were no errors on it before I attached it. Have a feeling it could be the mediasonic usb to sata enclosure. I’ve now connected it to my personal machine and trying out on fedora. First I’m running the btrfs scrub command again just to confirm all the data is intact. After that I will run the rsync command again

@homeserver78 That’s exactly what I’m doing first. Copying everything onto the btrfs x20 drive. After that I will run the scans etc on the older x18 drive

I’ll run the command you mentioned above once the copy is done smartctl -t long <dev>

3 Likes

So I did some more reading on this and apparently Seagate’s firmware keeps the VALUE at 100 even with large RAW counts on Current_Pending_Sector, Offline_Uncorrectable, and Reallocated_Sector_Ct. Which prevents the “SMART overall-health self-assessment” from failing as it should. :frowning:

I also found someone with the same “1” raw count on Current_Pending_Sector and Offline_Uncorrectable, also on a Seagate drive. A very old thread, but still. An interesting read but no real conclusion there except on that drive the values does not behave as expected according to the spec.

I would keep a close eye on those values. As long as they don’t increase during use/scans I would probably keep using the drive, making sure to run periodic scrubs and to keep working, up-to-date backups (as should be done with any drive).

BTW, I try to always run smartctl -a <dev> > smartctl-a-<driveid>-<date>.txt on new (to me) drives so I have something to compare later checks to.

1 Like

FWIW I had an 18 TB IronWolf Pro fail SMART a couple years ago with Offline_Uncorrectable = 1. Checked with Seagate and support said RMA the drive, which went without fuss.

2 Likes

FWIW I had an 18 TB IronWolf Pro fail SMART a couple years ago with Offline_Uncorrectable = 1. Checked with Seagate and support said RMA the drive, which went without fuss.

I would keep a close eye on those values. As long as they don’t increase during use/scans I would probably keep using the drive, making sure to run periodic scrubs and to keep working, up-to-date backups (as should be done with any drive).

I bought this drive from serverpartdeals. Will reach out to their customer support simultaneously and check if it’s worth sending it back

BTW, I try to always run smartctl -a <dev> > smartctl-a-<driveid>-<date>.txt on new (to me) drives so I have something to compare later checks to.

Since i’m a windows person I usually run the surface scan in MiniTools Partition wizard.

The good thing for now is… On putting the two drives in the local machine and running the rsync command I havne’t faced any issues till now. So it looks like the mediasonic enclosure could be the issue. Might have to build a separate low powered rig now.

Just wondering… If this x18 drive also ends up having bad sectors could it be due to high temperatures?
After running the rsync on the minipc when I went to take out the drives both of them were really hot to the touch