TrueNAS SMART Failed - How to read results?

Operating Systems & Open Source
1

Woke up with an email from my secondary NAS that a disk had failed SMART. No biggie, I have a million spares but I want to know what went wrong

Going into the section of “SMART Test Results” gets me, well, no results!

image
image1318×543 28.5 KB

Poked around in the UI and saw nothing, moved to CLI and did smartctl -a /dev/da10 and got this

Code:
smartctl 7.2 2020-12-30 r5155 [FreeBSD 12.2-RELEASE-p14 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               HGST
Product:              H7280A520SUN8.0T
Revision:             PAG1
Compliance:           SPC-4
User Capacity:        7,865,536,647,168 bytes [7.86 TB]
Logical block size:   512 bytes
Physical block size:  4096 bytes
Formatted with type 1 protection
8 bytes of protection information per logical block
LU is fully provisioned
Rotation Rate:        7200 rpm
Form Factor:          3.5 inches
Logical Unit id:      0x5000cca260bd5228
Serial number:        001649PB3R5V        VLKB3R5V
Device type:          disk
Transport protocol:   SAS (SPL-3)
Local Time is:        Sat Oct 15 09:18:21 2022 CDT
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Enabled

=== START OF READ SMART DATA SECTION ===
SMART Health Status: OK

Current Drive Temperature:     36 C
Drive Trip Temperature:        85 C

Accumulated power on time, hours:minutes 45940:36
Manufactured in week 49 of year 2016
Specified cycle count over device lifetime:  50000
Accumulated start-stop cycles:  37
Specified load-unload count over device lifetime:  600000
Accumulated load-unload cycles:  1940
Elements in grown defect list: 0

Vendor (Seagate Cache) information
  Blocks sent to initiator = 21759842658025472

Error counter log:
           Errors Corrected by           Total   Correction     Gigabytes    Total
               ECC          rereads/    errors   algorithm      processed    uncorrected
           fast | delayed   rewrites  corrected  invocations   [10^9 bytes]  errors
read:          0       23         0        23   18569405    1390483.328           0
write:         0       13         0        13   12180425     902791.191           0
verify:        0        0         0         0      84772          1.273           0

Non-medium error count:        0

SMART Self-test log
Num  Test              Status                 segment  LifeTime  LBA_first_err [SK ASC ASQ]
     Description                              number   (hours)
# 1  Background short  Failed in segment -->       3   45931                 - [0x1 0x5d 0xfd]
# 2  Background short  Completed                   -   45907                 - [-   -    -]
# 3  Background short  Completed                   -   45883                 - [-   -    -]
# 4  Background short  Completed                   -   45859                 - [-   -    -]
# 5  Background short  Completed                   -   45835                 - [-   -    -]
# 6  Background short  Completed                   -   45811                 - [-   -    -]
# 7  Background short  Completed                   -   45787                 - [-   -    -]
# 8  Background short  Completed                   -   45763                 - [-   -    -]
# 9  Background short  Completed                   -   45739                 - [-   -    -]
#10  Background short  Completed                   -   45715                 - [-   -    -]
#11  Background short  Completed                   -   45691                 - [-   -    -]
#12  Background short  Completed                   -   45667                 - [-   -    -]
#13  Background short  Completed                   -   45643                 - [-   -    -]
#14  Background short  Completed                   -   45619                 - [-   -    -]
#15  Background short  Completed                   -   45595                 - [-   -    -]
#16  Background short  Completed                   -   45571                 - [-   -    -]
#17  Background short  Completed                   -   45547                 - [-   -    -]
#18  Background short  Completed                   -   45523                 - [-   -    -]
#19  Background short  Completed                   -   45499                 - [-   -    -]
#20  Background short  Completed                   -   45475                 - [-   -    -]

Long (extended) Self-test duration: 63865 seconds [1064.4 minutes]

So I can see it failed on “Segment 3”, but what the heck is segment 3?

Never worked this hard in a NAS OS to see results before!

Is my best put pulling it out and throwing it into a Windows box and using hard disk sentinel etc?

I didn’t want to pull it and find the error was something stupid

2

Hard drives have the ability to do SMART self tests, where the drive checks itself and can report back any issues. Here it seems that TrueNAS is scheduling a selftest every 24 hours. So the error “Failed in segment 3” is an error from the harddrive itself after a self test. Normally a failure like this is an indication of possible future failure, so you can replace the drive before it fails. Given the drive is 5 years old I’d proceed with replacement.

If you want to do more testing you can do another background short test, maybe the failure was a fluke? Otherwise you can remove the drive and do a foreground long test - a long test tests every sector on the hard drive, and the hard drive becomes unusable during the test (hence foreground). So don’t do a long test while the disk is in the array.