How to reformat 520 byte drives to 512 bytes (usually)

Hardware
, ,
18

Edit: the method tried in this post was not successful. It formats, but does not change the block size.

Just joined up to comment that I tried both the original command, and the ‘–six’ option, and tried formatting at 528 bytes (which my drive came as) then immediately reformatting to 512, but none of these worked (all reported “Illegal request sense key” or similar). For the record, this one is a Seagate Constellation ES2 (DXS2D-H3R0SS/ST33000650SS) 3TB drive.

What appears to have worked in the end was sg_format -F -F -F --size=512 /dev/blah (ie, passing the --format option three times, which the doc says means “FORMAT UNIT command only”, I assume it doesn’t do mode sense / status command first, and I guess that was what was erroring out when the block size didn’t match.)

It is currently in the process of formatting so I am taking that as a success, but I’ll come back and edit this post if it turns out to have failed (if the block size didn’t change).

5 Likes
19

thanks for posting, denvercoder76

1 Like
20

This was the most comprehensive result I found from searching, so I figured this was the best place to put it. :slight_smile:
(Hello, future people tearing your hair out over uncooperative drives.)

2 Likes
21

fwiw I had a couple of stubborn drives but hot plugging them right before formatting seemed to get their cooperation. Since this original post, and my original 12 drives, four have died.

22

I might do this holy hell!

Next server upgrade might be in a few years but massive storage array of SSDs when. Then again I don’t really know of ZFS Raid Z2 or Z3 works well on SSDs yet. It might chew them up. Not sure.

See I would love to take down the caching to just mets data. I have way more RAM than I need but if I have SSDs those would be suitably fast enough to be metadata only caches for the l2 and primary cache. Keep the arc size smaller and Its memory usage lower

Though tbch if I was going this route I should really just Linus tech tips it. Grab an older threadripper or epyc. And use the insane PCIE lanes say 32 of them for 8x NVME expansion. That would be a crazy array but hey in 5 years time there light be hardware to feasibly do it for cheap!

Pretty sure the goal is to always make the CPU the bottleneck. LOL

23

Had an issue like this… here was my solution…

1 Like
24

Thank you! I was just coming back to report that my attempt using sg_format -F -F -F did NOT work, so I will try this instead now.

(For the curious - I was too hasty; it’s necessary to send the mode select command - which was failing - otherwise the format will just reformat using the current parameters.)

Also for a quicker reference - the tool linked to in the other thread is the ‘setblocksize’ repo on github:ahouston.

1 Like
25

Ok, after digging through the setblocksize source code and SCSI protocol manuals, here is a further update for the benefit of anyone having trouble with the setblocksize tool. It’s possible to replicate what setblocksize does, using just the sg_* tools (eg if you have problems getting the build to work), with the following process:

printf '\0\0\0\10\0\0\0\0\0\0\2\0' | sg_raw --send=12 /dev/blah 15 11 00 00 0c 00
This sends the SCSI ‘MODE SELECT (6), 512 byte block size, maximum capacity’ command directly to the unit. You should see the response SCSI Status: Good; if not, it didn’t work.

DO NOT UNPLUG/POWER OFF THE DRIVE AFTER THIS UNTIL IT IS FORMATTED
(Yep, I’m now looking at how to recover one of my drives from a ‘MEDIUM ERROR - format corrupted’ condition - oops)

(If you run sg_format /dev/blah (with no args) at this point, you should see a 512 byte block size and an error about ‘corrupt format’ at the READ CAPACITY step.)

sg_format --format --size=512 /dev/blah
This should now work fine. If it doesn’t, you can also try 'sg_format -F -F -F /dev/blah` (which skips the mode sense/select steps), assuming you successfully set the block size above this should probably work, but I haven’t tested it; the regular format command worked for me.

(Again for those curious – it appears that sg_format is trying to set some additional mode page parameters beyond just the block size, and for whatever reason sometimes the drive doesn’t like this. Sending the command with just block size and no mode page settings seems to work fine, which is what setblocksize does, and what the above command does.)

2 Likes
26

Final update - how I fixed the corrupt format on one of the drives.

If you do pull/power-cycle the drive between the resize and format steps, you may end up in a situation where the system will no longer attach and assign a /dev/blah device due to the invalid format. (This happened to me on FreeBSD; I don’t have a linux SAS system handy so I can’t confirm the behaviour there; if you still have the /dev/ file you should be able to just use sg_format with no problems.)

The error message is along the lines of

(pass0:mps0:0:39:0): SCSI sense: MEDIUM ERROR asc:31,0 (Medium format corrupted)
(pass0:mps0:0:39:0): Field Replaceable Unit: 0
(pass0:mps0:0:39:0): Descriptor 0x80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00
(pass0:mps0:0:39:0): fatal error, failed to attach device

In this case, sg_format can no longer operate on the device as it expects to be given a device file. Instead, you can use camcontrol format 0:39 with the bus and device number from the error message above (or use camcontrol devlist to list the devices and confirm the bus and target number).

(I’ll also note here that it should be possible to use camcontrol cmd to send the resize command instead of using sg_raw, if someone is reading this who uses BSD and for some reason can’t get the sg_tools to work. I haven’t tested the exact command because I’ve already resized all the drives I needed to for now.)

*Update to final update * – after doing the above camcontrol format 0:## to recover from a hotplug/power cycle, you may likely end up finding that your block size is still wrong. But now you have fixed the medium format corrupt error, so you can replug the drive again to get the /dev/da# device back and repeat the printf ... | sg_raw ... step followed by an sg_format or camcontrol format to bake in the desired size.

(In my case, strangely, after the first recovery attempt the block size had been reset to 520 bytes, even though it was originally 528 and I was trying to change it to 512, and at no point did I ask it to set 520!)

3 Likes
27

I picked up some HP 7.68tb SAS drives for cheap and they were 520, I used the sgformat command and they show as 512 now, but the reads and writes are not even close to the specs of the drives. These are PM1633a drives, with only 4.9TB written to them. The reads are only 300-400mb, but the writes are 68-72mb. I have tried these in Unraid, linux box and windows box with the same results. Could this be a TRIM issue?

28

could be trim, trim the whole drive.
check temps
check smart
are the drives burn your hand off on fire? that could explain it too

29

The drives are def hot to the touch, I put them back in my enclosure, which has super duper fans.

30

=== START OF INFORMATION SECTION ===
Vendor: SAMSUNG
Product: AREA7680S5xnNTRI
Revision: 3P03
Compliance: SPC-4
User Capacity: 7,681,501,126,656 bytes [7.68 TB]
Logical block size: 512 bytes
Physical block size: 4096 bytes
LU is resource provisioned, LBPRZ=1
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
Logical Unit id: 0x5002538a06ce7880
Serial number: S31RNA0HC02409
Device type: disk
Transport protocol: SAS (SPL-3)
Local Time is: Tue Nov 16 13:18:07 2021 CST
device is NOT READY (e.g. spun down, busy)
A mandatory SMART command failed: exiting. To continue, add one or more ‘-T permissive’ options.

31

I used the fstrim option and
/mnt/test: 14 TiB (15360851976192 bytes) trimmed

Speeds are now worse, the read and write are between 30-40mb

=== START OF INFORMATION SECTION ===
Vendor: SAMSUNG
Product: AREA7680S5xnNTRI
Revision: 3P03
Compliance: SPC-4
User Capacity: 7,681,501,126,656 bytes [7.68 TB]
Logical block size: 512 bytes
Physical block size: 4096 bytes
LU is resource provisioned, LBPRZ=1
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
Logical Unit id: 0x5002538a06cb9c00
Serial number: S31RNA0HC00288
Device type: disk
Transport protocol: SAS (SPL-3)
Local Time is: Tue Nov 16 15:13:33 2021 CST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
Temperature Warning: Enabled

=== START OF READ SMART DATA SECTION ===
SMART Health Status: OK

Percentage used endurance indicator: 0%
Current Drive Temperature: 35 C
Drive Trip Temperature: 68 C

Manufactured in week 50 of year 2016
Accumulated start-stop cycles: 10
Specified load-unload count over device lifetime: 0
Accumulated load-unload cycles: 0
Elements in grown defect list: 0

Error counter log:
Errors Corrected by Total Correction Gigabytes Total
ECC rereads/ errors algorithm processed uncorrected
fast | delayed rewrites corrected invocations [10^9 bytes] errors
read: 0 0 0 0 0 9328.722 0
write: 0 0 0 0 0 41497.532 0
verify: 0 0 0 0 0 3112.740 0

Non-medium error count: 59

No Self-tests have been logged

=== START OF INFORMATION SECTION ===
Vendor: SAMSUNG
Product: AREA7680S5xnNTRI
Revision: 3P03
Compliance: SPC-4
User Capacity: 7,681,501,126,656 bytes [7.68 TB]
Logical block size: 512 bytes
Physical block size: 4096 bytes
LU is resource provisioned, LBPRZ=1
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
Logical Unit id: 0x5002538a06ce7880
Serial number: S31RNA0HC02409
Device type: disk
Transport protocol: SAS (SPL-3)
Local Time is: Tue Nov 16 15:14:18 2021 CST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
Temperature Warning: Enabled

=== START OF READ SMART DATA SECTION ===
SMART Health Status: OK

Percentage used endurance indicator: 0%
Current Drive Temperature: 32 C
Drive Trip Temperature: 68 C

Manufactured in week 50 of year 2016
Accumulated start-stop cycles: 9
Specified load-unload count over device lifetime: 0
Accumulated load-unload cycles: 0
Elements in grown defect list: 0

Error counter log:
Errors Corrected by Total Correction Gigabytes Total
ECC rereads/ errors algorithm processed uncorrected
fast | delayed rewrites corrected invocations [10^9 bytes] errors
read: 0 0 0 0 0 9249.636 0
write: 0 0 0 0 0 41452.121 0
verify: 0 0 0 0 0 3112.810 0

Non-medium error count: 62

No Self-tests have been logged

32

OK so this is getting really strange. I am doing the preclear on both drives and initially they started at 30-40mbps, than after about an hour they are running at 699mbps.

Any idea what the heck is going on? Could there be bad blocks in the initial run?

33

so when you do trim it just tells it to trim. It wont actually trim until the drive is idle.

so once you give it trim leave it alone. for like a day since they are so big

also keep them cool during this period

the speed going up may mean that a “little bit” of the trim finished?

The Actual Speed Of Erasure on those drives is maybe 100 megabytes/sec. Maybe. Assuming optimal temperature. That’s many many hours of trimming

2 Likes
34

Thank you once again for your insight. I plan to let this run for the rest of the week and than play with it, once I get back from vacation.
I currently have 2ea 7.68tb SAS SSD drives already and I am adding these 2 to the array. My plan is to go all flash for my onsite storage and spinning rust at a remote site.

35

If and when the Trimming is done and if the speeds are still not where they should be, could it be the Dual/Single SAS link?

36

So dual on these drives is uusuallllyyyyy active passive

Multipathd is a thing

Bit if you just use SD whatever it’s not multipath

It could be noise on the sas cables or bad backplane

Or drives that are 12g and controller that’s 12g trying to sync at 6g

You can use some commands to force sync at 6g or 12g to rule that out. Writes under 100mb/sec is hardware issue. Hopefully lack of trim. Maybe something else.

Smart diagnostics long and short are also an option and usually that works reasonably well

37

smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.10.28-Unraid] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor: SAMSUNG
Product: AREA7680S5xnNTRI
Revision: 3P03
Compliance: SPC-4
User Capacity: 7,681,501,126,656 bytes [7.68 TB]
Logical block size: 512 bytes
Physical block size: 4096 bytes
LU is resource provisioned, LBPRZ=1
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
Logical Unit id: 0x5002538a06cb9c00
Serial number: S31RNA0HC00288
Device type: disk
Transport protocol: SAS (SPL-3)
Local Time is: Wed Nov 17 19:27:51 2021 CST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
Temperature Warning: Enabled
Read Cache is: Enabled
Writeback Cache is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART Health Status: OK

Percentage used endurance indicator: 0%
Current Drive Temperature: 33 C
Drive Trip Temperature: 68 C

Manufactured in week 50 of year 2016
Accumulated start-stop cycles: 10
Specified load-unload count over device lifetime: 0
Accumulated load-unload cycles: 0
Elements in grown defect list: 0

Error counter log:
Errors Corrected by Total Correction Gigabytes Total
ECC rereads/ errors algorithm processed uncorrected
fast | delayed rewrites corrected invocations [10^9 bytes] errors
read: 0 0 0 0 0 9336.101 0
write: 0 0 0 0 0 55122.977 0
verify: 0 0 0 0 0 3112.740 0

Non-medium error count: 300

Self-test execution status: 100% of test remaining
SMART Self-test log
Num Test Status segment LifeTime LBA_first_err [SK ASC ASQ]
Description number (hours)

1 Background long Self test in progress … - NOW - [- - -]

Long (extended) Self-test duration: 3600 seconds [60.0 minutes]

Background scan results log
Status: waiting until BMS interval timer expires
Accumulated power on time, hours:minutes 1230:25 [73825 minutes]
Number of background scans performed: 1, scan progress: 0.00%
Number of background medium scans performed: 1

Protocol Specific port log page for SAS SSP
relative target port id = 1
generation code = 4
number of phys = 1
phy identifier = 0
attached device type: expander device
attached reason: hard reset
reason: loss of dword synchronization
negotiated logical link rate: phy enabled; 12 Gbps
attached initiator port: ssp=0 stp=0 smp=1
attached target port: ssp=0 stp=0 smp=1
SAS address = 0x5002538a06cb9c02
attached SAS address = 0x500143803503d37f
attached phy identifier = 3
Invalid DWORD count = 862
Running disparity error count = 862
Loss of DWORD synchronization = 4
Phy reset problem = 0
Phy event descriptors:
Received ERROR count: 0
Received address frame error count: 0
Received abandon-class OPEN_REJECT count: 0
Received retry-class OPEN_REJECT count: 1
Received SSP frame error count: 0
relative target port id = 2
generation code = 4
number of phys = 1
phy identifier = 1
attached device type: expander device
attached reason: loss of dword synchronization
reason: loss of dword synchronization
negotiated logical link rate: phy enabled; 12 Gbps
attached initiator port: ssp=0 stp=0 smp=1
attached target port: ssp=0 stp=0 smp=1
SAS address = 0x5002538a06cb9c03
attached SAS address = 0x500143803503d37d
attached phy identifier = 3
Invalid DWORD count = 2
Running disparity error count = 2
Loss of DWORD synchronization = 1
Phy reset problem = 0
Phy event descriptors:
Received ERROR count: 0
Received address frame error count: 0
Received abandon-class OPEN_REJECT count: 0
Received retry-class OPEN_REJECT count: 0
Received SSP frame error count: 0