Formatting the Optane SSD P1600X SSDPEK1A118GA01 to use 4k blocks

Hey so I bought some of these 118GB optane disks and had some trouble switching them to use 4k blocks. I thought I would post something here saying how I did it.

$ echo 0 >/sys/module/nvme_core/parameters/default_ps_max_latency_us
$ echo 0 >/proc/sys/kernel/hung_task_timeout_secs
$ time nvme format -fl1 --timeout=3600000 /dev/nvme2
Success formatting namespace:ffffffff

real	14m19.103s
user	0m0.002s
sys	0m0.000s

Basically formatting this thing takes too long and I had to keep turning off things that would time out the command. If a timeout was hit I would get a line in dmesg about the nvme controller or something then normal reboots would hang. Then I would have to use echo b >/proc/sysrq-trigger to reliably reboot without having to go and hold the power button.

Hope this helps someone.

On an unrelated note… it looks like blkdiscard doesn’t work on these things or the 900GB 905Ps… :confused:

Edit:
Most of the info came from here: How to change Intel Optane P4800X sector size | tmikey’s fireplace

1 Like

There is no concept of discarding blocks on 3D Xpoint media.

Could you tell what distro and kernel version you were using. I’ve tried doing the same but I can’t reformat the drive.

I couldn’t do it quickly on gentoo because kernel lacked /proc/sys/kernel/hung_task_timeout_secs.

I’ve tried using ubuntu 22.10 and arch livecd.

On ubuntu kernel 5.15 I killed nvme format after 25 minutes. Drive was just dead and I had to reboot to check it’s state.

On arch 6.2.8 livecd kernel operation timed out despite writing 0 to default_ps_max_latency_us and hung_task_timeout_secs
Drive timed out, reseted itself and appeared back in nvme list

I’m still stuck at 512B sectors

I did format over the weekend (NVMe) using the tool provided by BIOS. I am guessing it took around that long. But I let it run all the way to the end and reboot itself. It’s probably normal to take that long.

I’ve tried again and managed to format this ssd.
ubuntu 22.10 livecd (kernel 5.15)

It finished under 15 minutes, just like in OP’s post.

2 Likes

Can anyone explain these observations?

I checked out the NVMe Secure Erase feature of an ASUS motherboard’s BIOS (ProArt X570-CREATOR WIFI), it can change the LBA settings.

With a Samsung 980 Pro SSD this seems to be locked at 512 Bytes:

With a Samsung PM1733 SSD there are multiple options (?) for 512 Bytes and 4 KB:

Note: Erasing and changing to 4 KB LBA here only took a few seconds.

Not sure if it’s related but some 1733 firmware has 512/520 and 4k/4k+dif which is the data integrity field formatting option. It might or might not be that.

I’m almost sure it is. This is what I get when attempting to do the same on a P5800X: 5 formatting options, 2 for 512 and 3 for 4k sectors. The first options for each sector size are usually for 0 metadata per sector. Since it is a consumer motherboard, we are probably not given the metadata sizes for the other options.

Man I gotta source some more enterprise flash. My main source of surplus in the tristate area retired and everything else is hyperinflated lol

@wendell

Do you happen to have experience with flashing Samsung enterprise drives?

Thanks to a user from the ServeTheHome forums the actual firmware files seem to have been made “available”, but I’m a bit worried damaging the drives:

I’ve done it only from Samsungs bootable iso which is a Linux distro. Back up the drives. I did it one drive at a time on a 5 drive raid array once and made sure each drive had the same md5 sum after every every one. If ones md5 sum changed I was gonna let the raid rebuild as I was worried about it

To answer your question I am using ubuntu server 22.04 LTS. I don’t know what kernel I was using. I’m currently on 5.15.0-69-generic for what it’s worth.

Glad you got it working.

1 Like

I also failed to convert my P1600X to 4096B block size and started digging, and it seems it’s due to P1600X support only 512B?

I can’t paste links so I’ll describe the link.
One is Intel support article 000057964, one is Intel community post 1478873.

Some Intel® Optane™ drives like the Intel® Optane™ SSD P1600X Series support only 512B.

I just received confirmation that P1600X does not support variable sector size, so it only supports 512.

smartctl -a /dev/nvme??? will tell you what it supports.
What does it say under “Supported LBA Sizes”?

Some Samsung drives give you lots of interesting options. Here are the LBA formats for a PM1725a (from smartctl):

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 -     512       0         1
 1 -     512       8         3
 2 -    4096       0         0
 3 -    4096       8         2

As @wendell correctly intuited, two of the formats include extra bytes per sector for data integrity. My understanding is that the lowest relative performance value gets you the highest absolute performance.

On NAND, sure. It just seems to be an issue with 3D Xpoint.

I have read that it does pass some “hints” to the controller that can help it optimize… something… but I’m not sure how significant the effect is, or if it even is at all.

That’s not accurate. Here are the LBA formats for a P1600X 58GB (firmware version U5110550):

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 -     512       0         2
 1 +    4096       0         0

And for a P1600X 118GB (firmware version U5110550):

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 -     512       0         2
 1 +    4096       0         0

This is really helpful, thanks. I’ve also had a lot of trouble changing the format of P1600X drives, even using the official Intel tool. On every other model NVMe I’ve ever used it’s a simple nvme format --lbaf=1 or whatever and it returns immediately. It seems to take forever on the P1600X; sometimes I literally have to reboot the system after an hour or two of waiting and try again, and then it usually takes about 15 minutes. I’ll try your workaround next time.

I haven’t run the Intel tool in a while but maybe you’d have better luck with that? You can download the CLI packages from here and the command is something like:

# intelmas start -intelssd 99 -nvmeformat LBAformat=1

And obviously please make sure you get the right block device index (i.e. not 99) and LBA format number before running that command. :slight_smile:

1 Like

I found the same thing on optane 4k sectors but does it have to do if the optane is connected via normal pcie lanes or the VMd/vroc lanes? Might be worth trying with and without VMd modes to see if that lets you format as 4k

Seems unaffected performance wise fwiw

In my case at least I’ve experienced the issue using CPU lanes to attach a P1600X on a SuperMicro C612 board, an ASRock DeskMini X300, and an Asus X570 motherboard.

I just wanted to add on my experience: I tried this same procedure on a few different machines, plus a couple others.

  • TrueNAS scale bluefin: Does not work, kernel resets the target device.
  • Ubuntu 22.04 live USB: Looked like it was working but never completes. Requires reset.
  • Windows using intelMAS: gives immediate error “operation not supported with this driver”.
    intelmas start -intelssd $index -nvmeformat LBAformat=1
    I tried both nvme-cli and intelMAS, as root (admin). Nothing actually worked.

Here’s what worked the first time with zero issues:

  • Asus secure erase utility in a z690 mobo. It took around 15 minutes per drive.

Thanks for the idea /u/aBav.Normie-Pleb

So. I plan to do some digging. I imagine that something in the intel firmware is being picky about the order of operations. The various OS are interacting with the drive to get it into a usable state for the system, and this may preclude it from using the nvme commands without a controller reset, or some other intermediate steps.

2 Likes

Using a ASUS motherboard with the bios erase feature worked for me to finally one of these drives converted to 4k lba. Thanks for the hint.

I don’t have another Asus system to use to convert the other drives I have, I’m still trying to figure a way to do it reliably using arch linux.

I came back to this a short time later and found a reliable method to reformat these drives, I boot using the latest arch linux iso and used these commands.

echo 0 > /proc/sys/kernel/hung_task_timeout_secs
echo 2400000 > /sys/module/nvme_core/parameters/admin_timeout
nvme format /dev/nvme1 -n 1 -l 1 -t 2400000 --force

It looks to perform the format uninterrupted without issue despite exiting with a error. the format took about 15 minutes per (118gb) drive.

4 Likes