Thank you once again for your insight. I plan to let this run for the rest of the week and than play with it, once I get back from vacation.
I currently have 2ea 7.68tb SAS SSD drives already and I am adding these 2 to the array. My plan is to go all flash for my onsite storage and spinning rust at a remote site.
If and when the Trimming is done and if the speeds are still not where they should be, could it be the Dual/Single SAS link?
So dual on these drives is uusuallllyyyyy active passive
Multipathd is a thing
Bit if you just use SD whatever it’s not multipath
It could be noise on the sas cables or bad backplane
Or drives that are 12g and controller that’s 12g trying to sync at 6g
You can use some commands to force sync at 6g or 12g to rule that out. Writes under 100mb/sec is hardware issue. Hopefully lack of trim. Maybe something else.
Smart diagnostics long and short are also an option and usually that works reasonably well
smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.10.28-Unraid] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Vendor: SAMSUNG
Product: AREA7680S5xnNTRI
Revision: 3P03
Compliance: SPC-4
User Capacity: 7,681,501,126,656 bytes [7.68 TB]
Logical block size: 512 bytes
Physical block size: 4096 bytes
LU is resource provisioned, LBPRZ=1
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
Logical Unit id: 0x5002538a06cb9c00
Serial number: S31RNA0HC00288
Device type: disk
Transport protocol: SAS (SPL-3)
Local Time is: Wed Nov 17 19:27:51 2021 CST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
Temperature Warning: Enabled
Read Cache is: Enabled
Writeback Cache is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART Health Status: OK
Percentage used endurance indicator: 0%
Current Drive Temperature: 33 C
Drive Trip Temperature: 68 C
Manufactured in week 50 of year 2016
Accumulated start-stop cycles: 10
Specified load-unload count over device lifetime: 0
Accumulated load-unload cycles: 0
Elements in grown defect list: 0
Error counter log:
Errors Corrected by Total Correction Gigabytes Total
ECC rereads/ errors algorithm processed uncorrected
fast | delayed rewrites corrected invocations [10^9 bytes] errors
read: 0 0 0 0 0 9336.101 0
write: 0 0 0 0 0 55122.977 0
verify: 0 0 0 0 0 3112.740 0
Non-medium error count: 300
Self-test execution status: 100% of test remaining
SMART Self-test log
Num Test Status segment LifeTime LBA_first_err [SK ASC ASQ]
Description number (hours)
1 Background long Self test in progress … - NOW - [- - -]
Long (extended) Self-test duration: 3600 seconds [60.0 minutes]
Background scan results log
Status: waiting until BMS interval timer expires
Accumulated power on time, hours:minutes 1230:25 [73825 minutes]
Number of background scans performed: 1, scan progress: 0.00%
Number of background medium scans performed: 1
Protocol Specific port log page for SAS SSP
relative target port id = 1
generation code = 4
number of phys = 1
phy identifier = 0
attached device type: expander device
attached reason: hard reset
reason: loss of dword synchronization
negotiated logical link rate: phy enabled; 12 Gbps
attached initiator port: ssp=0 stp=0 smp=1
attached target port: ssp=0 stp=0 smp=1
SAS address = 0x5002538a06cb9c02
attached SAS address = 0x500143803503d37f
attached phy identifier = 3
Invalid DWORD count = 862
Running disparity error count = 862
Loss of DWORD synchronization = 4
Phy reset problem = 0
Phy event descriptors:
Received ERROR count: 0
Received address frame error count: 0
Received abandon-class OPEN_REJECT count: 0
Received retry-class OPEN_REJECT count: 1
Received SSP frame error count: 0
relative target port id = 2
generation code = 4
number of phys = 1
phy identifier = 1
attached device type: expander device
attached reason: loss of dword synchronization
reason: loss of dword synchronization
negotiated logical link rate: phy enabled; 12 Gbps
attached initiator port: ssp=0 stp=0 smp=1
attached target port: ssp=0 stp=0 smp=1
SAS address = 0x5002538a06cb9c03
attached SAS address = 0x500143803503d37d
attached phy identifier = 3
Invalid DWORD count = 2
Running disparity error count = 2
Loss of DWORD synchronization = 1
Phy reset problem = 0
Phy event descriptors:
Received ERROR count: 0
Received address frame error count: 0
Received abandon-class OPEN_REJECT count: 0
Received retry-class OPEN_REJECT count: 0
Received SSP frame error count: 0
Could go in the controller options and force 6gbps or use the cli to do that.
Maybe worrisome if these are increasing fairly substantially especially running xfer tests
Would this be a drive issue, cable issue or controller issue?
This is a 24bay HP enclosure and there are currently only 4 drives installed. 4 of these 7.68tb SAS 12g SSD drives.
confirm the enclosure can support 12g in the docs? the enclosure doesnt always have the ability to say ‘hey, I can only do 6g’ its between the drive and controller. mostly. a lot of the time.
The controller is 12g and the enclosure is 12g.
Supermicro AOC-S3008L-L8E 8-Port 12Gb/s SAS PCIe HBA Host Bus Adapter
3PAR Storserve 20000 12 Gb/sec SAS SFF Drive Enclosure
Sorry here is the card I am using for the external enclosure
Lenovo N2226 12Gb SAS External HBA
cables, maybe then. If those numbers are going up rapidly esp. in heavy load
if they arent going up rapidly then prolly ok
Wendell,
I pulled these drives from the Unraid server and stuck them in my gaming system (which has the supermicro 12g SAS controller) and the drives now test at full speeds 900-1100mbps (R/W)
But upon putting them back in the Unraid server, they are still only hitting 80-90mbps. I am going to try each of the slots in the array and see if the issue is present. The weird part is that I have 2 other drives in the same system, that work perfectly. I am wondering if there is a controller issue.
Diagnostic strongly suggests cabling or backplane or controller issue
Sorry for bumping such an old thread, but ESXi can use the drives as-is. I did not however have any luck putting the EMC Samsung(SS162512 CLAR200) disks into vSAN drive groups. They did after formatting to 512, though.
I should mention that the connector on one Samsung CLAR200 pulled out with the SAS connector and the pads didn’t look so hot. I guess that drive came out of an abused appliance, but that’s why I do a trial buy of these kinds of things.
1 Like
I have problem with reformat this hard disk from 528 to 512 please help me note this is hard disk ibm and working flash ssd 0kb
I would try with a Linux bootable USB distro, such as System Rescue, and running the sg_format command there, instead of trying to do it on Windows.
You sometimes also need to first format with the existing sector size (528) and after that switch to formatting with the sector size you want (512).
There could be other problems, such as two common disk encryption methods, or similar.
Doing a quick search on the model, I also see there’s a bug and you need to update to firmware E1A8 or newer, asap.
1 Like
Just wanted to post on this so anyone in the future finding this would have an easier time. I spent hours wondering why the drives were appearing in the “SAS Topology” of my LSI HBA Card (LSI-9211-8i) that is flashed into IT mode. They didn’t show up in the drive boot up menu.
I then found this post and tried to do the steps in this but ran into an issue as the mpt3sas driver has seemingly been removed from the Kernel or something between 2018 and now. I found a Reddit post
https://www.reddit.com/r/AlmaLinux/comments/vf8rq7/mpt3sas_card_deprecated/
So, the solution to have your Netapp/IBM formatted SSD’s. In my case 200GB Samsung MZ-6ER4000/0G3 connected to your HBA but running under Ubuntu 18.04.6 LTS since this version is old enough to still support the older cards.
You can find the dev of these drives by running lsscsi and get the dev/ from that. Mine were dev/sda - sdd.
Run the first command Wendell gave that is formatting 520 byte sectors:
sg_format -v --format --size=520 /dev/sg2
BUT, for the second command if you drives are like mine, Mine were “IBM” formatted but they’re the same 10 byte mode sense but uses the 4 for checksums exactly like the Netapp drives do.
So, use this command instead:
sudo sg_format -v --format --size=512 --six /dev/sdx
If you’re formatting multiple drives, open multiple terminal windows and do them all at the same time.
It took me like 6 hours of troubleshooting to find all of this out as I’d no idea what I was looking for. Hopefully this helps someone else as well.
@wendell Question for you, you mention flashing firmware in the video.
I have an IBM drive that I can not change its block size of 528, I found the firmware for the OEM version of the drive (XS7680SE70094). Is it pretty safe to try and flash this drive with the OEM version of the firmware?
Udually pretty safe for samsung and toshiba. Dont know about others. If in doubt dump tbe chip via chip clip
Its Seagate so guess we will find out.