Extremely slow copy between two fast NVMe's

Hi all,

I’ve been meaning to join for ages but this time I have a problem I legit don’t know how to figure out, so I thought I’d ask.

Tl;dr - I’m trying to transfer my work files from one drive to another and it plummets immediately, only running at like 4-5MB/s… between a drive that reads at 3.5GB/s and one that writes at almost 6GB/s o_O Any guesses as to what can be wrong? It’s a bunch of large files mostly so it’s a lot of sequential reading and writing.

Long version:

I have rebuilt my Threadripper workstation, I’m a 3D artist and Architect so I need something that can lift some weight. 13900K, 3090 and 64GB memory are enough for now, what I was lacking is storage so I bought a bunch of drives.

I have a P990 Pro 1TB, P980 Pro 2TB, P980 and a Corsair MP510 for which I honestly have nothing but love and good words. Yes it’s a bit of a salad but really I just bought what I could within the budget (system, work, gaming/media in that order). Before you ask, drives came with new firmware from the start, don’t seem to be affected at all by the accelerated use bug.

I’m trying to copy from the 510 to the 980 Pro, I know they’re both fully functional but I cannot understand WHY transfer speeds just plummet to hell. Could it be because one is on chipset and other on CPU slots? Even so, as I’m watching it, it’s at 2.5MB/s and it’s destroying me, not to mention I don’t have 24h available to let it copy 850GB worth of files. At this point it would be faster to copy from network NAS, this makes no sense.

Anyway, any ideas would be greatly welcome!

Many thanks,
V

1 Like

I’m assuming you’re on windows, to rule out some of the weird things windows file explorer does, could you try doing the same copy using a 3rd party utility (I recommend FastCopyPortable)?

1 Like

Sorry, yes, I should have mentioned I’m on Windows.

It’s only from this drive that it’s slow as hell, in all honesty I’m considering the possibility of it just dying on me but crystaldisk and some other tests have turned up fine, no SMART errors either, I have no idea.

I’ve tried a few apps, FreeFileSync is my main choice, either way they all run extremely slow.

From any other drive to another it’s fine, from this one to any other, not fine. From any other to this one, not fine. I have a slight feeling the controller is dying, in which case this is about to be a very bad time cause it’ll take me 2-3 days to back up all files at this rate.

Any other suggestions?

1 Like

How old is the MP510? or more specifically how old is the data on the MP510?

Something no one ever brings up or benchmarks is what happens to data that has been sitting on an SSD for along time (>1 year-ish), there is a drift in the NAND cell’s charge and the controller has to work very very hard to try and figure out what the data is supposed to be; in theory trim/gc is supposed to prevent this but it does not under all circumstances.

2 Likes

The drive itself I think is about 3.5y at this point, although not hammered. What I’m currently copying honestly can range from 2 years to 2 days or 2 hours even.

The data, even “new” data from the last few weeks copies extremely slow. Every time it copies a new file, it bursts up to 120-150MB/s and then dies right down to like 2-3MB/s.

I’ve never heard of charge drift in NAND, christ I’ve joined this forum and in under 30 minutes I learned something new, thanks :slight_smile:

Is there anything I can do to it to speed things up? I hear TRIM is supposed to help with this sort of issue but again, I’m no expert in storage.

1 Like

Have you tried booting with some Linux live distro any moving things to a backup drive? Anything with ntfs-3g should be able to do basic copying just fine to/from windows partitions. That would rule out windows itself.

FreeFileSync is also Linux compatible, I love it for manual one off operations.

Also note that once things are copied over, you can use FreeFileSync to check to make sure both sides are truly bit identical with a content check, which you may want to do since the drive is suspect. This of course will take forever since both sides are fully read, again. It won’t catch errors in the original files, but should catch if the drive happens to have returned garbage at some point: FreeFileSync

Also, what is the size of the files you are copying? tens/hundreds of Thousands of Kb sizes files? Or mostly Mb/Gb sizes? Even NVMe still isn’t great when every sector is a file.

2 Likes

hmmm… that is concerning that the new data written is slow too.
It’s still possible its just never been trimmed and is clogged up with half empty cell blocks that need to be rearranged when written and read causing the slow down.

To check the trim status of the drive launch the “Defragment and Optimize Drives” tool (it is a built into windows tool so you’ll have it) and find the drive, it should mention a Last run date on the drive in question.

3 Likes

Thanks both of you for the informative reply, I have thought about booting up a quick live PopOS to try and copy but in all honesty, it’s already midnight here and I didn’t want to go down that rabbit hole this evening. If it copies overnight, I’m fine with it. EDIT: That said, you got me very curious so I’ll 100% try that out tomorrow morning! :slight_smile:

@twin_savage I just checked, makes sense for the new drives to not have run, but the one in question was never optimised either so I’ll give that a go as well. Seems to be scheduled for weekly optimisation but for some reason it never run on this one. Good shout, I’ll let it trim as well, hope it improves.

2 Likes

I completely forgot about the SSD version of “have you tried turning it on and off again”. Time to explicitly add it to my problem solving notes.

2 Likes

Well, trim did nothing to speed things up. I wish I knew what the problem could be as both reads and writes to and from this drive drag buttcheeks, with spasms of glorious 2-300MB/s for like, a second or two, even mid-file.

I have files ranging in size from 5KB to 50GB, makes no difference in speeds no matter which one it tackles.

EDIT: watching these copy brings back PTSD from the good ol’ peer-to-peer speeds from early 00’s :))

2 Likes

NVME via Chipset should work, at least better then 5MB/s, but I would check it and swap the drives

1 Like

I know perfectly well it works because I’ve copied a 50GB archive from one on cpu lane to one on chipset lane and vice-versa, speedy 2-3GB/s transfers both ways, so we can rule that one out.

1 Like

Windows logs checked?

1 Like

What would I check for? No, I haven’t

1 Like

It’s almost acting like its ignoring the trim command (it should take along time if it hasn’t been run in awhile). I really really hate to even bring this up because I loathe unnecessary vendor specific utilities but maybe the corsair ssd toolbox would have more luck trimming it or at least reading and correctly interpreting its SMART data.

1 Like

I use Linux, but there should be logs for hardware issues
IO wait or WHEA_UNCORRECTABLE_ERROR something like this

2 Likes

The fact you actually get (some sort of) decent speeds point to an intermittent problem. Those are mostly s/w related, so Win-OS is the culprit. As suggested earlier, Linux may give you proper copy speed, but most likely you’re running into the issue with Win-OS that copying files is a single-thread occurrence. I’ve seen Linus (that of LTT fame) use a program called choeazycopy for decent transfer speeds on Win-OS machines. Never tried it, as I’m on Linux :stuck_out_tongue:

1 Like

Are you on Win11?

Windows 11 has had a number of issues since it launched in October. One of the most frustrating was how Microsoft’s latest operating impacted the performance of some types of NVMe solid-state drives. If you’ve upgraded to Windows 11 and have seen reduced performance, you aren’t alone.

Microsoft promised to release a patch to remedy the issue, and now it has made good on that claim with Patch KB5007262, released on November 22 via the Windows Insider (opens in new tab) program. Once downloaded and installed, it should boost your device’s performance. However, you’ll need to manually install the update since it isn’t going out as part of an automatic Windows update.

1 Like

Surely that would manifest on all drives, no? It’s only this one that’s crapping out and the fact that it completely randomly jumps and dips in speed confuses me greatly. Again - only on this one drive, heck even the old spinning rust drive is consistently around 50-75MB/s.

1 Like

I don’t know, “some types of NVMe” sounds like not all are affected

Logs checked?

1 Like