At my wits end (AMD RAID 5)

I apologize for the length, but I have already done hours of troubleshooting and want to put as many details as I can to avoid suggestions of things I have tried.

 

So maybe this is the wrong sub-forum as I don't actually think the problem is with my HDD's, but something is whacky with the storage array I am trying to build.   Here is the setup, I am trying to use this as both a file server and a media center so I am running Win 7 (yes I know not ideal for a file server but it works and I need the usability of the desktop)

 

Proc: AMD Phenom II x6 1090T

Mobo: MSI 890GXM-G65

RAM: 16GB Corsair Vengeance 1866

GPU: Visiontek LP HD 7750

PSU: Seasonic 300W Gold TFX

SSD: 128GB Samsung 830

HDD: 3x 4TB Seagate ST4000DM000

Optical: Silverstone Slim BD-R

Add-Ins: ASUS PCE-AC66 Wireless

Silverstone EC-04 USB 3.0 Card

Rosewill RC-120 SATA 150 card

Case: In-Win BL040.300TB3LF

Yes there is a ton of stuff packed into a very small space, anyway...

I have 5 internal SATA ports, 3 of these are occupied by the 4TB seagates, 1 is my system SSD, and 1 is for an external HDD dock in a 3.5 bay.  I have the Rosewill card in there to add another ESATA port and to connect the BD-R drive.  My goal is to RAID 5 the 4TB drives.  When I set this up in the RAID BIOS on the MB this works fine, I am able to format in windows, and the drives show up as an 8TB partition.  GREAT! Time to transfer my files over from my externals.

 

Here is where things begin to get odd.  After a short time (exact amount varies, between 10 sec and 2 min I would say) either I get a blue screen with ahci64.sys showing as the issue, or the storage volume diappears and the file transfer has an error.  When I reboot the RAID BIOS shows the RAID 5 as critical, However the drive that is missing is still seen by the BIOS as an individual drive. (SATA Port 3 dropped out, the 4TB drives are on ports 2,3,4)

 

My first thought was bad HDD, Swapped the cables between ports 2 and 3, same failure during transfer and this time it is SATA port 2 that dropped out, SUCCESS I thought, I figured it out, but I should be sure.  So now I swap cables between ports 2 and 4 and try again.  PORT 3 AGAIN!? WTF??  Getting a funny feeling I swap cables between port 3 and port 5 (my external drive dock) and try again.  Bluescreens like normal, but this time no RAID failure.  Try this 4 times with the same results, so I put the only 4TB drive to not have dropped out yet on port 3 and try again.  Port 3 once again drops out.  OK I tell myself, maybe a bad sata port, so I avoid it all together and plug my externals into the PCI card, ditching the optical for now.

 

This seems to work for somewhat longer, but eventually blue screens again with the ahci64.sys problem as usual, only still no RAID going critical.  At this point I am suspecting the MOBO, conveniently I have a brand new spare on hand, I install that and am feeling confident.  Wrong.  Get it all set, start the file transfer, and wouldnt you know it, bluescreen sending the array critical with a dropped drive on port 3.

 

Right now I have the controller in AHCI mode so I can see SMART data, and am doing a full format on each drive, so far that is going well, even using port 3.

 

So I come to you guys for this.  Is there something that could cause the SATA interface to go wonky in RAID mode, and still carry over between motherboards?  The SATA controller is on the SB850 southbridge so I would expect changing the motherboard would have fixed it.  I am almost certain the drives are OK at this point based on the SMART data and the fact that the format is going OK.  I am fairly sure the motherboard is fine since the symptoms are identical on two boards (albeit identical model).  My only real thought at this point is that the CPU is somehow the culprit, maybe a bad hypertransport link or something.  It is an old and abused CPU so it would not surprise me if it was damaged somehow, but besides this odd quirk in RAID mode it seems to work fine.  I can pull the CPU out of one of my other rigs, but before I tear two rigs apart I figured I would ask for help.

 

This has been hours and hours of stress for me at this point so I appreciate the patience to read all of that and any help anyone may offer. 

Bump.

Sorry I don't have any help to offer, but I'd like to see someone try to help you out at least.

The only thing that jumps out at me is that you said the MB is the same make and model. Maybe that model MB doesn't do well with RAID 5... That's about all I can think of that you haven't mentioned already.

What version of Windows 7 are you running 32bit or 64bit?

64 bit Win 7.  The full format of the 4TB drives worked in AHCI mode with no problems.  I am trying RAID 5 with some extra drives I have laying around now (2x 640GB and a 320GB) to finalize that it is not the fault of the drives.  So far it did crash with a blue screen but did not boot out a drive from the RAID.  I am wondering if maybe the SB850 does not like having such a large array (8TB) even though this motherboard supports "2.2TB Infinity"

 

The main reason I am suspecting the CPU though is because I remember having issues with this CPU awhile ago with a RAID 0 array where it would drop out intermittently until I swapped ports around.  Was around when I was upgrading though so I never really tracked the issue down.

 

If anyone has a recommendation for a PCI raid card that will do RAID 5 on 4TB drives I would love to hear it also.  I have a Syba 4 port PCI card but it will not RAID 5 such big drives which seems to be a common issue with the PCI cards.  Not really surprising considering their age, but a PCI slot is all I have to use and I don't really care about speed.

OK, so to recap here is what I know so far.

 

PC BSODs with ahci64.sys showing as the issue, with 3x 4TB drives the RAID array will go critical with one drive falling out, but the RAID BIOS and Windows still see the drive just fine.  With 2x 640GB drives and 1x 320GB drives the system will BSOD but the array will not go critical.  I do not know why the 4TB causes the array to go critical, however it would not do this every time depending on what ports they were plugged into so I do not suspect the HDDs.  I have eliminated the following components as suspect.

Motherboard

RAM

HDDs

SATA Cables

PSU

 

I think I am going to swap out the CPU with an FX 8150 I have in another system and see if that fixes it.  Other than that I suppose it is possible that this MOBO just does not like RAID 5, in which case I will have to build an entirely new server :(

 

Well, tried the other CPU with no luck.  I may try a windows reinstall but I don't think that is it since i have tried a few drivers.  At this point I honestly think I give up, sadly my super compact server media center doesn't seem like it will happen despite my efforts to make everything fit.  

 

I wish AMD wasn't so shafted in motherboard selection, micro ATX AM3+ board are ALL based on 8xx series and older with most of them 7xx series chipsets which is just insane.  The mobo I am using isn't even made anymore and it is the ONLY AM3+ microATX board to ever support crossfire.

 

You look at intel boards and the vendors bend over backwards to add extra USB 3.0 and SATA 6 and WiFi+ BT on all of the SFF boards, nothing of the sort for AMD, even on the new Kaveri parts the selection sucks.  

Unfortunately, this falls onto mobo vendors and not actually AMD or Intel themselves. Because everyone perceives intel to be so far ahead of AMD at most price points (Not actually true, AMD is very competitive up to the 4670k range. It just doesn't have the ultra high end like intel does) the demand for Intel boards is much higher and thus Mobo manufacturers compete with various models and features to try to capitalize on as much demand as they can.

Because the demand for AMD parts is much lower, the selection suffers.

 

When i was searching for this issue in google, what i kept running into was that the driver package for x86 would install the x64 bit drivers along side the x86 ones even in only 32 bit systems which was causing issues. I didn't see anything posted about this happening to 64bit OS, depending on your RAM situation you might try running windows7 32bit and making sure the 64bit raid driver isn't installed. The other option would be getting it set up with something like FreeNAS or a linux distro and run SAMBA on it so it can integrate with other windows systems in the house.

Yeah I understand why, it just sucks, also somewhat of a self fulfilling prophesy.  If the selection and features are worse it may push someone to go Intel because there is a better feature set.  It really is unfortunate the bad rap AMD has.

 

Anyway.  I have found a few posts on the MSI forum relating to this board and RAID issues, but the pics have been deleted and they really don't go anywhere.  FWIW the BSOD error is DRIVER_IRQL_NOT_LESS_OR_EQUAL with the ahcix64.sys listed as the file.  At this point I have tried everything but a different model of mobo so I am thinking it has something to do with this particular board. Will try to reinstall windows fresh when I have a chance but I don't have high hopes.

 

Every computer in my house is AM3(+) based so it will be a shame to have to go with an APU if I end up with a new system.  Also going to suck on the wallet, but I really need to get my data off of my old 2.5" externals.

What partition style are you using for the raid? MBR or GPT?  

From my experience and I have quite a bit, the built-in RAID controllers on the mobo are crap. I have replaced motherboards over and over again because they kept dropping drives from the array. I bought a dedicated RAID card never had one drop again. I tried with an Asus mobo and a MSI one, same issue on both. It would set up the array fine and would run for anywhere between a couple days and a couple weeks but constantly drop 1 drive.

This seems like a much more money conscious step to try before replacing CPUs and MBs. Try to get a card that will allow you to expand your array in the future. If it doesn't work, you can probably return the card and then go about trying to solve your problem other ways.

Maybe so but after RMAing the mobo twice and switching to another brand mobo and having very similar issues, the RAID card is what ended up fixing it for me. 

I have run RAID 5 on an old P35 Intel board for the last several years without issue, it is just time to retire that machine.  This is my first experience with RAID 5 on AMD, though I have had RAID 0 and RAID 1 arrays run successfully on AMD for a year now.

 

I have looked into RAID cards, unfortunately the only slot I have open for one is a PCI slot (why the hell do they still put these things on mobos?) which most of the RAID cards on PCI will not support a 4TB drive, I have a Syba 4 port card based on a SiL3124 chipset but it wont work.  If anyone knows of a PCI card which will support RAID 5 with 4TB drives, I am open to suggestions.

 

As far as swapping CPUs and Mobos goes, up until now I have just been swapping and diagnosing with spare parts or parts from another system I already have, so no money spent.  I may have to get a whole new platform though if I can't get this to work.

I partition in GPT when Windows asks me to initialize the disk.  There is no option in the RAID BIOS so I just assume it does whatever it needs to.

Not sure how much this could contribute to the issue, but isn't that PSU cutting it a bit close if not short assuming perfect efficiency?

Can't seem to find that controller anywhere so I dunno how much it's pulling but it can't be too much. But assuming PCIE x4 is what I see a lot of the Rosewill cards using it'd be a max of 25W. So at peak usage that PSU couldn't handle it all.

Again, I have no idea if that's what's going on but that's the first thing I noticed with how much you have in that system.

I would probably just remove that graphics card, the system should be able to handle any media playback with onboard just fine.

Thought of this but I actually pulled the whole thing out of the case and ran it with a 520W corsair to diag and had the same issue.

 

While 300w is a bit close, assuming 125W for CPU and 55W for GPU that leaves me with 120W for HDDs and everything which should be plenty.  Especially considering it is never going to see full load on both CPU and GPU at the same time.  Max draw I have seen at the wall is 250W when I load up prime and furmark just to test though.  Being a SeaSonic PSU I don't worry about running it close to rated power either, the internals are based largely on the x-series which are some of the most solid PSUs out there.

Also, that rosewill card is a crappy PCI card just for an extra eSATA port and my Blu-Ray drive, the storage is all connected to the internal controller (AMD SB850) using the AMD RAID option ROM to set up the array.  Still have not gotten a chance to reinstall windows which is really all I have left at this point to troubleshoot unless someone else has another idea.

IT WORKS!!!!  I did a full reinstall of windows and used the drivers off of MSIs website rather than the newer ones off of AMDs website.  Interestingly though this time I have to set the array up in AMD RAIDXpert software otherwise windows won't see it.  Windows won't see what I setup in the RAID BIOS, but the RAID BIOS won't see what I setup with RAIDXpert.

 

So......yeah.......  Whatever, I guess it works, I can deal with whatever problems that might cause later.

Well I guess that was premature.  Files transferred over fine last night which was a first, but crashed and kicked out port 3 again tonight while doing a network transfer.  Now it is doing exactly the same thing as before.  Nothing changed between last night and tonight so I have no clue what is going on.

It sounds like an issue with the MB handling the array logically. I looked at some PCI slot RAID cards on Newegg, and they both said that they support RAID arrays larger than 2TB... but nothing about individual drives. I would order one of those and try it out using a single drive attached to the MB as the boot drive, and keep the RAID array separate. If the card doesn't work, send it back and don't let them charge you a restocking fee since the literature on their site was incorrect. That small fib is probably forgivable.

The RocketRaid 1740?  I am looking at that now and have a support ticked open with HighPoint to see if it supports Win 7 and 4TB drives.  My main issue with that is it looks like it somehow rewrites the drives to where they cannot be reformatted by another controller if I retire them eventually.  Other than that speed is not critical for me so a PCI controller under ~$100 is fine.

 

I did find one more interesting thing.  When re-creating the array after it kicks a drive out, if I delete the Logical Drive (LD) then reboot the system, the drive that was kicked out of the array and shown as a single disk now shows up as a failed LD (it is the only disk left in the LD).  So it is like somehow it kicks that drive out into its own LD that cannot exist along side the original LD.  Very strange.

 Also, with the latest RAID drivers from AMDs website, the BIOS and RAIDXpert software now talk to eachother.

I posted on the MSI forums, hopefully they can help, from there I will contact MSI support before I spend more money.