Hey Guys,
Just randomly my storage node went wonky, none of these disks seems to have failed but it’s still booting to emergency mode? I have no good way of parsing logs because the only access is through ikvm right now.
PVS Output
LSBLK output, highlighted drives are the one in the array
Are they healthy, from smartctl or some other diagnostic?
Did you build the array using sdX as the devices, or did you use uuids?
Try manually assembling the array using device uuids?
Any hints in dmesg?
Sadly it would appear that smartctl is not installed on oVirt node and there are no repos.
On mdam assemble verbose it stated that there was no superblock on sdf, but there’s a partition? I’m going to try assembling without that drive and see what happens? Does that sound good to you?
It’s worth a shot. I think if the array could have been assembled, even degraded, the kernel would have done so. i’m thinking maybe something happened to cause device naming to change, and the sdX labels don’t apply to the same drives they used to.
It’s been many years since I’ve used MD, my recollection is rather rusty.
In my experience, missing mdadm devices at boot time is usually an initrd issue. It happens frequently enough that I still create a dedicated /boot partition.
Make sure mdadm_udev is included in your initrd and reboot.
I’m thinking 2 of the drives may have failed…
