So, I am new to BSD. I have always just walked politely around it, but it seems that this time I am gonna need to have a talk with it, and I can use some help.
The setup here is this. I have a Truenas VM running on Proxmos to which I am passing an LSI-SAS2308 HBA, and there are 4 6TB HDD attached to this card.
Currently, on those drives, there is a raid z1 zfs pool which has the only copy of some valuable data, and as such, formatting them is not an option, but I can not boot my VM with them attached to the system ( plugged in )
I had to create the pool in Proxmox because to my utter amazement, after setting up the whole thing, I found out that md arrays are not supported in freebsd, and I had a 4 disk array that backing up the data on them was the whole reason we are here in the first place, so I did the whole thing in Proxmox, and after that, I was able to boot the VM, pass the HBA, and import the pool in Trunas ( in that order ) without any issue. The problem is that now I can’t reboot the VM. Somewhere during this process, one of those disks has ended up with a boot flag.
Here is a screenshot of the Proxmox console when I try to boot the machine:
This is the output of the LSI bios. After that, the Proxmox bios comes up, and it says booting from the harddisk and gets stuck in there. That pool is not bootable:
Here, I even tried to see if I can use the VM bios boot menu, but it only sees my VM disk ( option 1 ) which is what I choose when I got this screenshot.
I can boot the VM with no problem if I unplug the disks, and I can plug them in after and see the pool, although I think I have done that one time too many since now it shows the pool as offline, and the only way that I could find online to bring the pool back online is, apparently, to reboot the marching which brings us back to our original issue.
I tried to reinstall the Truenas, hoping that the installer would reset all the boot flags accordingly, but that didn’t change anything. I then used the installer CD and booted into shell and tried to see if I could use fdisk or something to check the flags on those disks, but the command interface is different which is why I need help here.
First of all, it seems that I only have access to fdisk and gpartd command. Some docs online mention gpt and gptboot command but I don’t have them in that shell. The only thing that I could find was that gpartd let you set and unset schema specific flags ( in my case, GPT ) of which “bootme” is one, but I can’t find a way to see the current attributes or their values of a given disk. So here are my questions:
1- what is INT13 ENTRY? is it the same thing as MBR boot flag?
2- How can I see the current GPT flags of a disk in freebsd? gpartd show and list don’t seem to show any GPT specific value.
3- how can I bring an offline zfs pool back online?
4- and I came across this situation while I was going through this, so is there a way to reclaim a pcie device after the VM to which it was passed is shutdown, not to be passed to another VM, but to be used on the host machine?
Thanks, everyone for reading this, any tips would be really appreciated.
