ASUS likes to automatically reorder itself so that the windows boot manager thing is the first option. "Oh, I see you've got a windows partition, let's make that the first option" NO YOU BASTARD, THATS FOR MY VM! Seriously, haven't they heard of block device passthrough?
EDIT: Every time I see ASUS, I feel like I have to scream it. Mildly annoying.
Oh, definitely. The general "non-standardization" regarding UEFI is infuriating.
When I installed Arch the first time, I did so on a separate drive from my Windows install. I removed all other drives as a safety measure since I knew I was still learning and didn't want to nuke my other OS installation. I had read Grub is smart enough to work itself around a Windows UEFI boot manager, so I figured it would give me the option to choose later.
After spending longer than I'd like to admit installing Arch, I plugged in all my drives.
Boom. No Windows Boot Manager.
After a lot of research (literally days), I found out that some Motherboard manufacturers delete UEFI boot options if the system boots without the device. i.e. if my Windows HDD isn't in the system, it deletes the entry in the UEFI for the boot manager.
It isn't smart enough to add it back. So since I didn't know how to edit the UEFI from the EFI shell, or how to fix it using my Arch install, I had to reinstall Windows.
The thing is, I didn't find out that information (that it deletes the UEFI entry if the disk isn't found) until I repeated the installation process for Windows and Arch a couple times.
Literally a couple weeks without a PC.
But.... hey, now I know how to install Arch without much issue.
... until situations like this thread happen.
I honestly am probably going to give up on UEFI in general from now on. Using BtrFS and subvolumes should alleviate my partition separation preferences and allow me to use BIOS/GPT.
Yeah, I've got Windows in a VM, but that's it. I'm done with windows on bare metal. It's too rage inducing when dual booting. It literally nuked my solus partition a month ago. Not sure why.
BIOS/GPT is the best option if you can use it.
On the BTRFS note, did you sort out your other thread about the raid1 situation? I responded with a solution for removing /dev/sdb if that's what you were aiming to do.
Yes, I will reply there.
That's my goal with my Work PC and eventually my home PC. I play video games and would like to do GPU passthrough to Windows, but my home PC only has 1 GPU and my understanding is that I need two if I want do that. One for Linux and one for Windows.
Or I could just boot into Linux through an SSH session to boot Windows within it, but I'd rather not do that, as I'd prefer to use Linux as my "everything else that isn't video games" OS, so that wouldn't solve the issue, I think.
Thanks for the help to both of you. :D
I'll reply on Monday with what worked.
Nope. As long as you have one descrete (read plugged into a PCIE slot) GPU, you'll be fine. I'm going to do a guide on how to hot-swap your GPU between your X11 session and the VM, but I need to iron out the kinks first. (Disclaimer: you must shut down the VM to remove or add the GPU) I don't want to say too much because I'm an evil tease, but it's going to be awesome.
I am very excited for that. Thanks for letting me know. :D
Serious question though: Will it require UEFI? Or can I use BIOS/GPT?
I'd like to know so I'm prepared for whenever you do get time to post that.
I'm actually fairly certain that's the reason I always try to use UEFI beyond "it's new and more features." even though it can be a bear to deal with in some situations, because that's always been my end-goal.
The article on Arch Wiki about GPU passthrough sounds like it requires UEFI to work.
https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Using_vfio-pci
But yeah, don't worry about explaining any specifics here. I'll wait as patiently as I can. :D
Host boot method shouldn't matter, but VM may require UEFI. Not 100% sure though.
The archwiki is always only used as a starting point. It's used to point you in the proper direction and you've got to figure the rest out by yourself.
I'm going to go into quite a bit of detail about it. And, for the record, I'm 1000 words into the next article. (the previous article on ZFS was about 3200 words) I hope to have a Christmas gift for the forums :D
The thing is though, that I am doing something tonight as a gift that I want to document. It's going to be an ongoing "User Acceptance Testing" sort of summary. My mother's 2009 macbook is getting Linux. She's used to the mac, so let's see how she handles Budgie. (PGO on Solus should be a huge help here)
Now I'm even more excited. :D
That sounds perfect. I'll provision my main PC with BtrFS then in preparation.
Sounds good. Remember, chattr +c <vm-image-directory> is your friend for disabling copy on write for vm images. :D
Also, remember, this coming article is only going to be discussing at length VFIO, the IOMMU and how PCIe works. Having an understanding of how it all works will help people troubleshoot problems themselves.
That said, I'll probably be typing an article on the 25th.
So, I've updated the BIOS to the latest version.
To go over my information:
MSI Z170A PC Mate Motherboard
Samsung 850 EVO 500GB M.2 SSD
Hitachi 2TB HDD
My system will not view the M.2 SSD as a UEFI bootable drive even after I install Arch Linux following the guide (as I've done it with many other installations) using GRUB2.
The M.2 SSD is partitioned as 3 partitions using a GPT:
/dev/sda1 is the EFI boot partition at 500MB in size.
/dev/sda2 is the root file system at 483.5GB in size formatted as BtrFS.
/dev/sda3 is the swap partition at 12GB in size.
The 2TB HDD is a partitionless BtrFS drive.
Both drives appear in my BIOS correctly according to their ports. The Samsung 850 is not an NVMe drive, but a SATA based M.2 drive. It shows up as occupying my SATA 1 port. The manual says this is normal for non-NVMe M.2 drives.
I've tried installing Arch with BIOS/GPT, but that didn't work either. It will not boot from the SSD even if I make a 1MiB BIOS partition and install there.
I've noticed that older version of this motherboard (Z87 PC Mate) have a UEFI Hard Disk BBS Priorities menu under boot options to adjust the order. My BIOS lacks this entirely. It only has the legacy Hard Disk BBS Priorities.
I am able to boot into the Arch ISO and verify that I am booted using UEFI. I could just use Easy2Boot to boot to the SSD, but then my boot times would be exceptionally long waiting on a USB drive and having to traverse menus.
I'd also rather not need a USB drive to use my PC.
I've considered putting the EFI or BIOS partition on the 2TB Drive but having the root file system on the SSD, but I've installed this multiple times already trying various things.
I don't really want to waste more time with it, but I also don't know what would be sure to work in using my SSD as the boot drive.
I'm not above going crazy and hard linking everything from the HDD to the SSD, but I've never tried that and can't be sure it'd work either.
I'd put the ESP on the 2TB drive and give that a go. Sounds like this is a motherboard we should recommend against people buying.
You technically don't need to reinstall, just make a partition on the 2TB drive, format it, mount it and reconfigure grub.
I'll try that next. If that fails, I'll go as vanilla as possible and fully install Arch on the HDD with BIOS to see if it's strictly a motherboard issue. I doubt that will have a problem though since the HDD has always appeared in the BIOS correctly.
I installed the entire installation to the 2TB HDD using UEFI. It did not work.
I installed using BIOS to the M.2 SSD. It worked.
See, I think this motherboard will just not boot into a UEFI device. I have an Easy2Boot USB stick. It has a function where it can boot to BIOS/CSM, then swap out to UEFI. So even if the system can't boot UEFI directly, if it boots to the stick at all, it can essentially switch to UEFI once that's happened.
This is being a bit of a curse as it is causing a fundamental misunderstanding of what the issue is when I install on some systems. Specifically, my (current) main PC at work will not boot unless I use the USB stick to do this UEFI swapping, as I installed the system with UEFI and I know it has an EFI BIOS, but it seems to not be able to see the boot options.
So I've been able to install with UEFI, but not boot from it, and that's apparently my issue with this MSI motherboard as well.
It also seems like this motherboard is flaky. It sometimes shows the USB stick as UEFI bootable, and sometimes it doesn't, when nothing has changed with the stick.
Regardless, I found this and this actually worked, so I'll just do this with BtrFS subvolumes:
mkfs.btrfs /dev/sda
grub-install /dev/sda
If I had known it'd be that easy, I would've done that to begin with. The Arch Wiki entry for GRUB does say the following though...
Warning: GRUB strongly discourages installation to a partition boot sector or a partitionless disk as GRUB Legacy or Syslinux does. This setup is prone to breakage, especially during updates, and is not supported by the Arch developers.
I figured that's technically installing to a partitionless disk, since that's effectively what BtrFS is when it's formatted directly to a device, but I'm thinking they mean a separate disk without any kind of formatting?
Not sure, but this works, and my config goes in the same usual place:
grub-mkconfig -o /boot/grub/grub.cfg
So I guess I'll just use this installation like this and move on.
Though I will be keeping up with my ticket to MSI for them to fix whatever the issue is with UEFI on this board.
Only problem/quirk with doing it this way? No UEFI speedy boot, and wtv else requires UEFI, but also no swap file. BtrFS doesn't support them. That's super weird to me. I don't need hibernation, but still.
That's what I've been starting to think. Tough break. Glad you have the Easy2Boot to make troubleshooting uhh easier...
How old is the PC? If I remember correctly, it's an OEM that your work issued to you. If it's not too old, an RMA may be in order. Usually companies have quite good repair/maintenance plans with their chosen hardware vendor.
The whole idea here is that when you mkfs.btrfs /dev/sda, you're not installing to a partition defined by GPT, DOS or any other partition table. This becomes a problem because it allows BTRFS to use every last sector on the disk but you've used the first few sectors for GRUB. If btrfs wants to write to those sectors, you'll be stuck without a bootloader. My advice?
sgdisk /dev/sdacreate a 1MB BIOS boot on /dev/sda1 and use the rest of the device for /dev/sda2 and use btrfs on it. I can't say a partitionless disk has ever bitten me personally, because I don't put bootloaders on them, but it's happened to someone I know.
Up to you. My vote is to reconfigure it all, but I know you've been working on this for the better part of two weeks.
I'm curious what MSI has to say about the board. Have they said anything about the board's UEFI capabilities?
Honestly, I've never been super concerned with BIOS speed. I actually prefer a slow bios so I have time (my monitors are slow to wake) to hit the BIOS key since I'm always tweaking it.
That's interesting about swap files on btrfs. Have a look at this github repo. Basically the script creates a file of the desired size, sets no-cow (chattr +C), loop mounts the file and formats the loop device as swap. Pretty cool, but there has to be a reason for BTRFS not supporting swap.
Apparently swap relies on a function that BTRFS doesn't currently support. Source: BTRFS FAQ
Yeah, I never hibernate my PC's. It was nice when we didn't have SSD, but now with fast disk, it's cleaner to just boot and reopen programs. Time cost is seconds at most.
Yeah...
It's a PC which I selected the parts myself. We aren't a large enough company to deal with OEMs to be honest. It's brand new.
I'm not understanding.
you've used the first few sectors for GRUB
Did grub-install do that?
I found this while researching this:
https://bbs.archlinux.org/viewtopic.php?pid=1300101#p1300101
AFAIK btrfs leaves the first 64 KiB empty. You can use that to install a boot loader with btrfs support.
I can't find a source for that, but I did do a test. Having formatted both my SSD and HDD entirely as BtrFS, I output the first 64KB of the HDD (no bootloader) to a text file, then I output the next 64KB. The first 64KB appear to be empty while the next 64KB appear to be formatted in a specific way but also empty (nano thought it was a Mac format while it didn't comment on the first 64KB's size).
Not conclusive at all, but I would figure that implies the first 64KB aren't formatted.
Re: I found a source.
http://linux-btrfs.vger.kernel.narkive.com/EgoOKEP1/64kb-boot-sector-gap
Yep. Worried my boss might go "... are you still working on that?" lol
I'm fine with it as it is tbh.
Support has been down right atrocious. Here is the log:
The one cool thing I've found with this motherboard is that it has a feature to where if you hold the Power button for 4 seconds, it goes straight to the BIOS setup screen, so no amount of "fast booting" can keep you from it.
That's neat. Thanks. :D
Same. It's either off or on thanks to SSDs and UEFI boot.
Edit: Using gdisk on either the SSD or HDD also reports that there is an MBR but an invalid/corrupt GPT.
That's cool one way or another. New companies are fun. See what MSI says. I don't really have much advice for you on this front.
Interesting. You should be fine then.
If you want to see the data better, use hexdump or ghex or something. You'll be able to see some data that way.
True. If you don't have much time, it's fine, especially considering what you found about the first 64KiB.
Holy shit. That's quite something. I would recommend trying a supervisor or something. I knew MSI was so/so with support, but I didn't realize they were no help at all.
That's damn nice. I'll have to see if ASUS has something like that. I'm mostly ASUS boards.
Glad I can help.
I wonder why that is. I'll have to look into it.
Yeah, I noticed that hexdump command in a thread I was reading. That source pretty much confirms it though. :D Very useful to know about hexdump though.
Yeah... See, it's from bottom to top. Meaning, multiple replies in, he just replies with a copy/pasted list of the motherboard specifications. Pretty ridiculous.
Literally every message on the MSI site for this has broken english and notable typos. Even the emails do. It's.... disconcerting.
And you have to save and submit each of these as notes on a trouble ticket. It's not a direct chat. Pretty obtuse to use.
Funny thing regarding that. I have an ASUS X99 Deluxe at home. The most trouble I've ever had with a system for similar reasons.
Even if it's not normally bad for normal users, for me, it was hell. Remember how I mentioned I'd reinstalled Arch like 8 times with UEFI trying to figure out why it wouldn't boot? That was on this motherboard. It literally took months to figure out what the issue was. Partially my fault, but that implementation of UEFI is ridiculous.
UEFI shenanigans (auto-deletes entries if you boot the PC without the drive attached and doesn't re-create them)
Booting with a custom made keyboard plugged into any of the onboard USB ports causes the system to hang on boot. Using the supplied USB 3.0 PCI-e card works fine though. It works fine on other machines. It's just my machine and this motherboard.
There are other issues I've had that I've forgotten since they happened a year ago, but I deeply regret getting this motherboard.
I personally feel like ASUS falls into the same trap as other companies who produce everything. The best quality you can expect is "ok". Most of it works, but from what I've seen, little is exceptional.
Having owned a monitor, router, motherboard, and GPU from them, the thing I'm happiest with is by far the router. But it's a bit of a premium product, so I'd expect it to be (RT-68U). Then again, the Deluxe mobo is premium, but I guess since it's not RoG, it just had to be worse off in some way.
Sorry to rant. I've had so much trouble with these things, it just gets to me sometimes.
:)
I would guess gdisk just looks at the section after the first 64KB, and if something is there, it calls it a GPT.
Apparently, part of what limits MBR to 4 primary partitions is the 64KB space limitation. GPT expands this by using the next 2MB after that, which is usually reserved and kept from the file system partitions. That's what allows GPT to have up to 128 partitions, and can expand by going beyond that 2MB.
This used to be a problem because some programs would use that for things and write to it whether something was there or not. So you could destroy your disk partition layout just by using a program that didn't know better.
I'm just parroting what I've picked up in researching this. It's pretty interesting though.
I'd hope gdisk would be smarter than just "if something is in that 2MB, it's GPT", but that part is just my guess.
To troubleshoot this I would deconnect the second drive, and see what output F11 (boot menu) gives me. Then go from there.
Some tips from my own experience (UEFI can be very tricky):
- Once done installing Arch, check using efibootmgr whether a boot entry was added to EFI
- Beware: having a FAT32 partition is not enough for EFI! Make sure the esp flag is set properly for the partition. I have the feeling you might be missing this flag on your EFI partition.
- On your motherboard, you should have the option to add an EFI entry manually. If you chose that option, does it detect the M.2?
- If it does, you can browse to the folder containing grubx64.efi or bootx64.efi and point to it manually. Then see what happens when trying to boot from it that way.
So I checked my trouble ticket today being the first day back at work after the holidays.
So it doesn't support booting from the M.2 using UEFI. Wow. What a waste of time.
I now have the system booting from the M.2 using BIOS/GPT, so I guess it works out in the end, but my goodness.
Solved
