UEFI dual booting

[heatopher]

Hi Guys. As you’ll see below, I have been asking about this stuff on another forum, but I’m just coming on here to get a fresh perspective, and also maybe because this forum seems to assume less prior knowledge of things. Linux-specific forums tend to be a bit intimidating at times!

I want to get back to dual booting this PC (Windows+Linux) after losing Ubuntu because of a failure of GRUB when I was upgrading from 16 to 18 last year. Everything here revolves around my confusion about the settings in the BIOS (all images attached below). When I was first setting up this system three and a half years ago, I remember having a lot of trouble getting Windows and Linux to sit together in harmony. At the time it was Windows 7 (subsequently upgraded to 10) plus Ubuntu 16. I can’t remember every twist and turn, of course, but the basic problem was that Windows is greedy, and just knocks everything else off the disk, while installing Ubuntu afterwards knocked out the Windows MBR (well, I suppose that you experts know all that!). So in the end (and I can’t remember where this came from) I was advised to put GRUB at the start of the second (data storage) hard disk, and to boot off that, while having the two systems on the primary SSD. I had to use the Boot Repair disk to fix this, and finally it worked! And I didn’t think about this matter again until the upgrade failure last year

After that failure I switched back to booting directly off the primary SSD, which meant Windows only, but having a bit more time to think about this stuff recently because of the lockdown and being furloughed, I decided to get back to it, and so I wrote this post on the Ubuntu forum a few weeks back. The same guy that responded three years ago reminded me about the Boot Repair procedure, but also suggested that life would be much simpler if I started again from scratch with UEFI settings. Not only that it’s simpler, but also (if I’m understanding correctly) more flexible, in that I’ll be free to experiment with installing different Linux distros to see which one I prefer working with, without a load of Boot Repair hassle.

Now, if I’ve correctly understood this page about restoring old Windows disk images under UEFI (again, this is a BIG IF!!!), I can reformat the SSD to UEFI settings, and then restore the Macrium disk image of Windows without having to reinstall Windows completely. This goes against what the guys on the Ubuntu forum told me, that I would have to start again with Windows because the disk image wouldn’t be recognised in the new settings. Well, I don’t know, but I’m not afraid to experiment a bit, and in the worst case I’ve got my laptop!

OK, so this is what the storage looks like right now on my system. Firstly, here’s the SSD, where you can see Windows first (presumably with the Windows booting procedure right at the beginning) followed by the Ubuntu stuff that I currently can’t access:

And here’s the big HDD, where you can see the GRUB, and some other stuff that I don’t really know anything about (but maybe related to GRUB?), and then all the data partitions (which will need a bit of reordering, but no need to go into that):

I would like some clear understanding about UEFI, and why it makes life simpler and more efficient (because it’s not doing that for me yet!!!)

I’ve taken pictures of what I assume are all the relevant settings in the BIOS, but here be many dragons for me So this is what I’ve had up until now. I assume that the most important setting is the first one. I don’t know if I need to change that to “UEFI Only”. Everything else there means very little to me, and I have no idea if I have the right settings, or if any of them matter at all.

With those settings I am able to set an ordered boot sequence, as might be expected:

Slightly more detail here:

If I change over to UEFI Only, that list is dramatically reduced to this:

But when I try booting, nothing actually happens. If I press F12, I see this:

And when I proceed with that only available selection, I get this:

Obviously I do have the Macrium DVD/CD (whatever it was) sitting in the drive waiting to kick in, so I don’t understand what’s going on! This does get seen when I have the “UEFI and Legacy” setting.

I guess my main question, after all this overload of screenshots (SORRY!), is whether the “UEFI and Legacy” setting is OK to proceed with, as long as I follow the instructions correctly for reformatting the SSD in UEFI settings. I don’t really understand why UEFI is such an amazing upgrade if it excludes so many booting methods. The guys on the Ubuntu forum told me something about GPT, not all of which I fully understood to be honest, but anyway I get that that UEFI allows you to have partitions larger than 2TB (if I remember correctly). Anyway, anyway, I’ll leave it there!

Sorry if anything here is unclear (it’s very very very hot in the south east of England right now!), and I’ll be happy to explain a bit more.

Muchas gracias in advance guys. Sorry so long, but it has been a bit of a long story, which I hope to simplify a bit soon.

[Mastic_Warrior]

I am not going to lie and say that I read everything, but I think I get what you are after.

Not all hardware is UEFI compatible. Not all boot software is UEFI compatible. So if you turn off CSM, you may have hardware initialization issues, boot issues, and boot options may go away as MBR partition disks cannot be booted with UEFI.

The argument of things being easier mostly stem from WinTel’s (Microsoft’s and Intel’s) points of view.
How to you force people to upgrade to newer hardware to deliver a “better” user experience?
How do you lock down a platform from nefarious or malicious softwares (also competitor platforms) so that the bottom of the barrel computer device user can check their email and face boot?
How do we force people off of Windows XP?
How do we ensure proper system diagnostic and low level control of the boot sequence with out just hopes and prayers (a real Intel enterprise concern)?
How do we better implement future upgradeability without having a clear plan of where we are going? (Vendor concern)

The above are just some questions that efi -> uefi was supposed to help and solve. In general, the concept of UEFI was good, but the implementation allows the vendors to piss all over the spec and do what they want (see issues with ACPI handled by UEFI tables).

Now I used UEFI in the enterprise space and I have not problems with it. It is actually pretty great to drop into an efi shell and figure out why my drive will not mount, or see that I have some HDDs that are about to go bad and see what SMART is logging on startup from the BIOS level. It also makes it easier to perform firmware upgrades.

I also use UEFI at home with my AMD setups. There is not really much benefit there except to manually boot from a UEFI disk after a botched ArchLinux upgrade that broke UEFI stub or something.

Essentially, if all of your hardware is newer than 2016, it should be UEFI aware and you can turn off CSM (legacy mode). You probably wont notice a difference.

Basically like a Bill in Congress, the intentions were good, but what we got after all of the back room deals was something so transmutated that it seemed to cause more harm than good, at least initially.

[heatopher]

Blockquote[quote=“Mastic_Warrior, post:2, topic:160579”]
I am not going to lie and say that I read everything, but I think I get what you are after.
[/quote]

Ha! That’s OK You do indeed seem to have got the general idea of what I’m trying to do, and I think I get the overall idea of what you’re saying - that it’s mostly about cynical commercial motivations, as opposed to genuine upgrade. Why should I be surprised???

So does this mean that the problem is that my Macrium CD/DVD/whatever is not UEFI compatible? I’ve found this page about creating them, which talks about different PE versions.

My God, it’s all so much hassle!! And of course it’s all bloody Microsoft’s fault, if I’ve understood correctly. I’m getting close to the point where I’m ready to just exist with Linux and Windows VMs, but not quite there yet, so I will probably have to try creating a more recently updated Macrium DVD. And then let’s seeeeee. Thanks very much!

[Mastic_Warrior]

It very well could be. I would suggest turning off secure boot in your BIOS/UEFI and see if that works. It could be that the Rescue disk is not receiving a signature or the right signature during authoring.

I have not been in the MS Windows camp since 2001 so the only MS systems that I deal with are mostly in the enterprise world. We don’t really use Rescue Disks due to our expensive backup solutions.

[heatopher]

It was the rescue disk which needed updating. I was able then to boot into the CD, and followed, as faithfully as possible, the instructions on the Macrium page. So I do now have an EFI partition, an MSR partition and then the restored Windows partition (confirmed - I can see it all from Gparted USB), but… that’s not booting

I don’t know if it’s because the boot procedure is looking in the wrong place. Is it supposed to be looking in the EFI partition? When I was “fixing” boot problems in Macrium, one window asked where i wanted to boot from, and only provided one option - the C: drive. I ticked that, but I’m not sure that I should have.

Otherwise, is it something else that I need to change in the BIOS settings? I do have the UEFI SSD as one of my boot options, but maybe it’s one of those other options that I showed in my first post?

Or is there something else that needs to go in the EFI partition, which the diskpart procedure didn’t provide?

Well, no rush to reply, as I’m away from that PC for a week and a bit anyway. TIA

[Mastic_Warrior]

Sounds like, it did not create a boot entry in the UEFI. So you may have to drop into a UEFI shell and added the BSP location in the directories to search for the UEFI boot binary on that disk/partition.

[heatopher]

@Mastic_Warrior

you may have to drop into a UEFI shell and added the BSP location in the directories

OK, thanks again. Do I do that off a USB? And what does BSP stand for?

[Mastic_Warrior]

That was supposed to be ESP, not BSP (backspin was on my mind).

You should be able to do that from your bios or your bios boot menu. Details differ per vendor but most of them have it built in. IF not, then yes, you would need to put the shell executable on the thumb drive or CD.

[heatopher]

OK, well, I don’t know about the bios menu yet (not being in front of the PC), and I will try a bit when I get back, but if that’s no good then as for the USB stuff, would this link here be the right kind of thing?

[Mastic_Warrior]

yeah pretty much.
But you may need to pull a more generic shell.
https://wiki.archlinux.org/index.php/Unified_Extensible_Firmware_Interface#UEFI_Shell

[heatopher]

Hi, I’m back in front of the PC, and have spent all day going around and around in circles, but I suppose that even if I don’t seem to have made any progress at all, I must at least have learned something.

I couldn’t see any way to get a UEFI shell from the BIOS. Clearly it’s a pretty old motherboard by now, and I didn’t see much sign of any BIOS update. I also couldn’t really understand what exactly I needed to download from the the ArchLinux page that you posted. Honestly, some of those GitHub pages don’t explain anything at all. Very difficult for the uninitiated. Anyway, I did stumble across rEFInd in the midst of all the confusion, and put that on a USB. But rEFInd certainly didn’t find my restored Windows partition. rEFInd has an option to go into the UEFI shell, but to be honest I’m not really sure that playing around in there is going to make any difference. Of course I don’t know anything about this stuff, but it really does seem to me that there are many many dragons in this cave.

I did take a couple of screenshots while I was attempting to get the Macrium MBR>GPT route to work (countless times!), and the process looks like it went OK, apart from the fact that more often than not a fairly random incorrect drive letter is shown (here it’s H:, but I saw plenty of others, also including C:, but without any apparent sense of why it gets it right or wrong):

Anyway, I’m a bit tired of this now. It took me at least a couple of hours to get things back to how they were before, just to have any kind of functioning system (problems getting the MBR system back with Macrium). I can say that I’m well aware now of what UEFI is, and I guess I’ll be more than ready to grapple with it when I get around to my upcoming PC building project, but maybe I’ll be wasting my energy a bit if I keep flogging this horse. I can still put Linux on using the Boot Repair disk, even though it’s a rather inelegant process.

So I’ll call it quits for now. Thanks very much for your suggestions, and of course I’ll be happy to hear back if there’s anything that’s worth trying which I haven’t had a go at already.

[Mastic_Warrior]

I think you pretty much called it. Your MB may just have a lack luster implementation of UEFI. If you have a method that works now, then use that.

When you do the new PC rebuild, just ensure to use GPT partitions and make sure that all of your hardware is newer than 2015 and they all should support UEFI in some degree.

Sorry that your experience has not been the best so far. It gets better.

[heatopher]

Having said that I was ready to give up, I didn’t. I just decided that I’d be wasting less time to start from scratch with a new Windows installation, which of course is tedious, but a lot less annoying than all the time I wasted trying to transfer that Macrium Reflect backup from MBR to GPT. I can say from what I’ve just done that there’s no UEFI hardware problem with the motherboard, as I’ve managed to install Windows, Ubuntu and Linux Mint all together (just wanting to test out different distros) this evening. So the bottom line appears to be that Macrium’s claim about that transfer being possible is pretty bogus. I really followed the instructions to the letter, several times, and just ended up almost literally tearing my hair out. Either the claim is bogus, or there’s some extra detail that they forgot/neglected to mention on that page.

But anyway, onto the next steps now. I want to try installing rEFInd, just to have a nice graphical boot option screen, and then the idea is to try out Linux Mint versus Ubuntu, and have a look at other distros. I’ve downloaded Manjaro, to try something a bit different. Any other recommendations?

[Mastic_Warrior]

I am a Debian and ArchLinux shill.

I flirt with OpenSuSe Tumbleweed but I never use it as a main.

[jlittle]

@heatopher

Any other recommendations?

I suggest looking at KDE on some distro. A lot have KDE spins. If you don’t need up to date software, something based on an Ubuntu LTS would be good. If you sometimes do, and can tolerate breakages, maybe Manjaro KDE. I find Kubuntu a good compromise, usually.

[heatopher]

I suggest looking at KDE on some distro. A lot have KDE spins. If you don’t need up to date software, something based on an Ubuntu LTS would be good. If you sometimes do, and can tolerate breakages, maybe Manjaro KDE. I find Kubuntu a good compromise, usually.

Yes, as it happens Manjaro KDE is the next one sitting on my USB stick. I’m not really ready to go for it yet with Debian or Arch Linux bare bones installations. I could try them on a virtual machine some time, I guess, just to have a look.

Anyway, having said that everything is working fine… Oooops, I spoke too soon (which seems all too easy to do with this stuff). As I said above, after trying to install rEFInd, Windows got knocked out, and so I had to use Macrium to put the backup back (and glad at least to know that that worked). But unfortunately that seems to have interfered with both the Ubuntu and Linux Mint installations. They’re both stuck at similar points. This is LM:

And this is Ubuntu (I think I’ve got them the right way round, if that matters at all):

Both saying something about failing to bind an enclosure. Does that mean some link has gone? I’m sure that it’s Windows’ “fault”, whatever it is.

I don’t mind experimenting with this for a bit more. I hadn’t started customising either Linux install as yet, and as long as I know that I’ve got Windows going, I can do the day-to-day stuff. But of course I want to get it stable eventually. And stability seems to be all about understanding the workings of that EFI partition and getting it under control

To be very precise, by the way… I’ve got Windows 10, Ubuntu 18 and Linux Mint 20 Cinnamon.

[Hammerhead_Corvette]

https://wiki.archlinux.org/index.php/Systemd-boot

Since you are dual booting, have you considered systemd-boot it’s pretty clean…

[heatopher]

Since you are dual booting, have you considered systemd-boot it’s pretty clean…

Hi. No, I hadn’t considered that one, because I hadn’t seen any mention of it yet. I still consider myself very much new born to all this. As far as I can tell from very quickly scanning the documentation, you can only install systemd-boot from within Linux, and unfortunately I currently don’t have either installation working. Unless I can install it from USB?

What I need to know right now is whether there’s a quick fix to get the Linux installs booting properly. If not, should I just reinstall them both, and will it be possible just to overwrite the partitions that they’re currently sitting on?

(Gotta say, by the way - this forum is very friendly and responsive!)

[Hammerhead_Corvette]

@heatopher

As far as I can tell from very quickly scanning the documentation, you can only install systemd-boot from within Linux …
Unless I can install it from USB?

This is true. Considering you have a Live USB, of a distribution (Manjaro/Fedora/Arch etc…) You can mount your root and make all the necessary changes to your system from your live distro. chroot or systemd-nspawn cover those needs. If grub2 is corrupted, you could even regenerate your grub-config if needed as well, even reinstall it !

@heatopher

What I need to know right now is whether there’s a quick fix to get the Linux installs booting properly.

I’ll re-read your post within an hour, and see how I can jump in and help. (I have errands to run…)

[heatopher]

You can mount your root and make all the necessary changes to your system from your live distro. chroot or systemd-nspawn cover those needs. If grub2 is corrupted, you could even regenerate your grub-config if needed as well, even reinstall it !

OK, I think I understand in principle, but I’ve got no idea how to approach this. Let me just recap on my understanding up until now (everything will be followed by question marks!), and please correct me if there’s something fundamentally wrong about what I’m saying:

So, what I need to do is to get onto the EFI partition? If I can understand what goes on there, and take control of it, I will be OK? All the other partitions are specific to one and only one of the OS’s on the disk? (not talking about shared data partitions, of course!)