[Build/Dev Blog] Christmas Workstation Build (EFI RAM Disk Fun)

Am I understanding correctly that you are actually rewriting EFI to remove the hardcoded requirement of using FAT and replacing it by a different file system? Or at least offer the option of using a different FS for the EFI partition.

If so, and you get it to work, please make your patch/code available to the Linux Kernel devs, in order to properly implement EFI w/o the safety hazard that is FAT (or any other M$ file system :roll_eyes: )

1 Like

Oh god no, not the entire EFI. :rofl: Not really something you can ‘patch’ out because the board firmware ROM at the lowest level is looking for a FAT partition.

There’s an old pair of files - a driver RAMDISK.EFI and a utility MKRAMDISK.EFI - that can allocate a chunk of memory as a RAM disk but are currently hard-coded to initialize it with a FAT16 filesystem between 4MB and 512MB. I’m only removing that piece of initialization code from this particular utility and updating them to use larger integer sizes so I can allocate more memory than is currently possible.

3 Likes

I would like to murder the person who formatted this code in the EFI-Toolkit.

2 Likes

This is gonna take a while.

Debugging this mess is a fascinating experience. I’ve found that my original problems with getting a copy of RAMDISK.EFI and MKRAMDISK.EFI to work with the EFI shell weren’t related to the shell at all, but rather there’s problem somewhere in this source code release:

MKRAMDISK.EFI
main()
MakeDisk()
LoadDriver()
LoadImage()
GetDevicePath()
GetFileMapping() <env.c>


♫♫ Hello printf() my old friend, you’re fixing someone’s bugs again~ ♫♫

2 Likes

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

Found the problem. We’re back in business, disk is ready and FAT16 is gone. (0x8000_0000_0000_0009 is EFI_OUT_OF_RESOURCES which is exactly what I wanted to see, this only has 4GB of RAM)

Now to write the chunk of code that will do a block-for-block mirror of another device to this raw block device. Which is surprisingly easy now that I know what I’m doing. Kinda.


I suspect Linux will know how to handle this. But, I foresee the issue of Windows BSoD’ing immediately when it doesn’t know how to access the IO device it’s being booted from. Kinda like the old days of trying to boot a Windows XP ISO from USB and having it BSoD because it didn’t have USB drivers. We’ll deal with that when we come to it. I might be learning how to write Windows drivers too.

4 Likes

Am in the process of finishing up BLKMIRROR.C - a program which will accept two device GUIDs which have an associated BlockIO protocol and mirror one to the other. In theory, on the actual system it will be mirroring an iSCSI LUN/ConnectX-3 to the RAM disk in memory.

It’s going to be somewhat slow. I think. I’ve allocated a chunk of memory to act as a buffer, which it first fills with a BlockRead() from one device and then dumps to the other one with a BlockWrite(). Simple enough. I do not know how to perform both a read and write at the same time, in a way that I can double buffer the data and have it both filling one buffer and writing from the other simultaneously.

But, seeing that the destination device is just another place in system memory, it’s probably fine for this application. :person_shrugging:


… I seem to have completely broken printf() entirely and now it just hangs the system, will have to figure out what that’s about. :rofl:


Also, another thought occurs. This entire process will have to happen before GRUB launches (you need the RAM disk to already exist before GRUB is called), but also after the OS (read: block device) to mirror has been selected in the GRUB menu shown up there ^. That’s obviously a problem, logically.

Soooo, I’m probably looking at adding another layer to this:

  • System boots the GRUB.efi above which displays a menu.
  • Each menu entry can then chainload a different EFI RAM disk + block mirror shell script for each OS.
  • Shell script will then call a second, different GRUB.efi which would bootstrap the image that was just loaded.

Maybe. Haven’t thought this one through yet.

2 Likes

Maybe see if there’s something useful here?

1 Like

Yeah, you’ll need some kind of multithreading to get real concurrency. That’s if you want to read and write different data.

How to parallelize reads and writes on the same data without getting bogus results adds another layer of complexity on top of that.

Well that’s a fascinating little piece of software… I’ll have to take a look at that when I get to a point that it doesn’t completely derail me lol. Just reading the readme.txt and I fundamentally have no idea how they’ve implemented threads on the UEFI.

I wonder if I could write a separate driver image that would disconnect the existing BlockIO for both the source and target device and then attach a new BlockIO driver to both that could do a direct memcpy() from one to the other. Probably not.

image


I think the worst part about this whole thing is that I just spent all day trying to find where DevicePathFromHandle() is defined… just an example, but the TianoCore toolkits are just a shotgun blast of bits and pieces of UEFI and while there’s an Intel reference provided on what exists, there’s nothing that tells you where it is.

That’s probably what occupies the majority of time I spend writing code. Often it’s not hard to figure out conceptually what needs to be done, but if the code happens to be particularly(cough) eccentric, then it can take forever just to figure out where to do it, and what else might be affected by it.

1 Like

Just adding more fuel to the fire :slight_smile:

And the reason why it looks promising to me is that it has an example of how to create a producer/consumer workload using two different tasks/units of code, potentially running on two different processing units:



1 Like

image

Wow thanks for that completely useless page.
I’m getting very tired of the people that are maintaining this project. This is in the Driver Writers’ Guide - no useful information, no direction where to find information, just someone’s useless edit with no relation to the specification doc provided with the code and that’s it.

2 Likes

@Tzerne5, if you are talking about Zen 4 EPIC, we probably won’t see Zen 4 EPIC being available for OEM until sometime in the first quarter of 2023 and DIY until the late second quarter or maybe early the third quarter of 2023. I suspect Zen 4 EPIC won’t be available until the first quarter of 2023 because AMD will continue manufacturing Zen 2 EPIC until January; my statement is based on an article I read (probably Tom Hardware).

I imagine Zen4 EPYC ( → Threadripper) will be launching sometime early 2023 hopefully along with the Sapphire Rapids HEDT, that’s mainly what I’m waiting for. I think.

But, I haven’t really looked much into hardware yet… Keeping in mind this is for all intents and purposes just a regular bare metal install just booted weirdly, all I need for PCIe is 16+8 lanes of expansion and preferably an M.2 for literally any drive that can hold less than 20MB of this bootloader mess masterpiece. The only ‘issue’ is needing an asston of memory.

The kicker is I’d like as much clock speed as possible for VR in the gaming OS… honestly if I could just slap like 512GB of RAM on an i9-12900K or 7950X that would be great, but my baseline OS install with utilities/etc that sit on the OS drive is around 180GB, so I’d rather not drop to 256GB if possible.

103485945_128906782157780_5318645619815569829_n

2 Likes

Note to self for when I get back tonight.

This code base is abhorrent. More on that later.

Waiting patiently for the rant

1 Like

It’s just… there are so many places that the code violates its own specification that ships with it, and I’m here having to rewrite chunks of the library so that my application that does follow the EFI spec will run on it.


image

…this is how you would write a console application in C and is how most of the provided applications are formatted, and does appear to work… somehow… but -


- is how it should be handled for an EFI executable, so that the EFI environment services can be accessed through the EFI_SYSTEM_TABLE, and command line arguments are contained under the EFI_HANDLE.


image

malloc() and free(), again, appear to function as as expected, but cause issues when interacting with the EFI library functions. They should be using the EFI runtime service memory functions AllocatePool() and FreePool() instead.

image


image

Same with printf() should be using Print(L"").


image

And variables should be using the EFI definitions of UINTN/UINT32, CHAR8, CHAR16, etc.


image

There are places all throughout the libraries where instead of dynamically allocating a buffer of the correct size, there are just some magic numbers that are “bigger than they should need to be” defining the size of things. Which has caused problems as PCs have a lot more hardware now than they did ten years ago.


image

image

They’re trying to make this available globally but this person obviously doesn’t know how extern works. Also why are we giving shit single letter variable names.


image

image

image


image

Same thing again, this should be:


Things return SUCCESS when they fail.

Initialization functions report the value requested by the user in their status messages instead of the actual value used after being adjusted by MIN/MAX/etc. (“Successfully created 2048MB RAM disk” when it only makes a max of 512MB).

Etc etc etc

It’s not all bad to the point of needing rewritten, but there’s obviously either one person or a team of people that wrote a significant chunk of the TianoCore library that have the qualifications of a freshman CS student in their first Intro-to-C++ class.

Not trying to be an ass here but this is why I look down on people who claim to be ‘software engineers’ until they’ve proven they actually know what they’re doing. Some of these are released with Intel’s name on them. Sorry. It’s getting frustrating, and if I asked someone at work to write a library for something and got this, it would be scrapped. Times like this I understand why people complain about silicon valley tech company interviews so much, it’s to weed this sort of thing out.

2 Likes

Why do you want to go through the EFI pain rather than create a tmpfs in the initramfs, rsync a filesystem there, then switch_root? *reads* Oh, Windows…

1 Like

Something I’ve also had interest in, but never pursued because I know it’d be a hassle and I don’t even have the RAM or NICs to make it worth a damn yet.

1 Like