[Devembary 2022] Universal RAM Disk Bootstrapper

NorthernWing · December 3, 2022, 4:02pm

This is an enormous blob of text, so I recommend using the Table of Contents over there.
Maybe not fair for Devember since actual work started in the beginning of November in the other thread, but someone might find it interesting anyway so here we go. Although most of that was me just deciding how to even approaching this.

1 - Introduction

1.1 - Overview

This project exists because of two changes I wanted to make to my home system that I’m rolling into one:

Move all local storage except for the boot drive onto more reliable network storage.
Instead of building another workstation PC, pick a way to run one of several operating systems on this one. Windows for gaming, Linux for development, etc. Probably through a hypervisor.

And so, this is the solution that came from trying different things in the old dev blog thread:

Create and store several OS images on remote storage over InfiniBand.
At power-up, use the UEFI to install an empty RAM disk in memory.
Use GRUB or a similar bootloader menu to select an operating system to load.
Each GRUB menu entry will chainload an EFI binary that will mirror a corresponding OS image onto the RAM disk, find the OS’s EFI partition, and execute it.

There are other, easier ways of doing this. Windows PE loads from a RAM disk in a way that could theoretically be ported over to desktop Windows, and the Linux kernel has about a dozen ways of creating a RAM disk. But by creating a block storage device in memory, in the EFI environment before any OS is even located, this loader can be made completely independant of what’s actually being loaded.

All it needs are the correct architecture, any device with an OS that can do block reads, and enough RAM.

Also I just think it’s neat.

1.2 - Goals

Practical goal, create a flexible + functional RAM disk bootloader and utilities so I can get started spec’ing hardware for my new monster desktop PC.
Familiarize myself with the UEFI specification and how the UEFI and its software actually functions. Things like the EFI System Table, how ‘protocols’ and ‘images’ are handled, etc.
Familiarize myself with the UEFI codebase/API and the fundamentals that will be needed in pretty much any UEFI application.
Stretch goal, but it would be a good time to clean and organize my projects folder, which at this point is scattered across two PCs and several flash drives. Time to give everything an SVN repo so I can go back to something I’ve worked on a year ago and actually know where I was at.

1.3 - Deliverables

Image of the EFI partition containing the parts of this project integrated together (GRUB, my own utilities, etc). Or, the image that anyone who just wants to use it can download, unzip to a flash drive, do some minor edits to a configuration file for their system, and be good to go.
Miscellaneous UEFI utilities. (blkmirror.efi, ramdisk.efi, efiloader.efi)
Source code for anything I’ve made as noted above.
Documentation on using said source code.

I’ll be hosting the deliverables for this one on the same little ~~cash register~~ web server that hosted my stuff for the last Devember for a few months. Will update the top of this post with a link once I have enough content to put it up.

2 - Project Components

2.1 - `blkmirror.efi`

The majority of the actual work in this project.

2.1.1 - blkmirror set

The EFI device shell mappings (BLKx, FSx, etc) can/will change depending on the current hardware/partitions present. The SET, GET, and CLEAR commands can be used to add/remove a nonvolatile EFI environment variable which will store the absolute device path of a device mapped in the EFI shell to save a unique device across reconfigurations.

Creating a new device alias:

Updating an existing device alias:

2.1.2 - blkmirror get

Displays an existing device alias for debugging purposes.

Querying an existing device alias:

2.1.3 - blkmirror clear

Removes an existing device alias from nonvolatile memory.

Removing an existing alias:

2.1.5 - blkmirror transfer

Starting at LBA 0, mirror all blocks of one EFI shell block device to another.

If the source device is smaller than the destination, leftover blocks will be erased.

If the source device is larger than the destination, data which cannot fit will be truncated.

Transferring between two devices of equal size:

Transferring to a larger device:

Transferring to a smaller device:

2.1.6 - blkmirror ramdisk

Load the RAM disk driver, initialize, and map a raw RAM disk block device in memory.

Creating a new RAM disk:

2.1.8 - blkmirror zero

Write 0 to all bytes on a specified device or device alias.

2.1.9 - blkmirror boot

Scan a block device for a GPT partition with a valid EFI partition signature and attempt to load the EFI executable on that partition at the specified filepath.

2.2 - GRUB

2.3 - 56G InfiniBand

3 - Walkthroughs

3.1 - Setting up the TianoCore EFI-Toolkit.

I’ll be using the original TianoCore EFI Toolkit for this, for no reason other than it’s simpler. I haven’t tried GNU-EFI, so can’t comment on it, but the newer TianoCore EDK2 appeared to be bloated with libraries and framework I have no intention of using, so I’m skipping it. The EFI Toolkit implements all of the EFI function calls and definitions provided by the Intel UEFI specification - TianoCore is a reference implementation of the EFI standard - and so it’s more than adequate for our purposes.

The EFI Toolkit can be found here:

To start with the EFI Toolkit you need a C compiler + linker, and a Make utility. Your favorite build environment for C will probably be fine here if you have one.

If you want zero additional configuration of your setup environment, the EFI Toolkit is configured out-of-the-box for Windows using the Windows Driver Development Toolkit, version 5.2.3790.1830, installed at C:\WINDDK\3790.1830. I’ll be using this syntax in examples from here onward.

This is a compiler that’s two decades old which you’d expect to be hard to find, but is up on WinWorldPC at:

Add the bin folder to your environment PATH variable so you can run nmake from a command line and you’re good to go.

The EFI Toolkit includes several reference projects like osloader and mkramdisk that we won’t be using, but to make a new project the easiest way will be to copy one of these folders and update the projectname.mak file with your new project’s name.

From there, open a command line in the toolkit’s top directory, run build to set the environment variables (this file controls the architecture being targeted, file locations, and a few other things you might want to check), and then you can either nmake all to build all projects, nmake clean to clear objects that are already built if you have a problem, or cd <project> nmake -f project.mak to build only one specific project. Your EFI binary will be placed at build/<architecture>/bin/.

Other useful documents you’ll want to read are the EFI specification linked below, and the library + design guides included with the toolkit at /doc/EFI_DG.pdf and /doc/EFI_LIB.pdf.

https://uefi.org/specifications

3.5 - Configuring GRUB.

3.6 - Configuring the InfiniBand network.

3.7 - Installing an operating system.

4 - Errata

4.1 - Known Issues

The blkmirror tool searches for the RAM disk driver “RAMDISK.EFI” at the absolute filepath of /EFI/SHELL/RAMDISK.EFI. I can improve this behavior in the future but I don’t feel like taking the time to figure out how to programmatically get the directory that blkmirror was called from right now.

4.2 - Warnings

Be very careful with blkmirror transfer. There is no confirmation before proceeding, and there is little to no recovery from sequentially writing every byte of a drive starting at LBA 0.
On a UEFI which presents itself as a block storage device (my test system appears to do this for ‘BIOS’ flashing), it may be possible to brick a motherboard entirely by calling blkmirror to overwrite the UEFI firmware without a USB BIOS flashback controller present. Fortunately these devices don’t appear to be mapped by default in the UEFI shell, so this would at least take some amount of effort to do.

5 - Dev Log

GigaBusterEXE · December 3, 2022, 8:11pm

for real though I’m looking forward to it

NorthernWing · December 5, 2022, 2:11pm

2022-12-05 | Dev Log

Writing EFI code continues.

The actual ‘block mirror’ part of BLKMIRROR.EFI - the utility that will mirror a network device to the local RAM disk - is officially done. It can read from any 512B/4096B-per-sector device and transfer that to any other device, either zero filling leftover space or truncating data if the source is larger than the destination, with all of the edge cases now handled when you’re going from a device of one sector size to another.

Work has begun on writing the RAM disk UEFI driver and initialization part of the tool, more to come shortly.

Well then. I don’t think I’ve ever gotten so much code written in one day that I’ve sat back down and found things that I don’t even remember doing that are already finished lol

Here’s a snip of where it stands right now - the blkmirror.h file that defines anything that’s implemented and complete so far.

#include <efi.h>
#include <efilib.h>

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// CONFIGURATION                                                                                                      //
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#define BLKMIRROR_BUFFER_SIZE           8388608

#define RAMDISK_DEFAULT_DRIVER_PATH     L"EFI\\SHELL\\RAMDISK.EFI"
#define RAMDISK_DEFAULT_BLOCK_SIZE      4096

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// DEFINITIONS                                                                                                        //
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#define EFI_VARIABLE_NONVOLATILE_GLOBAL 0x00000007
#define EFI_VARIABLE_VOLATILE_GLOBAL    0x00000006

#define MIN(x, y) (x < y ? x : y)
#define MAX(x, y) (x > y ? x : y)

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// blkmirror.c (MAIN) --> ENTRY POINT                                                                                 //
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
extern EFI_GUID GUID_SHELL_DEV_PATH_MAP;
extern EFI_GUID GUID_LOADED_IMAGE_PROTOCOL;
extern EFI_GUID GUID_LOADED_IMAGE_DEVICE_PATH_PROTOCOL;
extern EFI_GUID GUID_DEVICE_PATH_PROTOCOL;
extern EFI_GUID GUID_BLKMIRROR_ALIAS;
extern EFI_GUID GUID_BLOCK_IO_PROTOCOL;

EFI_STATUS GetMaxVariableSize(
   IN  EFI_SYSTEM_TABLE * systemTable,
   OUT UINTN            * maxNameSize,
   OUT UINTN            * maxDataSize);

EFI_STATUS GetDevicePathFromVariable(
   IN  EFI_SYSTEM_TABLE * systemTable,
   IN  CHAR16           * deviceName,
   IN  EFI_GUID         * variableType,
   OUT EFI_DEVICE_PATH  ** devicePath,
   OUT UINTN            * devicePathSize);

EFI_STATUS GetHandleFromDevicePath(
   IN  EFI_SYSTEM_TABLE * systemTable,
   IN  EFI_DEVICE_PATH  * devicePath,
   IN  UINTN            devicePathSize,
   OUT EFI_HANDLE       * deviceHandle);

EFI_STATUS ParseCommandLine(
   IN  CHAR16           * command,
   IN  UINTN            commandSize,
   IN  CHAR16           ** args,
   OUT UINTN            * argc,
   OUT CHAR16           *** argv);

CHAR16 * ByteCountToString(
   IN  UINT64           count);

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// blkmirror-set.c                                                                                                    //
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
EFI_STATUS SetDeviceAlias(
   IN  EFI_SYSTEM_TABLE * systemTable,
   IN  CHAR16           * aliasName,
   IN  CHAR16           * shellMapName);

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// blkmirror-get.c                                                                                                    //
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
EFI_STATUS GetDeviceAlias(
   IN  EFI_SYSTEM_TABLE * systemTable,
   OUT CHAR16           * aliasName);

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// blkmirror-clear.c                                                                                                  //
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
EFI_STATUS ClearDeviceAlias(
   IN  EFI_SYSTEM_TABLE * systemTable,
   IN  CHAR16           * aliasName);

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// blkmirror-transfer.c                                                                                               //
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
EFI_STATUS BlockTransferAlias(
   IN  EFI_HANDLE       mainImageHandle,
   IN  EFI_SYSTEM_TABLE * systemTable,
   IN  CHAR16           * aliasSource,
   IN  CHAR16           * aliasDestination);

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// blkmirror-direct.c                                                                                                 //
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
EFI_STATUS BlockTransferDirect(
   IN  EFI_HANDLE       mainImageHandle,
   IN  EFI_SYSTEM_TABLE * systemTable,
   IN  CHAR16           * deviceSource,
   IN  CHAR16           * deviceDestination);

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// blkmirror-ramdisk.c                                                                                                //
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
EFI_STATUS CreateRamdisk(
   IN  EFI_HANDLE       mainImageHandle,
   IN  EFI_LOADED_IMAGE * mainLoadedImage,
   IN  EFI_SYSTEM_TABLE * systemTable,
   IN  CHAR16           * aliasSource,
   IN  CHAR16           * aliasRamdisk);

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// ramdisk.c                                                                                                          //
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
EFI_STATUS InitializeRamDisk (
   IN  EFI_HANDLE       mainImageHandle,
   IN  EFI_LOADED_IMAGE * mainLoadedImage,
   IN  EFI_SYSTEM_TABLE * systemTable,
   IN  UINT64           ramDiskSize,
   OUT EFI_HANDLE       * ramDiskHandle);

EFI_STATUS LoadRamDiskDriver(
   IN  EFI_HANDLE       mainImageHandle,
   IN  EFI_LOADED_IMAGE * mainLoadedImage,
   IN  EFI_SYSTEM_TABLE * systemTable,
   OUT EFI_HANDLE       * driverHandle);

EFI_STATUS GetRamDiskDeviceHandle(
   IN  EFI_SYSTEM_TABLE * systemTable,
   IN  EFI_HANDLE       driverHandle,
   OUT EFI_HANDLE       * deviceHandle);

EFI_STATUS MapRamDisk(
   IN  EFI_SYSTEM_TABLE * systemTable,
   IN  EFI_HANDLE       ramDiskHandle,
   IN  CHAR16           * ramDiskAlias);

Nic_s · December 8, 2022, 6:56pm

Just wanted to make sure I understand this… the idea is to have, for example, a PC which you can then at boot specify what OS to load from over the network from a server? Kinda like starting up a container from an image hosted on a local server?

If so, this is very interesting… way above me, but interesting.
Would changes made, like installing software, be stored somehow so that it will just be there the next time you start it up?

NorthernWing · December 8, 2022, 9:46pm

Yep, that’s a pretty good summary. I’ll be using a network server in this instance, but actually it can load from anything that supports the Block I/O protocol in UEFI, which is a surprising amount of stuff. Like tape drives for example.

This is just good old GRUB and none of the OS entries call anything yet, but that’s the end goal eventually, and proof of concept that GRUB and Linux at least can both see a RAM disk built by the UEFI out-of-the-box.

That’s the plan, but I haven’t decided how I’m going to handle this one yet.

At first I thought about using the OS to be constantly mirroring changes made to the RAM disk back over the network, but if you have something like Blender caching physics calculations to what it thinks is a hard drive at like 20GB/s, you’d run out of buffer space really fast.

My plan right now is to figure out how to make the system not shut down entirely when the OS tells it to, but to drop back to a UEFI shell and then do the reverse of what it did when it booted and send an image of the current OS RAM disk back over the network to the storage server, which will then commit it once the whole thing is received to avoid corruption if it’s interrupted. I imagine there’s a way to catch the ACPI power-off call somehow, but that’s just a guess and I haven’t looked into this at all yet.

That’ll probably be January’s problem to solve, along with possibly writing a Windows storage driver so it can actually boot from this thing.

NorthernWing · December 9, 2022, 12:47pm

2022-12-12 | Dev Log

The RAM disk driver part of the project is done:

Yeah those pictures suck and my monitor in the office needs cleaned, but you get the point… that’s the best I can do at the moment.

There are two things left to do with this part of the code:

Right now, the RAM disk protocol is created and initialized but the device isn’t mapped by my blkmirror tool. You have to run map -r in the EFI shell which will assign the RAM disk an EFI shell device name. I’m in the process right now of figuring out how to programmatically find and modify the EFI shell mapping table, which is possible but not exactly trivial.
Since I’ve already implemented 100% of the required code, I’m adding a blkmirror zero <device> command that will just zero-write an entire block device. Because why not.

NorthernWing · December 15, 2022, 12:09am

2022-12-14 | Dev Log

Unfortunately no pictures this time, but I’ve started on the boot part of the “RAM disk bootstrapper”.

What this really means is that I’m currently neck deep in address maps of the GPT partition scheme, MBR compatibility, different partition GUIDs, and how to actually traverse a partition table.

This is where we stand:

~~Create a RAM disk in memory.~~
~~Transfer an image from a device onto the RAM disk.~~
Traverse through all GPT partitions on the newly copied RAM disk, looking for one which is an EFI system partition.
Attempt to mount the EFI partition which should contain a valid FAT32 filesystem.
Load and execute the file passed to the blkmirror boot command as an argument, which in our case is the bootloader of the actual operating system. For example, bootmgfw.efi for Windows.

Unlike the last couple portions of code, I don’t see anything that I don’t really know how to handle. It’s just going to be a lot of calculating offsets and finding the right data fields in each partition, etc etc, and for that reason I suspect this one’s going to take a while. Debugging is going to be a royal PITA.

NorthernWing · December 15, 2022, 12:18pm

2022-12-15 | Dev Log

My new boot drive for this system is on its way:

They didn’t have the 128GB version of this thing or I would’ve just bought that instead.

NorthernWing · December 20, 2022, 8:24pm

2022-12-20 | Dev Log

Got a lot of little things finished, but not a lot that’s interesting to read about. Mostly cleanup and writing out dumb code that I didn’t feel like dealing with.

Finally wrote some functions to get the file path of the currently running image, so it can call on external files relative to itself and not at an absolute file path. Sounds dumb, but less so when you realize there is no filesystem. TL;DR: put RAMDISK.EFI beside BLKMIRRIR.EFI instead of at /EFI/SHELL/.
Cleaned up memory management calls and made it uniform across all functions. No change in behavior but it sucks a lot less to work on.
Added an Apache License 2.0 header. Woo.
Dusted off the ~~cash register~~ web server to put my little domain back online and host this pile of stuff for people to look at.

Etc etc. A more interesting update should be following soon™, because I’m almost done with BLKMIRROR BOOT which means it’s almost time for a demonstration of booting a PC from weird storage devices.

NorthernWing · December 28, 2022, 7:18pm

2022-12-28 | Dev Log

Christmas and setting up a home office both kinda brought this to a halt for the last week but I’m getting back to work on it now lol

The find-a-boot-partition-utility is becoming more involved than I expected, but here’s something to keep everyone entertained until that’s done:

I just propped up the phone to record while I flew through testing something, but what you’re seeing is:

Boot into the EFI shell, move to FS1: and navigate to EFI/SHELL/ which is my flash drive partition with the shell + all the dev stuff.
blkmirror ramdisk 200000000 fsz - Creates a RAM disk, 200MBytes, mapped as FSZ.
blkmirror transfer fs0 fsz - Mirrors FS0 which is the Windows EFI partition for the PC I’ve been testing this on, onto FSZ.
mount fsz to mount the filesystem that should now be on FSZ. vol just to see that it’s there.
Navigate to EFI/Microsoft/Boot which is where BOOTMGFW.EFI - the Windows UEFI boot loader - is stored, and run it. It predictably fails because there isn’t anything to actually load.

NorthernWing · December 29, 2022, 5:24pm

2022-12-29 | Dev Log

Yeah my workbench screen is filthy and my phone is propped up against a soldering iron again, but you get the point lol. I can point to a block device, give it a file path, and it will try to find and execute that file on any partitions/filesystems it finds on the device.

I’ve got the tools now to allocate memory, copy something to it, and then find + launch an EFI executable. Now it all gets scripted together (…or I might just make one big function that does all three…) and integrated with GRUB.

NorthernWing · December 30, 2022, 1:51pm

~~Aaaaaaaaaaaaaa it’s so close to working as intended~~

~~After loading the Arch install ISO (it’s small, I’m not testing this thing with a 50GB OS install every time), it almost boots from RAM.~~

I’ve come to the conclusion that I still have a lot of work to do. I need to write a module that will add an SSDT table to the ACPI tables which describes the RAM disk interface to make it visible to the OS once loaded.

This is going to take a lot of work to figure out.

https://docs.kernel.org/firmware-guide/acpi/namespace.html

2022-12-30 | Dev Log

So I’ve learned a lot today, after realizing that I’m missing some critical abstraction ontop of the RAM disk as seen by the UEFI.

We need to dig into the ACPI specifications to make our RAM disk visible to the OS. The UEFI has a pointer on the system table, which points to a structure called the Root System Description Pointer (RSDP). The RSDP contains some header information and a pointer to the Extended System Description Table (XSDT), which contains pointers to the Fixed ACPI Description Table (FADT), which describes fixed system hardware, but more importantly it also contains pointers to the Secondary System Description Tables (SSDT) which describe expansion hardware that can’t be enumerated and so needs described to the OS in an ACPI table.

Make sense?

There just so happens to be something we can use here that both Windows and Linux will understand - the NVDIMM interface. If I allocate a RAM disk in the UEFI and load it with whatever OS (this part’s done), then install an SSDT with the parameters of that RAM disk stating that it’s an NVDIMM, both Windows and Linux should recognize when they enumerate the ACPI tables that they need to load a driver for that device.

Windows has what they appear to be calling the “Storage Class memory bus driver (Scmbus.sys)” to do exactly this. And if Windows has it, I imagine Linux at least has something similar.

TL;DR: The OS can’t ‘enumerate’ a RAM disk because it’s just a chunk of system memory. It needs something to tell it that the RAM disk is A) present, and B) what its parameters are. So we need to use the ACPI table to tell it there’s a storage device on the memory bus, and the OS will load a driver for it.

NorthernWing · February 1, 2023, 2:32pm

2022-02-01 | Dev Log

~~I’m still alive, I promise. I see a lot of posts over in the other Devember thread that I haven’t read yet, so I’m guessing things are wrapping up.~~^{^{Nope I just haven’t checked L1T for that long lol}} Our office moved about 40 minutes north and so did I, have been otherwise occupied since Christmas selling + buying + renovating a house, again, but I’m finally moving things and had a chance to look at this again.

I will make this POS work.

I’ve learned an insane amount about working with ACPI but have unfortunately reached the end of both my knowledge, the UEFI/ACPI specification, and the information that’s publicly available.

After generating and installing an NFIT (NVDIMM Firmware Interface Table) and compiling an SSDT (Secondary System Description Table - this is written in an entirely different language with its own compiler), I’ve reached the point where Windows enumerates the Storage Class Memory root device and correctly reports the chunk of memory as reserved by hardware (2GB here), but is reporting an invalid ACPI configuration.

There is no documentation past this point, and I don’t have a Microsoft FAE on call. NVDIMMs and persistent memory are both so new and so far an enterprise rabbit hole that developing with them as an individual appears to be virtually impossible.

I suspect the problem is this: The NVDIMM standard actually has several variants and also supports a “Virtual Disk” configuration that trims most of the configuration used for hardware (SMART, vendor IDs, power control, etc). This is the mode I’m using here. I suspect that Windows, despite supporting the NVDIMM standard, does not support NVDIMM virtual disks and I’m going to have to emulate the entire NVDIMM device in firmware. That includes reporting back SMBIOS information, device health, diagnostic information, faking an SPD, etc.

I’ll post the EFI binary tools shortly, minus the RAM disk driver while I get it sorted out. They do work perfectly.

NorthernWing · February 1, 2023, 8:57pm

https://bugzilla.redhat.com/show_bug.cgi?id=1899530

This thread is a pretty good example of my current problems. But, it also gives me an idea:

The outcome of this discussion is that the Windows native drivers support NVDIMM-N while the device emulated by QEMU is NVDIMM-P. They are not compatible.

It’s not clear to me whether NVDIMM-P drivers are available for Windows and work with QEMU’s emulated device. I suggest checking that.

If there are no drivers then it may be possible for QEMU to emulate NVDIMM-N or the Hyper-V NVDIMM interface instead. Unfortunately this is a guest-visible change and may cause compatibility problems with non-Windows guests that lack support for those devices.

… if I’m interpreting that correctly - “the Hyper-V NVDIMM interface” - does Hyper-V support hosting a virtual NVDIMM interface to be consumed by the guest systems? Even if it doesn’t, I bet there’s a hypervisor out there that does. And if it does, I can pretty easily set up a VM and run an ACPI dump of the ACPI tables presented to the guest OS by Hyper-V and then mimic those in my UEFI utility.

~~At this point I’m pretty much writing the entire virtual disk service for a hypervisor.~~

Nope. Close, but not going to work. Hyper-V does support persistent memory devices, but the VHD needs to be located on an existing persistent memory volume supporting DAX/direct byte access. In short, it can pass through a real NVDIMM but won’t emulate one.

BUT. VMWare/ESXi can do exactly what I’m asking, so that’s the next step.

coltmarshmallow · February 19, 2023, 3:55am

This looks awesome and is way above my undrestanding. I would love to help if I could understand half of this

I think what would be an interesting use case for your project is for bare metal infrastructure as code. I would love to be able to write some terraform that would get a device up and running that is powered on and connected to the network (maybe even just connected if using kvm over ethernet). I don’t like setting up devices then running some ansible script to get a new device to accept my stateful set.

I don’t get something and it’s totally fine to just say it’s for the novelty aspect as this is very cool, what I’m wondering if the need for OS boot off the network? Might be a dumb question seeing as I’m not a low level developer but thought typically storage was separated on different volumes and mounted from the network? Thus allowing you to have the OS local

[Devembary 2022] Universal RAM Disk Bootstrapper

1 - Introduction

1.1 - Overview

1.2 - Goals

1.3 - Deliverables

2 - Project Components

2.1 - `blkmirror.efi`

2.1.1 - `blkmirror set`

2.1.2 - `blkmirror get`

2.1.3 - `blkmirror clear`

2.1.5 - `blkmirror transfer`

2.1.6 - `blkmirror ramdisk`

2.1.8 - `blkmirror zero`

2.1.9 - `blkmirror boot`

2.2 - GRUB

2.3 - 56G InfiniBand

3 - Walkthroughs

3.1 - Setting up the TianoCore EFI-Toolkit.

3.5 - Configuring GRUB.

3.6 - Configuring the InfiniBand network.

3.7 - Installing an operating system.

4 - Errata

4.1 - Known Issues

4.2 - Warnings

5 - Dev Log

2022-12-05 | Dev Log

2022-12-12 | Dev Log

2022-12-14 | Dev Log

2022-12-15 | Dev Log

2022-12-20 | Dev Log

2022-12-28 | Dev Log

2022-12-29 | Dev Log

2022-12-30 | Dev Log

2022-02-01 | Dev Log

[Devembary 2022] Universal RAM Disk Bootstrapper

1 - Introduction

1.1 - Overview

1.2 - Goals

1.3 - Deliverables

2 - Project Components

2.1 - blkmirror.efi

2.1.1 - blkmirror set

2.1.2 - blkmirror get

2.1.3 - blkmirror clear

2.1.5 - blkmirror transfer

2.1.6 - blkmirror ramdisk

2.1.8 - blkmirror zero

2.1.9 - blkmirror boot

2.2 - GRUB

2.3 - 56G InfiniBand

3 - Walkthroughs

3.1 - Setting up the TianoCore EFI-Toolkit.

3.5 - Configuring GRUB.

3.6 - Configuring the InfiniBand network.

3.7 - Installing an operating system.

4 - Errata

4.1 - Known Issues

4.2 - Warnings

5 - Dev Log

2022-12-05 | Dev Log

2022-12-12 | Dev Log

2022-12-14 | Dev Log

2022-12-15 | Dev Log

2022-12-20 | Dev Log

2022-12-28 | Dev Log

2022-12-29 | Dev Log

2022-12-30 | Dev Log

2022-02-01 | Dev Log

2.1 - `blkmirror.efi`

2.1.1 - `blkmirror set`

2.1.2 - `blkmirror get`

2.1.3 - `blkmirror clear`

2.1.5 - `blkmirror transfer`

2.1.6 - `blkmirror ramdisk`

2.1.8 - `blkmirror zero`

2.1.9 - `blkmirror boot`