Hello there,
this is a short* dump of my adventures with trying to boot windows10 from an iscsi target.
I’ll ignore the iscsi target setup since that is the smallest problem.
The PXE ,
i played around with iPXE and Flexboot (basically the same) on Mellanox network cards.
Those can only do legacy boot, so FFFFFF if what you want to boot is installed in U-EFi mode.
Yes, ipxe is available as efi version, but it behaved flaky for me and didn’t work with the Mellanox-Cards.
Since Connect-X4, there is a EFI capable PXE rom on the cards, but i have Cx3 and 2.
NOTE, ipxe can only use 3 nic ports, so if you have more, FFF good luck figuring out which is which.
The DHCP Server telling iPXE what to do took me like two years of intermittent work to get it working on freenas, but i won’t write about that here right now. If you have questions, ask them!
Hint: read everything before attempting anything!
Install windows to iscsi target
All Windows Installers seem to be iBFT capable.
0: Set proper boot order in BIOS.
1: Boot into PXE: ipxe / flexboot / what ever.
with just an install-USB-stick
Proper boot order: First the pxe, second the Installer.
2: sanhook the Target that you want to install to.
sanboot is also ok since that hooks and then fails to boot, if the target is empty.
3: exit iPXE
4: the installer should now boot, if not, check the boot-order.
5: proceed through the installer
6: add relevant drivers, for example the mellanox nic drivers.
7: The previously hooked target should be available as install target since ipxe left a iBFT entry. If it isn’t, well there was a version with bugged iBFT, or try the later mentioned keep-san.
8: install windows
Notes:
You need to hook, and install over the network-card that is later used to boot because of specific driver un-binding that the installer will do for you.
My Asus Maximus ix Apex, failed to boot the windows10 installer after exiting iPXE/Flexboot even though the correct boot order was set. That is a BUG and could be due to other settings, or the fact that it is a modded BIOS to run Coffe-Lake on Z270.
I suggest you try the WinPE route, or you manually prepare and transfer an image, described in the other spoiler.
with winPE + install USB
Make sure winPE has the correct Nic-Drivers, for example for the mellanox nics.
2: sanhook the install-target.
3: sanboot or just boot the WinPE.
4: Start the installer “setup.exe” from within winPe. Idealy this is build into winPE or on an attached USBstick or where ever.
5: proceed the installation.
Has it worked? I had “can not install to this disk” error, but i was to dumb to add the mellanox drivers to winPE.
NOTE, ipxe / flexboot can only attach to two iscsi targets simultaneously. A third one will fail to attach.
Error was “no space”. Maybe the 8GB of ram weren’t enough.
done.
NOTE, Flexboot has a keep-san option that iPXE also has.
It shouldn’t matter but Mellanox release notes state different things so you might want to set keep-san 1 with flexboot just in case.
Prepare existing W10 for iscsi boot
Any way described here under workarounds should work.
In short, what you need to do is to tell the NDIS Lightweight Filter Driver not to bind the nic you are booting from.
You do that by disabling the binding to the Nic.
The link above contains two ways, the bindview UI application, and a Nvspbind cmd too.
There are Powershell-cmd-lets that allow you to do the same but i forgot where i found those.
And in the end, all of this should be just registry-entrys, buti didn’t dive into those.
Soem kees are mentioned in here.
https://support.microsoft.com/en-us/help/2507616/0x0000007b-stop-error-after-replacing-or-switching-to-an-alternate-isc
If you want to use BindView, well i needed about 25GB of space with visual studio and many different libs and sdks just to compile it.
Here is my compiled version, It should work. Haven’t had problems with it jet. AMD_64.
https://github.com/Ragebone/MS_bindview/releases/tag/1.0
shortened bindview steps from M$
- open bindview with admin-privileges
- In “Show bindings for” – Select “All Services”
- You will see two “WFP Lightweight filter … MAC …” node. Expand it to see its binding paths. There are two of those, ipv4/6.
- Right-Click on “Binding Path #” for the iSCSI boot NIC/NICs and select “Disable”. If the Disable option is not available, it means the binding is already disabled. You also see if it is disabled.
that’s it!
Transfer your Windows to the iscsi target.
Yay, now you should be able to successfully boot windows form an iscsi target.
If you get a inaccassable_boot_devie BSOD, set the Gateway of the boot nic to 0.0.0.0, and if that doesn’t work, well, keep on reading.
Pain with Hardware-Changes:
The moment you significantly change the hardware in a way that changes the IDs of the Boot-Nic, Windows will detect it as a new device, and bind the NDIS LWF (WFP) Driver to it.
That will then cause a “inaccassable boot device” blue screen.
The only way, i currently know of, how to recover from that is to transfer the target to a “real disk”, boot that disk to get new bindings.
Then disable the boot-nic bindings and transfer it back to the iscsi-target.
Windows 10 seems to at least be fault tolerant that you can replace the Nic in case of a fault.
With other versions, you will need to apply those M$ patches to even be able to change the nic with an identical one.
Funny observation. My two W10 tests couldn’t shut-down properly, but restarts work fine, so do applying-updates.
Upgrades on the other hand, have the extra potential of breaking.
Haven’t done one jet.
Why did i do this?
Well. friends and i do overclocking and i got really annoyed by the time that we waste every sessions, shuffeling ssds, installing, reinstalling, updating and recovering windows-es.
So we dreamt of snapshots and “all disks in one place”.
My choice because it is already in place and running, freenas with ZFS ZVols, snapshots and clones.
The only thing that i haven’t found jet:
One image that “multiple” boot “ReadOnly”, and each gets a few gigabytes of separate space for differences and tCopyOnWrite handles changes of the boot image.
M$ sever can do that if you need it.
good luck with all of that, if you know a way to disable the NDIS LWF driver completely or just for a family of nics so that i can boot any machine with one base-image, please let me know!
.