Dd or doo doo? [seemingly solved]

I’ve just put together a new Ryzen 5950X box. It would appear to be working (yay! But I need to stress it to be sure.) I’m attempting to install an operating system. It has USB drives, no CD/DVD drive. I’ve created several supposedly bootable USB sticks for Windows 11, Windows 10, Debian 11, Manjaro, Kubuntu and Fedora. Only the Kubuntu and Manjaro sticks will boot. I created them all via dd on Debian 10 running on a 2011 MacBook Pro.

I have now created a bootable Windows 11 USB drive using Microsoft’s MediaCreationToolW11.exe on a laptop running Windows 8.1. It’s currently installing.

I did some googling. The inability to create reliably bootable USB sticks via dd is not unique to me.

I did e.g. sudo dd if=<path-to-iso> of=/dev/sdb1 status=progress conv=fsync && sudo sync /dev/sdb1

I tried /dev/sdb and /dev/sdb1 with the same results.

I compared the Windows 11 ISOs I downloaded on Windows and Linux. They’re identical.

Anyone know what’s going on and how to get a 100% reliably bootable USB on Linux? It seems it’s always worked for me in the past, although I’ve mostly been installing into VMs on other already existing machines (Mac, Linux).

Thx.

PS: Why do many Linux distros? I’m a Debian aficionado but I believe it’s not going to play nice with the RX 6600 in this new box. I will likely need a different distro so I have to test drive them first (Manjaro is a known quantity, as is Ubuntu although I haven’t previously installed Kubuntu. Seems that Fedora might be closest to Debian 11.)

There is a difference between /dev/sdb and /dev/sdb1. The former is the device (where you want it) the latter is a partition and would only make sense if you’re making a multi-OS bootable drive. So, here it is:
sudo su #become root
dd if=/path/to/image of=/dev/sdb bs=4k status=progress
sync /dev/sdb
exit #exit root

Note that sudo is not a great tool for admin tasks. Some tasks require the root UID outright, sudo does not supply that, it uses the user UID instead and this might make commands fail, although they appear successful. Sometimes you need to be root with full root permissions, not a user with a (incomplete) subset of allowed root permissions. IMO.

1 Like

Thanks. I did actually try e.g. /dev/sdb and /dev/sdb1. There was no difference as far as I could tell.

I’ll try it as root, although I don’t believe I did that in the past. If there’s a subtle timing issue then it might have just happened to work previously and not now.

i use balena etcher for my boot media

i also needed to install debian from one of these (on a 5800X):
https://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/

I don’t remember why exactly, but I’m pretty sure it was a UEFI thing.

1 Like

Actually, creating a Windows 10/11 boot drive from Linux is a long and complicated affair. Doing it through DD has quite a few Gotchas, but if you do it through something like Ventoy you should be able to get it running.

1 Like

I have a firm belief, possibly wrongly held, that I should be able to do everything from Linux with the tools on hand, or at least in a repo. But I’ll give Balena, or Ventoy as pointed out by @wertigon, if I can’t get the next Windows install going (due today, hopefully). Thanks.

Holy Mother of Bob! I created a Debian 11 USB stick from root and it’s operational today! This is the same USB stick I created in the same way with the same Debian bits but with sudo instead of actually logging in as root (well, superuser).

Maybe sudo isn’t all-powerful as I’d been led to believe? Next you’ll be telling me that “sudo make me a sandwich” (per the famous XKCD comic) isn’t infallible?!

Thanks for that suggestion.

1 Like

Sudo has only ever the rights you give it in the /etc/sudoers-file aka via the visudo-command!

The following should usually be enough if you did not mess with the permissions.
sudo dd if=/path/to/image of=/path/to/blockdevice bs=1M status=progress oflag=sync

2 Likes

That’s pretty much what I did except I didn’t bother setting buffer size. I’m still not 100% convinced since some USB sticks I created via sudo worked, and some didn’t.

Windows (8.1, laptop) has complained when I’ve inserted some of the USB sticks that there were errors that needed correcting but they all seemed readable whether I let it correct them or not. (I didn’t pay close attention to which were OK and which weren’t, just swore liberally and went back to the Linux machine and tried another tack).

I do remain convinced however that I should be able to do what I need to do with the tools that are available on Linux.

Thanks!

1 Like

also… sudoers sudo group has the same permissions - ALL - as root, and I’m a paid-up sudoer (well, in the context of FOSS)

#
# This file MUST be edited with the 'visudo' command as root.
#
# Please consider adding local content in /etc/sudoers.d/ instead of
# directly modifying this file.
#
# See the man page for details on how to write a sudoers file.
#
Defaults        env_reset
Defaults        mail_badpass
Defaults        secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

# Host alias specification

# User alias specification

# Cmnd alias specification

# User privilege specification
root    ALL=(ALL:ALL) ALL

# Allow members of group sudo to execute any command
%sudo   ALL=(ALL:ALL) ALL

# See sudoers(5) for more information on "#include" directives:

#includedir /etc/sudoers.d

What does it output when you run groups under your user?

For easily making a bootable flash drive from an ISO, here are good options.
Fedora’s Media Writer Windows/Linux (My personal favorite, though it’s not well known)
Etcher Windows/Linux(AppImage)
Rufus Windows

I also HIGHLY recommend you look into Ventoy, which allows you to drop multiple ISOs onto the drive whenever you want, boot into ventoy’s loader screen and select whatever you want to run. The documentation is a bit wonky, but it’s well worth doing. You can also still use the drive to store files, which is really nice if you need to store drivers or other setup files.

1 Like

<me> cdrom floppy sudo audio dip video plugdev netdev bluetooth lpadmin scanner libvirt wireshark docshare devs

1 Like

Alright, then I have no idea what went wrong. Since you are in the sudo group you should have the same permissions as root when you prepend sudo to your command. Have you done integrity checks to the isos you downloaded. Maybe you have a faulty USB stick, but other than that I am not seeing that you did anything wrong.

1 Like

Thanks for the suggestions. I read the article linked by @wertigon above and it does sound like it’s a good candidate. I’m aware of Balena Etcher but haven’t tried it as yet (since until recently I haven’t had any issues - Windows and this new machine have uncovered them)

Two for two!

Gentlepersons, I think we have a winner. I just re-wrote the non-booting Fedora stick (using the same Fedora ISO as before) using the suggestions from @Dutch_Master and @anon34403656, basically running the original command line from root instead of my account via sudo, and in this case speeding things up a bit with bumping up buffer size.

I did the same with the previously non-booting Debian 11 stick (minus the buffer size parameter) and that boots also.

Maybe I’ll try tails next… (er, no. Maybe it’s time for Arch though?)

Thanks again for the suggestions and advice.

1 Like

Thanks. Me either. As I said, this has seemingly worked flawlessly in the past (emphasis on seemingly). The ISOs are good - checksum’d them, also downloaded on Linux and Windows and diff’d. Tried different USB sticks and they’re brand new (not entirely proof of goodness I know). I did try formatting at least once (the full fat format), and I have to say that Windows does appear to prefer formats that it created rather than (ex)FAT partitions created under Linux. That’s something else I might want to look into, but as of this moment, it appears I’m off to the races again with ISO images transferred to USB.

I really ought to try cutting another Windows USB stick, I know the exact candidate: I don’t think the Windows 10 USB stick I created under Linux yesterday was working!

Bah. Windows 10 did not yield to the incantation. So it looks like Ventoy or Etcher or Rufus or the Fedora solution. Except … I have a bootable Windows 11 USB and I’m going to be installing that. But, also … what the hell. I need to try out these other solutions that have been kindly suggested, and I also need to try running Windows 10. Can you believe I’ve survived this long without running Windows 10 anywhere? (I’ve been in Mac and Linux land, and Windows 8.1 has been fine once you accept that it has a Metro interface, but you can only be driven nuts by software so much before you become inured to it (and then you jump ship to Linux, right?))

Holy Megadodos! Windows 10 USB created with Ventoy is bootable.

But wait! There’s more…

I went back and re-created a Windows 10 USB using the - by now - tried and trusted method of dd from root above and that works too!

So it seems that there’s some flakiness with using dd as long as you’re not doing it from a root account. Also, in my haste I may have been guilty of pulling the USB stick before all the bits were finished writing, even after doing a sync. (I also may not have been, but I wouldn’t swear to it on a stack of Bibles, even if I believed their contents.) On these most recent attempts I’ve waited until the LED on the stick stops blinking, or just waited an eon (by CPU standards) before unplugging the USB stick.

As they say, “your mileage may vary”…

+1 for Ventoy

The whole approach of getting a USB drive to emulate a CD is using obsolescent technology. USB drives now start at 32 GB for a few dollars, and isos are only 3 or 4 GB, so it’s often handy to have several isos on one stick. Write the iso using your favourite file manager, drag’n’drop.

1 Like