I am new to the world of custom kernels on unsupported architectures. I am able to build the kernel for the processor, but I am unsure of how to integrate the OS (RHEL 7) into it.
I am lacking in the knowledge of how a kernel is integrated into an OS, or vice versa.
-What I think I know: I can build the kernel for the device.
-What I am sure I don’t know: How a certain kernel gets married to an OS and will, at the very least, boot.
So how does the process play out? Do the specific OS and it’s kernel interconnect and therefore the whole process needs to take one into account of the other? Or can I build one and arrange it to marry another?
It depends on what you mean by unsupported architecture.
If you mean it officially supports up to say intel 5th gen cpus, and it works fine on intel 6th gen chips, then that is fine.
If you mean it supports intel/amd x86 cpus, and you want to run it on say an arm cpu, that will not work. The reason is that most software packages that are complied( where the software is written in a compiled language like C or C++), have different versions for different CPU types. So you would have to recompile a very large amount of packages to get it to work. In this case, try a different distro that has support for your CPU type. These are like AMD64, i386, flavors of ARM, flavors of power, etc.
Part of this depends on the bootloader. Grub does things differently then systemd-boot does things differently then u-boot.
Generally, most linux software is very tolerant of different kernel versions. The biggest thing that is not is any software that has added in kernel module(s), such as the Nvidia drivers, vmware workstation, virtualbox guest additions, wireguard(pre kernel 5.6), and a number of obscure drivers.
The integration should be handled by the distro as long as you use the right tools. If you are using Grub, you probably need to use the grub tools.
If you are using an EFI aware OS, then you many need to use mkinicpio or mkcpio. If you look at you distro’s documentation on compiling a custom kernel, it will answer what you need. RedHat has excellent documentation, but you need to create an account as they have moved all of the good stuff behind their logon wall.