thanks for your great videos regarding FreeNAS and server grade hardware (inkl. IPMI)! I am currently in the process of installing a new FreeNAS system based upon the ASRock Rack E3C226D2I mainboard. Now I have a few questions:
- I have found that the BMC of this board has a network bond device bond0 consisting of eth0 and eth1 by default. This is a bit confusing to me as there is only one physical Realtek device, right, but there are even two MAC addresses. So what exactly are eth0 and eth1 then? Above that I can only assign an IPv6 address to that BMC if I deactivate the bond0 device and configure the "two" (?) network interfaces eth0 and eth1 separately.
- What exactly is the purpose of the interfaces tab on FreeNAS? The network devices already got an IPv4 address from my DHCP server. Is this only needed if you want to add further configuration options to that device?
- I want to limit the fan speeds on the motherboard and make them dependent on the system's temperature. But there is no documentation regarding details of smart fan configuration in the mainboard's manual (seems these options came with a firmware update and the manual is not up to date). Do you have a hint on that?
Thanks in advance and best regards
P.s.: Sorry for my broken english, it's not my mother's tongue.
This is a bit confusing to me as there is only one physical Realtek device, right, but there are even two MAC addresses.
Wrong, on the board you stated. There is one dedicated port for IPMI and two Gigabit LAN ports for the host. The IPMI can not be accessed from within an OS and needs to be configured from the BIOS.
bond0 is an indication of two physical networks being teamed. (Or LACP) eth0 and eth1 are the actual physical network ports.
What exactly is the purpose of the interfaces tab on FreeNAS?
So you configure network interfaces. If you are using DHCP (which is a bit odd for a file server IMO) then the only reason you'd ever go there is to configure any additional you may wish to add. Primarily it offers control of anything network based, so setting up static IP, different DNS, different gateways etc... As mentioned before with the bond0 device, you can play with bonding settings. (I only really advice this if your switch can support such functionality)
I want to limit the fan speeds on the motherboard and make them dependent on the system's temperature.
This functionality differs from board to board (as I do not own this board, I am stabbing in the dark). Usually on server grade boards, its all done automatically with predefined settings within the BMC. This does require that you are using PWM fans (unless the board supports Voltage regulation as well)
Any software suites that enable the user to tinker, are most probably Windows or Linux based. Not FreeBSD which FreeNAS is based upon.
Regarding the bond0 device I am pretty sure that I am NOT wrong. I've been connected to the mainboards's BMC via the separate Realtek nic and get the bonding option there (not in the BIOS). Both eth0 and eth1 have Asrock MAC addresses (BC:5F:F4:X:X:X), not INTEL (D0:50:99:X:X:X)! If you take a look at the board you will find 4 MAC address labels in total (two sticking on top of the single Realtek nic, two on the two INTEL nics). So I am still confused.
Regarding the fan speed settings I think you are right. I have plugged the case fans (3-pin) into the mainboard's front and rear fan connector (4-pin). That leads to fans running full speed all the time. So what can I do? Changing the fans or make them run at 7V with additional hardware I think.
RE the bond thing, the IPMI thing may take a bit to get your head around. There really are (should be?) three physical network interfaces. Two accessible to the OS and one for the IPMI. You *can* bond the IPMI interface to either or both (!) of the intel physical adapters. This neat because the hardware injects IPMI packets and it means you can get away with only one network interface. It's really handy when you setup vlans because you can have one physical cable (one switch port) but two vlans--one for out of band management (it's slightly less secure, but not as bad as not out-of-band management) and one for regular traffic. Or not, you can have two Mac address on one physical wire. However, with freebsd and the intel drivers, you're best off to just setup the IPMI to not bond and run a physical wire for the realtek adapter to your switch and do it that way. Some cheaper hardware does not have the PHY interface even though it's got the realtek chip. in that case you have no choice but to bond in the IPMI to intel adapter 1 or 0. It's not truly a bond-- it just shares the PHY interface to the wire. I would advise if you do this that you use one intel adapter to bond with the ipmi and the other one not bonded. In my ipmi it says 'either' in the option and when you do that it bonds the ipmi to both phy.. but rarely would you really need that.
tldr: if possible, disable bonding in the ipmi and use a second physical cable for the ipmi interface. less chance of crazy. And the intel bonded stuff under freebsd, if you want to use both interfaces, is almost guaranteed to work better.
the network gui in freenas is also kind of important to setup for security contexts sometimes. e.g it's okay to talk to the lan, but not other subnets.. that kind of thing
Thanks for your explanation, Wendel! So I will end up using three wires with my new NAS, a BMC wire on the management VLAN and a LACP trunk on the data VLAN.
Ok, now that I disabled networking bonding (menu Configuration / Network Bond) I can choose from eth0 or eth1 to be the LAN interface (menu Configuration Network) for the IPMI. But the Realtek interface does not seem to be really separated from the Intel Nics because I only get access to the IPMI as long as there is an additional link to whatever device I chose to be the LAN interface. If I choose "eth0" I have to have a wire on the Realtek interface plus the eth0 Intel Nic. Using just the Realtek Nic won't do it.
Sorry for spamming the thread but I think I have found an explanation on the FreeNAS forums:
"I just got a reply from ASRock support. Very fast reply. There are some issues with the IPMI sharing the intel ports. The solution is to go into the IPMO (Megarac SP). Select Configure->Network->Network Bond, enable it and choose eth1 (??!!). This will make the IPMI leave the intel NICs alone and only work on the dedicated network interface. The changes to the settings makes no sense at all but it solves the problem."
So basically the nomenclature of the network options make no sense at all. Thanks, ASRock, for that lousy documentation.
Thank you so much for posting the above message. I recently picked up an E2C226D2I board second-hand - everything seemed to work, except for IPMI networking. Enabling the Network Bond as you described made everything work perfectly. Thanks!