Journey into SBCs (ThatGuyB in ARMland)

Long awaiting, here’s is a smol build guide to Pine64 official RockPro64 case.

Pine64 does not recommend you run all 4 slots, HDD and SSD with their included power cable. So I had to make my own. Behold this monstrosity:

2 buck converters on top of each other. Pink or yellow cables is 12v output, red or bare is 5v. Both have a 19v 7A input.

Picture from the top. There is no standard case for buck converters, so I manufactured recycled my own. It passed the drop test while in use, lmao.

Input Y cable.

The two leads going to the case. It is entering an Orbit Junction Box. Please do not do what I do, this thing is flammable, it is not flame safe.

And here is inside it. 4 Wago connectors, 6 wires for 12v, 4 wires for 5v. The heatshrink is just aesthetic in this picture, I miss ketchup and mustard cable. Actually, k&m is the best thing possible when working with cables that can be mismatched and plugged in the wrong connector.

This is the connection coming from 2x 5.5mm x 2.1mm barrel jack into the sata cable. It is literally a 12v and a 5v rail. No solder, so I ripped a lot of material and did a strong wire twist, then heatshrink which actually does serve its purpose, then black tape on top. The yellow heatshrink tube was too large, so I only used it on top of the black one that went on top of the red one, for the 18awg cable (use tubes that fit your gauge).

Oh, the HDDs.

This is a very tight fit.

Do not buy straight sata cables for the HDDs, get the 90 degrees ones, you will hit the fan blades. Same for the power cables. Get the kind that goes up and down. Besides, those are the best cables, because the wires are snapped into metal teeth, from many people’s testings, those are the kind that are least likely to catch on fire.

The SSD cables do not work if they are 90 degrees. Use straight ones. They are also very tight fit, but the 90 deg cables would not allow the cage to screw in the base.

After mounting it inside, everything is tight.

Cable routing a little.

The cables have barely anywhere to go. You must use zip ties and train them to go in.

Zip ties!

All closed in.

The noctua fan uses the silent adapter. It is inaudible.

This thing has been running for about 1 day and a half without catching fire. It has not been under much load. Runs FreeBSD. Need to configure vlans and routes on boot and rc.conf doesn’t really work for some reason. PEBKAC most likely.

I haven’t even set up the SSD ZFS pool, I only imported the pool I had on the HDDs, but I can barely wait to put this in production, I want to use the Pi 2 for something else.

7 Likes

@HaaStyleCat

3 Likes

Out of curiosity, is that a ASM1166 based adapter and would you mind sharing model/link and what version of FreeBSD you’re running? I can at least verify that rc.conf works as expected on aarch64 :wink:

1 Like

It’s a StarTech 6 port AHCI SATA card. dmesg shows it’s an ASMedia 116x card, so one of these. It says it’s AHCI v1.31. Supposedly has 32 ports in dmesg, with port multiplier not supported.

FreeBSD 13.1-RELEASE arm64.

hostname="rkpr64n"
#ifconfig_DEFAULT="DHCP"
#####
cloned_interfaces="dwc0.3 dwc0.4"
ifconfig_dwc0.3="inet 192.168.3.7 netmask 255.255.255.0"
ifconfig_dwc0.4="inet 192.168.4.7 netmask 255.255.255.0"
ifconfig_up="dwc0"
ifconfig_up="dwc0.3"
ifconfig_up="dwc0.4"
defaultrouter="192.168.4.1"
#####
sshd_enable="YES"
sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
growfs_enable="YES"

When I set them up manually with ifconfig and route, it all works ok. But until then, networking doesn’t work.

It looks like the interfaces get created, but they never come up. So I’m left just setting the ip and turning them up, then adding the default route. More specifically, I need to turn on dwc0 on as well, until I can ping and ssh into the nas.

2 Likes

Startech 6P6G PCI-E SATA card.

3 Likes

Fixed the rc.conf. I later remembered the rc.conf is sequential.

hostname="rkpr64n"
#ifconfig_DEFAULT="DHCP"
#####
vlans_dwc0="3 4"
ifconfig_dwc0="up"
ifconfig_dwc0_3="up"
fconfig_dwc0_4="up"
iifconfig_dwc0_3="inet 192.168.3.7 netmask 255.255.255.0"
ifconfig_dwc0_4="inet 192.168.4.7 netmask 255.255.255.0"
defaultrouter="192.168.4.1"
#####
sshd_enable="YES"
zfs_enable="YES"
sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
growfs_enable="YES"
3 Likes

Place Holder to update as progress is made-

Libre ROC-RK3328-CC “Renegade” https://libre.computer/products/roc-rk3328-cc/

Am considering this as a good alternative for Pi. Would need to either build a OS or convert a supported one from the website. Seems to be able to run Debian (I dunno how you feel about it) but it’s something I am familiar with. While there’s not as much about Raspberry Pi, there are some good posts/articles/“rants”/projects out there.

I was inspired to post here because of the original goal of this post. I also didn’t see much about this board here (this post and or L1 in general). Also, Pi prices are so dumb now a board like this is very appealing. I know it uses a different APU and has other differences, but I don’t see many. I chose this board over there “Le Patato” due to the upgrade from a 100 Mb Fast Ethernet NIC to a 1GB NIC. Figured this would offer more flexibility for network focused tasks (IE OpnSense/PFSense/file server). Also, it uses 4GB DDR4-2133 SDRAM which I believe is faster than Pi and may be better for VM’s and a server like Proxmox or similar uses.

@ThatGuyB - You might have avoided this for a specific reason. IE not compatible with certain OS’s or what not. I’d be happy to test to the best of my technical ability’s (or coaching) if you would like add this to your archive here for others and your own reference.

Let me know what you would like me to play with/attempt with this board. My plans or things I might like to try are making a HA (High Availability for those who don’t know) stack maybe with Kubernetes and some service like pi-hole or another service like Nginx to do a proxy of some kind because I’d like to lear to use that program.

Just a thought. I added a link to the devices web page above. Has all the specs so I wont repeat that here. Also, the wiring diagram is under the “Help” section, which seemed weird for me. I would have thought it would be under “Downloads” at least in my mind. I only mention that because I tried searching the net for one before I found it on their page lol.

Obligatory Pics for viewing- Seems like a quality board from a physical inspection. Traces and soldering all seem solid so build and quality checks seem good.

Also- I know you had been having problems with SD Cards… This board has the option to install a eMMC v 5.x Chip to boot from on the back of the board. This may remove the SD image corruption/flashing issues you were having at one point if I remember right. Also, it is supposed to help with processing speeds via much faster IOPS (not sure thats would be the right term).

Full build I’ll be using for testing (I added the aluminum heat sinks-I like things running cool. Also, there is a way to get the clock speed up to 2Ghz from 1.5 I think stock).

Front of board (Tried to get best images I could with my phone on my work station/desk).

Back of board. (3rd and 4th DDR4 dim for 4GB of Ram)

2 Likes

I like the compatibility with Pi 2 and 3s in the form factor. This can lead to a lot of cool projects. And it’s a good replacement for old guides that were using these. I still have my Pi 2 and 3 and I’m not giving them up until they say goodbye to this world.

I don’t mind Debian. The only reason I haven’t been running it, was because I found weird bugs in apt and apt-get, where the download speed on aarch64 was horrendous. Like, ludicrously bad, we’re talking 5 kbps speeds. On the same network, x86(_64) apt was working flawlessly. This only happened to me on Debian arm, I was forced to run the official Odroid Ubuntu at some point (but that ended up being useless due to the deprecated kernel). Never found the root cause for the apt slowness bug, didn’t care to report it either, I just stopped using it.

There was some other thing with the debian kernel, that also affected void, but I managed to fix it by adding some build tools from github to. I suppose the same fix would have applied for debian when building the zfs dkms.

Keep in mind OPNSense / pfSense only run on x86_64. But I can guide you to set up a Linux or BSD router and file server. I am running FreeBSD on my RockPro64 (you probably already read that, I didn’t look through all the posts you liked, I just assume you did).

I’m not particularly interested in the boards themselves, for me they are just different computers, like servers from HP and Dell are for others. The reason this whole thread exist is to show the steps I took in the setup process and the pain I’ve been through along the way. I should probably do a wiki entry for them, to help people skip all the garbage flood comments. The thread was also a way for me to remember what I tried and what not (and it was helpful to some degree, although finding the information I was looking for was difficult).

Anything you can test on aarch64 will do. If debian runs well for you and you are ok with it, by all means use it. Debian is a good distro.

I would suggest that if you install k8s / k3s / k0s / microk8s, to go with lxc / lxd first on the board. You can make multiple linux containers, enable nested containerization, then install whatever k8s stack you choose inside those linux containers. That way, you can test your HA stack on a single computer at first, without having to buy more of these. Of course, nothing stops you from setting up pi-hole or pi-vpn or whatnot inside lxc directly, but I suppose you are interested in OCI containers.

Maybe I was sounding too annoyed, alarming or overzealous, but I never once had a problem with the SD cards. The only thing even remotely close to this would be with my Toshiba Encore on Windows, when dumb windows just decides that “there is an sd card inserted, but cannot be accessed” (unplugging and plugging back fixes it, but of course, corruption can occur, so I just had to reboot to make it work).

I thankfully never had an SD card go bad on me yet. And my SDs have seen quite some torture. I had issues booting SD cards, mostly because some isos don’t like dd that much, but work just fine with rufus or balena etcher on windows (which is completely insane to me). I also had lots of issues with booting in general, non-related to SD cards, because of the bootloaders (u-boot and to a lesser degree, petitboot, but the later was very smart about many things, just that the linux kernel was not loading something properly, maybe not having support for the boards compiled into it, I assume).

I did talk about netboot in my adventures a lot. It had made experimentation so much easier, than having to plug and unplug SD cards constantly to change back to the old initramfs, kernel and dtb. Although it seemed like SDs and eMMCs are better for booting, apparently PXE has some issues when loading kernel images, that the boards can load from SDs just fine.

The discussion about netboot also kinda devolved into trying to avoid SD card problems. I also talked about the Pi 3 running alpine, which boots everything into RAM, protecting my SD cards. But I since then switched to emmc, because of the aforementioned boot issues that aren’t a thing on local storage. The only exception is the hc4, which only has an sd slot, but it’s currently off (no disks and I couldn’t really boot something with zfs properly on it).

2 Likes

Sorry about that. My misunderstanding. Thought it might be useful to try different SBC’s (Single Board Computers) for different usage scenarios due to Raspberry Pi being so expensive right now.

I must have read it somewhere else. I’ve been away so long I’ll have to review my notes.

Yeah this board compared to some others also offered a USB 3.0 socket for booting from a external drive as well.

Status Update Rant

Heck, I’m just happy I got my Postfix done on my main Proxmox Server to send me Email’s about my ZFS scrub results every month, and fairly sure I have my SmartDisk info set to email me if there is an issue there from the self checks everyday (Fast tests) and weekly Long Tests. I’ll have to figure out how to simulate a smart failure just to see if I get notified.

I think my next big project is going to be finding good software for a NVR (Network Video Recorder) I can run on a VM in the home server that has Motion detection and notification either via email or text. I hope to get PTZ (Pan/Tilt/Zoom) PoE Cameras for the house. I have the green light from the wife for low voltage wiring in the house AND to finally add drops in the rooms that need it vs 1 out of the 5 that should have it but do not…Argh

Realized I’m ranting so I will hide this less relavent info. lol

1 Like

I think even if the RPi was normal priced, that there are better options out there anyway. The RockPro64 and Odroid N2+ are way better computers for software use only. The only advantage the Pi has is the hardware accessories built around it. Especially the Pi CM4, but I believe that to be a bad direction we are heading to.

Even the HomeAssistant officially supported board kit is using the Odroid N2+, you can buy it from them. And the RockPro64 has a lot of support from many linux distros and BSDs. The HC4 is still somewhat new, so support isn’t exactly there yet, but I can barely wait for when it will be able to run FreeBSD.

Not sure about Radxa and Khadas boards, but they are out there too. As long as you can install linux on them, you can just run whatever software you want, k8s, lxd, nfs, you name it. The only downsides to the less supported boards is that you will have a hard time building ZFS on them, which at this point I think it should be included everywhere. Which is why I’m not using the HC4 yet.

PoE? Nice.

1 Like

The major issue with SBCs are that if they’re unsupported you’re going to have issues keeping them up to date since they’re unsupported upstream (both u-boot and OS/distro).

For now at least you should have very low expectations regarding Amlogic support on FreeBSD, no one is actively working it.

2 Likes

Compiled zfs dkms on the odroid n2+. I also had an upgrade from kernel 5.15.82 to 5.15.85. I only built it for .85. Built the uinitrd, changed the symlinks… nope, no boot. Silly me for not checking if the new kernel would even boot on it before I compile it, because last time I tried upgrading to kernel 6, I couldn’t boot.

1 Like

I seriously don’t understand how the bootloader works. I noticed it on u-boot on both the n2+ and the rkpr64. If I boot a clean image, without messing with the kernel, it just works if I switch the symlinks around (pointing to the proper kernel image, initramfs and dtb). If I compile ZFS or the stupid realtek wifi driver, fails to boot instantly.

I am now on a clean 6.0.15 kernel on my n2+. On my rkpr64, I’m still on 5.18.19 (bad security practice to tell people that you are running an old version of a kernel and the exact version), because I can’t get the dkms module to build into linux 6. Supposedly the kernel 6 and 6.1 have the driver for rtl88x2bu, but the usb never shows up if I boot those cleanly.

ZFS is not a must for my n2+, but would have been nice to run zfs FS through iscsi on a zvol. But I can just as well do ext4 and still benefit from zfs in the backend anyway. But the driver module for realtek if a must. I am stuck right now. Maybe at some point, I’ll just work around the wifi thing by using ethernet (and finally be able to run OpenBSD on the rockpro64).

I netbooted to an armbian kernel I know worked, using petitboot (thankfully, it’s a matter of changing a switch position on the n2+). Chrooted into the emmc rootfs, uninstalled zfs and using the previously working uInitrd (the one without zfs module in), I booted just fine on emmc. Just now upgraded to the latest linux 6.1 on my n2+.

I can’t even begin to imagine where I need to look to troubleshoot kernel modules boot-up, as u-boot has no output on the screen on the n2+. Maybe I could use the uart I got for the odroid stuff. I don’t know if the same uart is compatible with the rockpro64, it would be nice if it was and I just needed to jump cables on other pins.

Today I was shocked to see that OpenBSD of all things boots on the HC4 (if you delete petitboot and use u-boot).

I’ll be just formatting the iscsi zvol target as ext4 and install linux containers on it that way for now.

2 Likes

While I haven’t messed around much with compiling Linux kernels are you sure that it gets compiled with the correct dtb/dts(i) file? I wouldn’t also be too surprised if something broke along the way, that seems to occur relatively frequently in Linux ARM-land unfortunately.

USB wifi works poorly at best in general, I would highly recommend you to get a wifi bridge instead if possible. Pretty much any “wifi extender” with an ethernet port supports it or you could have a look if you have a device (router or access point) that supports OpenWrt which also act as one.

UART should be the same but baudrate probably isn’t.

Irregardless of OS/distro, boots doesn’t necessarily mean that you can use it as a daily driver so to say. :wink:

1 Like

If I was compiling the kernel, it might have been easier (just build the drivers instead of relying on dtb), but I’m not compiling it at all, I only get the repo version and build a module. The only thing that gets compiled is the dkms zfs module, which just gets added via modprobe. The compilation is successful, but the boot is not. So we can remove the variable, it’s the same kernel and the same kernel image.

The difference lies in the initramfs / uInitrd (which I’m regenerating from the freshly made initramfs containing the kernel module). Remove the module from the system (/usr/lib/modules/linuxX.Y/extras/zfs) and use the previously generated uInitrd and the system boots as usual.

You mean daily driving me nuts?
:troll:

In all seriousness, what can a SBC that has no OS running on it do?
:man_shrugging:
Correct me if I misunderstand, but did you mean to say that even if the OS boots properly, it doesn’t mean I should rely on the SBC?

1 Like

This shouldn’t be very difficult to debug. Does the kernel module load on the running system with the original initrd without causing trouble?

If you add init=/sysroot/bin/sh or rd.break or break to your boot command-line you should drop into the initramfs shell. There you can step through the init process, manually loading the module, and see where things are going wrong.

2 Likes

I realized I should have modprobed it after I uninstalled it. Interestingly, kernel 6.1 fails to build dkms (something about being unable to include x86/crypto, which is weird, because I’m using aarch64, not x86, probably broken build scripts).

I installed linux 6 again, built dkms and tried to modprobe. Got segfault. At least the zfs boot mystery is solved. I might do the same for the wifi module, which is more important for me.

dmesg
[  625.044650] spl: loading out-of-tree module taints kernel.
[  625.048773] Unable to handle kernel paging request at virtual address 97fff71697ffe56e
[  625.048787] Mem abort info:
[  625.048788]   ESR = 0x0000000096000004
[  625.048790]   EC = 0x25: DABT (current EL), IL = 32 bits
[  625.048794]   SET = 0, FnV = 0
[  625.048796]   EA = 0, S1PTW = 0
[  625.048797]   FSC = 0x04: level 0 translation fault
[  625.048800] Data abort info:
[  625.048801]   ISV = 0, ISS = 0x00000004
[  625.048802]   CM = 0, WnR = 0
[  625.048804] [97fff71697ffe56e] address between user and kernel address ranges
[  625.048808] Internal error: Oops: 96000004 [#1] SMP
[  625.048863] Modules linked in: spl(O+) algif_hash af_alg ip6table_nat ip6table_mangle ip6table_filter ip6_tables iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_filter ip_tables x_tables nfnetlink scsi_transport_iscsi 8021q garp mrp bridge stp llc snd_soc_hdmi_codec dw_hdmi_i2s_audio meson_gxl ir_nec_decoder rc_odroid meson_ir snd_soc_meson_g12a_tohdmitx meson_saradc meson_gxbb_wdt industrialio snd_soc_meson_axg_tdmout meson_vdec(C) snd_soc_meson_g12a_toacodec snd_soc_meson_codec_glue snd_soc_meson_axg_tdmin snd_soc_meson_axg_frddr realtek snd_soc_meson_axg_toddr dwmac_generic mdio_mux_meson_g12a snd_soc_meson_axg_fifo videobuf2_dma_contig v4l2_mem2mem videobuf2_memops meson_dw_hdmi dw_hdmi mdio_mux dwmac_meson8b snd_soc_meson_t9015 videobuf2_v4l2 stmmac_platform meson_rng drm_display_helper videobuf2_common ao_cec_g12a amlogic_thermal stmmac videodev mc pcs_xpcs snd_soc_meson_axg_sound_card phylink snd_soc_meson_card_utils
[  625.048947]  snd_soc_meson_axg_tdm_interface snd_soc_simple_amplifier snd_soc_meson_axg_tdm_formatter snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine snd_pcm uio_pdrv_genirq leds_gpio nvmem_meson_efuse uio cpufreq_dt snd_seq snd_seq_device snd_timer snd soundcore vhost_vsock vmw_vsock_virtio_transport_common vsock vhost_net vhost vhost_iotlb tap hci_vhci bluetooth ecdh_generic rfkill ecc vfio_iommu_type1 vfio uhid uinput userio ppp_generic slhc tun loop cuse fuse xhci_plat_hcd xhci_hcd dwc2 dwc3 usbcore rtc_pcf8563 udc_core ulpi mmc_block crct10dif_ce polyval_ce polyval_generic ghash_ce sha2_ce i2c_meson sha256_arm64 sha1_ce reset_meson_audio_arb axg_audio sclk_div pwm_meson clk_phase panfrost rtc_meson_vrtc meson_drm dwc3_meson_g12a gpu_sched meson_canvas roles phy_meson_g12a_usb3_pcie phy_meson_g12a_usb2 drm_shmem_helper cec meson_gx_mmc rc_core drm_cma_helper usb_common pwm_regulator display_connector drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops fixed drm
[  625.132618]  pwrseq_emmc mmc_core gpio_regulator dm_mirror dm_region_hash dm_log dm_mod btrfs blake2b_generic xor xor_neon raid6_pq libcrc32c aes_neon_bs aes_neon_blk aes_ce_blk aes_ce_cipher
[  625.235501] CPU: 5 PID: 7939 Comm: modprobe Tainted: G         C O       6.0.15_1 #1
[  625.243175] Hardware name: Hardkernel ODROID-N2Plus (DT)
[  625.248437] pstate: 20000005 (nzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  625.255336] pc : mod_sysfs_setup+0x1ac/0x2a0
[  625.259563] lr : mod_sysfs_setup+0x17c/0x2a0
[  625.263789] sp : ffff800010de3a90
[  625.267066] x29: ffff800010de3a90 x28: ffff800010de3c40 x27: ffff80000a2d2c58
[  625.274139] x26: ffff8000019c9e70 x25: ffff800010de3c40 x24: ffff8000019c3e50
[  625.281211] x23: ffff8000019c71e0 x22: ffff8000019c9b58 x21: ffff8000019c9b90
[  625.288284] x20: 0000000000000000 x19: ffff8000019c9b40 x18: 0000000000000000
[  625.295356] x17: ffff0000077b95f0 x16: ffff0000077b95b0 x15: ffff0000077b9570
[  625.302429] x14: ffff0000077b9530 x13: 00646e69625f6461 x12: 657268745f716b73
[  625.309501] x11: ffff0000077b9770 x10: ffff0000077b9730 x9 : ffff800008df24e0
[  625.316574] x8 : 0101010101010101 x7 : 7f7f7f7f7f7f7f7f x6 : 736877645e727872
[  625.323646] x5 : ffff000004f92219 x4 : 0000000000000000 x3 : 0000000000000000
[  625.330719] x2 : ffff8000019c9b58 x1 : ffff8000019c9b90 x0 : 97fff71697ffe4a6
[  625.337792] Call trace:
[  625.340207]  mod_sysfs_setup+0x1ac/0x2a0
[  625.344088]  load_module+0x934/0xa90
[  625.347624]  __do_sys_finit_module+0xac/0x130
[  625.351936]  __arm64_sys_finit_module+0x28/0x34
[  625.356421]  invoke_syscall+0x78/0x100
[  625.360130]  el0_svc_common.constprop.0+0x58/0x1a0
[  625.364874]  do_el0_svc+0x34/0x44
[  625.368151]  el0_svc+0x34/0x140
[  625.371256]  el0t_64_sync_handler+0xf4/0x120
[  625.375483]  el0t_64_sync+0x19c/0x1a0
[  625.379107] Code: 540004a0 f9401700 aa1603e2 aa1503e1 (f9406400) 
[  625.385144] ---[ end trace 0000000000000000 ]---

Yes, there can still be missing drivers and/or instability issues.

…and as for support if the currect dts/dtb isn’t supplied (available) you will most likely see hangs at boot.

2 Likes

Today my RockPro64, or rather FreeBSD install pooped its pants. I did not have swap configured on it, because why should I? It seems like yesterday, my mountd service got killed for lack of RAM, just today I noticed. I haven’t seen any hung nfs mounts (I only had 1, which was working fine, maybe it happened before I remounted it, idk).

Today I installed htop and it wasn’t showing the memory, it said 0k, but it was however yellow, meaning a lot of stuff was cached in RAM, likely ZFS. vmstat reported basically all used, very few pages free. zfs destroy of a snapshot - hung. zfs list - hung. service mountd restart - hung. kill -9 the pid of zfs list - hung. reboot - hung.

I left it for more than 15 minutes, then I gave up on it, unplugged it. Plugged an HDMI into the board, to try and troubleshoot - no output on bootup. Also no ethernet coming up. If I have to unscrew this box to fix it (to access the emmc), then I will be really upset, that thing is packed tight.

It’s not even funny, these days I was thinking of moving my main scratch disk from the pi 2 over to the rockpro64 nas. Well, the HDD was formatted ext4, not sure if freebsd could read it, but I could just move the data and format the disk (the data on it is not as important). I’m sure glad I didn’t.

All these failed experiments are making me want to go back to intel so badly, where I know if there’s a problem, the thing will still post just fine and I can easily just plug a usb and boot from it to troubleshoot. Heck, the Odroid N2+ is smarter than this, it has an outside boot switch, so I can move from emmc uboot to spi flash petitboot and netboot it in case of boot trouble, but this basically guarantees I can’t retire the pi 2 just yet (it is my tftp and nfs server for netboot).

The data should probably be fine, but I’m very disappointed right now. i might give ARM SBCs another shot when they get to 16GB variants, like the radxa rock 5B. I haven’t used radxa, I don’t know what bootloader it uses, but if it’s not uboot or petitboot, I’m not even going to try it. Maybe I’ll wait for QuartzPro64 with 16GB of RAM to replace the 4GB RockPro64 in the NAS.

The zimaboard looks appealing right now, I could just unplug the rkpr64 and plug this in and have a pci-e sata card fed into it. It only has 8GB of RAM, but should still be manageable.

I’m not planning any electronics purchases yet, I want to think deeper before I hop in and buy yet more stuff. Worse case, I can sell my TR build and make something more power efficient that I could use as a multi-purpose box (homelab and windows vm). Or I could build something smaller and even lower powered (I kind of need a PC, I keep getting errors in dmesg that the vgpu memory gets full on my pi 4, and the video playback becomes laggy) and do my homelab on the PC that would be powered on 24/7 anyway. I’m not even sure what I would get, probably a low powered i3, to pair with my rx6400 which does nothing in my tr box (other than waste space and pcie lanes).

These days, if the nas doesn’t boot, I might flash an armbian on a sd to troubleshoot it and pray that it boots from sd directly (I don’t remember the pin order I used, I think I left it default, IIRC sd is first boot after spi and the spi is wiped clean).

I’m pausing this rant, I can’t even.

1 Like

htop is known can be a bit wonky reporting memory usage using zfs, consider using top instead as far as memory reporting goes.

Display output works in general however it sounds like something has happened to the storage device. If you’re using UFS on rootfs it may be stuck waiting on input early on because the filesystem is dirty. Unless you’ve set boot_serial=“NO” in /boot/loader.conf output will be on serial port. I don’t know what services you ran on your box but ZFS likes memory and you may run into memory exhaustion if you have applications that uses /tmp (which uses tmpfs) extensively. Also selfhosting (compiling ports and possibly installing large ports) with only 4G of RAM will give you issues especially without any swap but this is not limited to aarch64 though when using ZFS. You can of course tune both ZFS and tmpfs which may be needed in your case or adding a small swap file/partition will simply solve your issue.

It would’ve been interesting to see the log what made it hang but I doubt it’ll find it in /var/log/message .

That being said, I’ve been running RockPro64 boxes for months without any issues at all (13.1-RELEASE).

You probably want to look into vfs.zfs.arc.max and vfs.zfs.arc.min and set appropriate depending on your workload.

1 Like