Nutral's HomeLab blog

xnd · November 16, 2023, 6:57pm

Update: I changed SATA SSD to m.2 (Samsung 980 Pro 1TB with heatsink)

~8 W has system with applied powertop --auto-tune
~2 W after applied script to enable ASPM L1 state for PCI devices

Basically no change copared to SATA SSD I have used previously.

my powertop report: JS Bin - Collaborative JavaScript Debugging

lspci output, smartctl output:

BEFORE:

dietpi@DietPi:~$ sudo lspci -vv | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM )'
00:01.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 (rev 05) (prog-if 00 [Normal decode])
		LnkCap:	Port #2, Speed 32GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <4us, L1 <16us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
00:1c.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #1 (rev 11) (prog-if 00 [Normal decode])
		LnkCap:	Port #1, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk-
00:1c.2 PCI bridge: Intel Corporation Device 7aba (rev 11) (prog-if 00 [Normal decode])
		LnkCap:	Port #3, Speed 8GT/s, Width x1, ASPM L1, Exit Latency L1 <64us
		LnkCtl:	ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO (prog-if 02 [NVM Express])
		LnkCap:	Port #0, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
		LnkCtl:	ASPM Disabled; RCB 64 bytes, Disabled- CommClk+

AFTER:

dietpi@DietPi:~$ sudo lspci -vv | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM )'
00:01.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 (rev 05) (prog-if 00 [Normal decode])
		LnkCap:	Port #2, Speed 32GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <4us, L1 <16us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
00:1c.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #1 (rev 11) (prog-if 00 [Normal decode])
		LnkCap:	Port #1, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk-
00:1c.2 PCI bridge: Intel Corporation Device 7aba (rev 11) (prog-if 00 [Normal decode])
		LnkCap:	Port #3, Speed 8GT/s, Width x1, ASPM L1, Exit Latency L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO (prog-if 02 [NVM Express])
		LnkCap:	Port #0, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+

smartctl output:

dietpi@pc:~$ sudo smartctl --all /dev/disk/by-partlabel/root
smartctl 7.3 2022-02-28 r5338 [x86_64-linux-6.1.0-13-amd64] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       Samsung SSD 980 PRO with Heatsink 1TB
Serial Number:                      S6WSNS0T212452K
Firmware Version:                   5B2QGXA7
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 1,000,204,886,016 [1.00 TB]
Unallocated NVM Capacity:           0
Controller ID:                      6
NVMe Version:                       1.3
Number of Namespaces:               1
Namespace 1 Size/Capacity:          1,000,204,886,016 [1.00 TB]
Namespace 1 Utilization:            260,609,507,328 [260 GB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            002538 b221405179
Local Time is:                      Thu Nov 16 19:03:46 2023 GMT
Firmware Updates (0x16):            3 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x0057):     Comp Wr_Unc DS_Mngmt Sav/Sel_Feat Timestmp
Log Page Attributes (0x0f):         S/H_per_NS Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg
Maximum Data Transfer Size:         128 Pages
Warning  Comp. Temp. Threshold:     82 Celsius
Critical Comp. Temp. Threshold:     85 Celsius

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     8.49W       -        -    0  0  0  0        0       0
 1 +     4.48W       -        -    1  1  1  1        0     200
 2 +     3.18W       -        -    2  2  2  2        0    1000
 3 -   0.0400W       -        -    3  3  3  3     2000    1200
 4 -   0.0050W       -        -    4  4  4  4      500    9500

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        33 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    2,152,014 [1.10 TB]
Data Units Written:                 2,615,497 [1.33 TB]
Host Read Commands:                 37,854,537
Host Write Commands:                39,435,101
Controller Busy Time:               25
Power Cycles:                       43
Power On Hours:                     7
Unsafe Shutdowns:                   26
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               33 Celsius
Temperature Sensor 2:               33 Celsius

Error Information (NVMe Log 0x01, 16 of 64 entries)
No Errors Logged

dietpi-benchmark:

┌─────────────────────────────────────────────────┤ DietPi-Benchmark ├─────────────────────────────────────────────────┐
│                                                                                                                      │
│ Benchmarks completed:                                                                                                │
│  - CPU performance : Duration = 1.00 seconds (lower is faster)                                                       │
│  - CPU temps       : Idle = 52 °C | Full load = 63 °C                                                                │
│  - RootFS I/O      : Write = 1099 MiB/s | Read = 2098 MiB/s                                                          │
│  - RAM I/O         : Write = 3415 MiB/s | Read = 6657 MiB/s                                                          │
│                                                                                                                      │
│ Compare these results online with other users, using the link below:                                                 │
│  - https://dietpi.com/survey#benchmark                                                                               │
│                                                                                                                      │
│                                                        <Ok>                                                          │
│                                                                                                                      │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

For comparison:
Odroid H3: (Intel N5105 with m.2 Samsung 980 Pro 1TB and same Crucial DDR4 3200 but 16GB) 

┌─────────────────────────────────────────────────┤ DietPi-Benchmark ├─────────────────────────────────────────────────┐
│                                                                                                                      │
│ Benchmarks completed:                                                                                                │
│  - CPU performance : Duration = 2.37 seconds (lower is faster)                                                       │
│  - CPU temps       : Idle = 31 °C | Full load = 38 °C                                                                │
│  - RootFS I/O      : Write = 558 MiB/s | Read = 1573 MiB/s                                                           │
│  - RAM I/O         : Write = 1706 MiB/s | Read = 4391 MiB/s                                                          │
│                                                                                                                      │
│ Compare these results online with other users, using the link below:                                                 │
│  - https://dietpi.com/survey#benchmark                                                                               │
│                                                                                                                      │
│                                                        <Ok>                                                          │
│                                                                                                                      │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

nutral · November 16, 2023, 8:57pm

I get 4w from the wall with a 19v adapter and the system idle. But once I have any application running on the M2 980 it jumps up. But I’ll have to test if that is the same with data ssds.

nutral · November 16, 2023, 9:00pm

Thanks! I agree that this stuff is hard to setup for beginners. Lots of guides are based on a higher skill level. Or gloss over things.

I’m thinking of making a guide with a modified version of my setup while also going through the steps of setting up a Linux server in a beginner friendly setup.

Zonte · November 17, 2023, 2:29am

Thank you for your reply… If you do make a guide I will definitely use it. Im still collecting hardware but will need it eventually.

xnd · November 23, 2023, 5:55pm

Update: I have changed from Debian 12 stable to testing and also linux kernel from 6.1.0-13-amd64 to 6.5.0-10-amd64.

First I have upgraded kernel and after restart everything ok, like before. No significant changes noticed.

Then I updgraded from stable to testing (using this tutorial).

the reason was to have newer version of packages, especially powertop 2.15 because 2.14 was quite old for Intel 13th gen and doesn’t show C1, C2, … C10 states correctly - insted it shows only 3 states: C1_ACPI, C2_ACPI, C3_ACPI
- see github issue: What does _ACPI suffix mean? · Issue #140 · fenrus75/powertop · GitHub

I did measurements again and idle power consumption is still good - it seems it’s like before:

H610t_idle_02

idle, headless (only LAN connected), fanless, 1 active SSH connection, measured on DC side (power supply is not included)

powertop outputs (and other details):

ezgif-1-f023e4e909

or in text form:

PowerTOP 2.15     Overview   Idle stats   Frequency stats   Device stats   Tunables   WakeUp                            

Summary: 38.5 wakeups/second,  0.0 GPU ops/seconds, 0.0 VFS ops/sec and 0.3% CPU use

               Usage       Events/s    Category       Description
           139.1 µs/s      10.4        Timer          tick_sched_timer
            25.7 µs/s       6.9        kWork          flush_to_ldisc
           132.9 µs/s       4.3        Process        [PID 101] [hwrng]
            83.9 µs/s       3.1        Interrupt      [3] net_rx(softirq)
             3.2 µs/s       2.2        Timer          napi_watchdog
            31.4 µs/s       2.1        Timer          watchdog_timer_fn
            27.2 µs/s       2.0        Process        [PID 76] [kcompactd0]
            38.4 µs/s       1.6        Process        [PID 16] [rcu_preempt]
            84.9 µs/s       1.0        kWork          pci_pme_list_scan
             5.6 µs/s       0.9        kWork          psi_avgs_work
             7.8 µs/s       0.9        kWork          vmstat_shepherd
           407.9 µs/s       0.7        Process        [PID 1338] powertop -t 45
            15.5 µs/s       0.8        Timer          hrtimer_wakeup
            27.1 µs/s       0.4        kWork          handle_update
            86.8 µs/s       0.3        Process        [PID 1336] sudo powertop -t 45
           652.3 µs/s      0.07        Process        [PID 78] [khugepaged]
             3.9 µs/s      0.26        kWork          flush_memcg_stats_dwork
           406.1 µs/s      0.00        Process        [PID 48] [kworker/3:0]
             1.8 µs/s      0.13        kWork          kfree_rcu_monitor
            84.9 µs/s      0.07        Process        [PID 999] /usr/sbin/dropbear -EF -p 22 -W 65536 -2 8
             0.2 µs/s      0.09        kWork          output_poll_execute
            15.0 µs/s      0.07        kWork          i915_hpd_poll_init_work
             0.2 µs/s      0.07        kWork          lru_add_drain_per_cpu
           115.9 µs/s      0.00        Interrupt      [126] eth0
             0.0 µs/s      0.04        kWork          pm_runtime_work
           103.6 µs/s      0.00        Interrupt      [7] sched(softirq)
             0.5 µs/s      0.02        kWork          neigh_managed_work
             0.2 µs/s      0.02        kWork          neigh_periodic_work
             0.2 µs/s      0.02        kWork          crng_reseed
            45.6 µs/s      0.00        Timer          process_timeout
            45.0 µs/s      0.00        Timer          delayed_work_timer_fn
            42.3 µs/s      0.00        Interrupt      [9] RCU(softirq)
            41.9 µs/s      0.00        Interrupt      [1] timer(softirq)
            28.8 µs/s      0.00        Process        [PID 9] [kworker/0:1]
            27.8 µs/s      0.00        Process        [PID 1283] [kworker/u16:0]
            18.2 µs/s      0.00        Process        [PID 63] [kworker/u16:1]
            11.9 µs/s      0.00        Process        [PID 68] [kworker/u16:6]
             7.9 µs/s      0.00        Timer          tcp_orphan_update
             5.8 µs/s      0.00        Process        [PID 113] [kworker/6:1]
             5.3 µs/s      0.00        kWork          wq_barrier_func
             5.3 µs/s      0.00        Interrupt      [6] tasklet(softirq)
             3.5 µs/s      0.00        Process        [PID 82] [kworker/1:1]
             3.4 µs/s      0.00        Process        [PID 28] [migration/4]
             3.4 µs/s      0.00        Process        [PID 40] [migration/1]
             3.4 µs/s      0.00        Process        [PID 17] [migration/0]
             3.3 µs/s      0.00        Process        [PID 52] [migration/5]
             3.3 µs/s      0.00        Process        [PID 34] [migration/6]
             3.3 µs/s      0.00        Process        [PID 58] [migration/7]
             3.3 µs/s      0.00        Process        [PID 46] [migration/3]
             3.3 µs/s      0.00        Process        [PID 22] [migration/2]
             2.3 µs/s      0.00        Process        [PID 111] [kworker/4:1]
             1.9 µs/s      0.00        kWork          vmstat_update
             1.8 µs/s      0.00        Timer          fq_flush_timeout
             1.7 µs/s      0.00        Timer          mix_interrupt_randomness
             1.4 µs/s      0.00        Process        [PID 112] [kworker/7:1]
             0.8 µs/s      0.00        Timer          dev_watchdog
             0.8 µs/s      0.00        Timer          pm_suspend_timer_fn
             0.7 µs/s      0.00        Process        [PID 305] [kworker/2:2]
             0.6 µs/s      0.00        Process        [PID 93] [kworker/5:1]
             0.5 µs/s      0.00        Process        [PID 1335] [kworker/3:2]
             0.3 µs/s      0.00        Timer          tcp_write_timer

PowerTOP 2.15     Overview   Idle stats   Frequency stats   Device stats   Tunables   WakeUp                      


           Pkg(HW)  |            Core(HW) |            CPU(OS) 0   CPU(OS) 1
                    |                     | C0 active   0.0%        0.0%
                    |                     | POLL        0.0%    0.0 ms  0.0%    0.0 ms
                    |                     | C1_ACPI     0.8%    0.3 ms  0.0%    0.0 ms
C2 (pc2)    0.7%    |                     | C2_ACPI     4.3%    2.1 ms  0.0%    0.0 ms
C3 (pc3)    0.0%    | C3 (cc3)    0.0%    | C3_ACPI    94.8%   59.9 ms 99.9%   40.4 ms
C6 (pc6)    1.1%    | C6 (cc6)    1.2%    |
C7 (pc7)    0.0%    | C7 (cc7)   94.7%    |
C8 (pc8)    0.0%    |                     |
C9 (pc9)    0.0%    |                     |
C10 (pc10) 93.9%    |                     |

                    |            Core(HW) |            CPU(OS) 2   CPU(OS) 3
                    |                     | C0 active   0.0%        0.0%
                    |                     | POLL        0.0%    0.0 ms  0.0%    0.0 ms
                    |                     | C1_ACPI     0.7%    0.7 ms  0.0%    0.0 ms
                    |                     | C2_ACPI     0.0%    0.0 ms  0.0%    0.0 ms
                    | C3 (cc3)    0.0%    | C3_ACPI    99.3%   52.9 ms100.0%   72.3 ms
                    | C6 (cc6)    0.0%    |
                    | C7 (cc7)   97.5%    |
                    |                     |
                    |                     |
                    |                     |

                    |            Core(HW) |            CPU(OS) 4   CPU(OS) 5
                    |                     | C0 active   0.3%        0.2%
                    |                     | POLL        0.2%    0.2 ms  0.1%    0.1 ms
                    |                     | C1_ACPI     0.1%    0.0 ms  0.1%    0.1 ms
                    |                     | C2_ACPI     1.6%    2.0 ms  1.9%    3.3 ms
                    | C3 (cc3)    0.0%    | C3_ACPI    97.8%   90.0 ms 97.7%  109.9 ms
                    | C6 (cc6)    0.8%    |
                    | C7 (cc7)   97.4%    |
                    |                     |
                    |                     |
                    |                     |

                    |            Core(HW) |            CPU(OS) 6   CPU(OS) 7
                    |                     | C0 active   0.0%        0.0%
                    |                     | POLL        0.0%    0.0 ms  0.0%    0.0 ms
                    |                     | C1_ACPI     0.6%    0.6 ms  0.1%    0.5 ms
                    |                     | C2_ACPI     0.1%    0.1 ms  2.6%    1.8 ms
                    | C3 (cc3)    0.0%    | C3_ACPI    99.3%   62.8 ms 97.3%  109.5 ms
                    | C6 (cc6)    0.0%    |
                    | C7 (cc7)   96.6%    |
                    |                     |
                    |                     |
                    |                     |

                    |             GPU     |
                    |                     |
                    | Powered On  0.0%    |
                    | RC6       100.0%    |
                    | RC6p        0.0%    |
                    | RC6pp       0.0%    |
                    |                     |
                    |                     |
                    |                     |
                    |                     |

neofetch:

dietpi@pc:~$ neofetch 
       _,met$$$$$gg.          dietpi@pc 
    ,g$$$$$$$$$$$$$$$P.       --------- 
  ,g$$P"     """Y$$.".        OS: Debian GNU/Linux trixie/sid x86_64 
 ,$$P'              `$$$.     Kernel: 6.5.0-4-amd64 
',$$P       ,ggs.     `$$b:   Uptime: 1 hour, 25 mins 
`d$$'     ,$P"'   .    $$$    Packages: 249 (dpkg) 
 $$P      d$'     ,    $$P    Shell: bash 5.2.15 
 $$:      $$.   -    ,d$$'    Terminal: /dev/pts/0 
 $$;      Y$b._   _,d$P'      CPU: 13th Gen Intel i3-13100 (8) @ 3.400GHz 
 Y$$.    `.`"Y$$$$P"'         GPU: Intel Alder Lake-S GT1 [UHD Graphics 730] 
 `$$b      "-.__              Memory: 191MiB / 31459MiB 
  `Y$$
   `Y$$.                                              
     `$$b.                                            
       `Y$$b.
          `"Y$b._
              `"""

┌─────────────────────────────────────────────────┤ DietPi-Benchmark ├─────────────────────────────────────────────────┐
│                                                                                                                      │
│ Benchmarks completed:                                                                                                │
│  - CPU performance : Duration = 1.13 seconds (lower is faster)                                                       │
│  - CPU temps       : Idle = 30 °C | Full load = 40 °C                                                                │
│  - RootFS I/O      : Write = 1244 MiB/s | Read = 2153 MiB/s                                                          │
│  - RAM I/O         : Write = 3230 MiB/s | Read = 6690 MiB/s                                                          │
│                                                                                                                      │
│ Compare these results online with other users, using the link below:                                                 │
│  - https://dietpi.com/survey#benchmark                                                                               │
│                                                                                                                      │
│                                                        <Ok>                                                          │
│                                                                                                                      │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Here is full powertop report (using command sudo powertop -t 60 -r): PowerTOP report
full dmesg log: H610T @ DebianDebian GNU/Linux trixie/sid @ 6.5.0-4-amd6 - Pastebin.com

wertigon · November 24, 2023, 11:04am

While there is nothing wrong doing this, especially not for a non-critical server, I do hope you are aware of Debian Backports, as well? https://backports.debian.org/

So many Debian users still don’t know how Debian operates, it seems. Glad you could get your issues sorted though!

xnd · December 18, 2023, 10:24am

I’ve tried to connect M.2 to 2.5G LAN Adapter (15 Eur from Aliexpress, with RTL8125B, here are some details: 2.5Gbit Ethernet for TinyMiniMicro labs | «WatchMySys» Blog to mainboard Asus Pro H610T D4-CSM.

good news: it’s possible to connect it directly into M.2 slot (Key E, type 2230) which is a few milimeters under M.2 slot (key M) for NVME SSD. I was able to have Samsung 980 Pro With Heatsink connected above this card.
when it was recognized in system, idle power consumption was the same like before:
2 — 3 W

So at first I could not get it to work. System probably recognized it, but it’s not working.
Output from lspci:

04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
		LnkCap:	Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
		LnkCtl:	ASPM Disabled; RCB 64 bytes, Disabled- CommClk+

whole command:

dietpi@pc:~$ sudo lspci -vv | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM )'
00:01.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 (rev 05) (prog-if 00 [Normal decode])
		LnkCap:	Port #2, Speed 32GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <4us, L1 <16us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
00:1c.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #1 (rev 11) (prog-if 00 [Normal decode])
		LnkCap:	Port #1, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk-
00:1c.2 PCI bridge: Intel Corporation Device 7aba (rev 11) (prog-if 00 [Normal decode])
		LnkCap:	Port #3, Speed 8GT/s, Width x1, ASPM L1, Exit Latency L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
00:1c.3 PCI bridge: Intel Corporation Device 7abb (rev 11) (prog-if 00 [Normal decode])
		LnkCap:	Port #4, Speed 8GT/s, Width x1, ASPM L1, Exit Latency L1 <64us
		LnkCtl:	ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO (prog-if 02 [NVM Express])
pcilib: sysfs_read_vpd: read failed: No such device
		LnkCap:	Port #0, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
		LnkCap:	Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
		LnkCtl:	ASPM Disabled; RCB 64 bytes, Disabled- CommClk+

output from lspci -vv:

scroll below to 04:00.0 Ethernet controller:

03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
	Subsystem: ASUSTeK Computer Inc. PRIME B450M-A Motherboard
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 18
	IOMMU group: 14
	Region 0: I/O ports at 4000 [size=256]
	Region 2: Memory at 81304000 (64-bit, non-prefetchable) [size=4K]
	Region 4: Memory at 81300000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [70] Express (v2) Endpoint, MSI 01
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 10W
		DevCtl:	CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 4096 bytes
		DevSta:	CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
			ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
			ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1
			TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
			 10BitTagComp- 10BitTagReq- OBFF Via message/WAKE#, ExtFmt- EETLPPrefix-
			 EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
			 FRS- TPHComp- ExtTPHComp-
			 AtomicOpsCap: 32bit- 64bit- 128bitCAS-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ 10BitTagReq- OBFF Disabled,
			 AtomicOpsCtl: ReqEn-
		LnkCap2: Supported Link Speeds: 2.5GT/s, Crosslink- Retimer- 2Retimers- DRS-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
			 EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
			 Retimer- 2Retimers- CrosslinkRes: unsupported
	Capabilities: [b0] MSI-X: Enable+ Count=4 Masked-
		Vector table: BAR=4 offset=00000000
		PBA: BAR=4 offset=00000800
	Capabilities: [100 v2] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
		AERCap:	First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
			MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
		HeaderLog: 00000000 00000000 00000000 00000000
	Capabilities: [140 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [160 v1] Device Serial Number 01-00-00-00-68-4c-e0-00
	Capabilities: [170 v1] Latency Tolerance Reporting
		Max snoop latency: 3145728ns
		Max no snoop latency: 3145728ns
	Capabilities: [178 v1] L1 PM Substates
		L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
			  PortCommonModeRestoreTime=150us PortTPowerOnTime=150us
		L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
			   T_CommonMode=0us LTR1.2_Threshold=306176ns
		L1SubCtl2: T_PwrOn=150us
	Kernel driver in use: r8169
	Kernel modules: r8169

04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
	Subsystem: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 19
	IOMMU group: 15
	Region 0: I/O ports at 3000 [size=256]
	Region 2: Memory at 81210000 (64-bit, non-prefetchable) [size=64K]
	Region 4: Memory at 81220000 (64-bit, non-prefetchable) [size=16K]
	Expansion ROM at 81200000 [disabled] [size=64K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] Express (v2) Endpoint, MSI 01
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 10W
		DevCtl:	CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 256 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
			ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
			ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x1
			TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
			 10BitTagComp- 10BitTagReq- OBFF Via message/WAKE#, ExtFmt- EETLPPrefix-
			 EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
			 FRS- TPHComp+ ExtTPHComp-
			 AtomicOpsCap: 32bit- 64bit- 128bitCAS-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ 10BitTagReq- OBFF Disabled,
			 AtomicOpsCtl: ReqEn-
		LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
			 EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
			 Retimer- 2Retimers- CrosslinkRes: unsupported
	Capabilities: [b0] MSI-X: Enable+ Count=32 Masked-
		Vector table: BAR=4 offset=00000000
		PBA: BAR=4 offset=00000800
	Capabilities: [d0] Vital Product Data
pcilib: sysfs_read_vpd: read failed: No such device
		Not readable
	Capabilities: [100 v2] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
		AERCap:	First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
			MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
		HeaderLog: 00000000 00000000 00000000 00000000
	Capabilities: [148 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [168 v1] Device Serial Number 01-00-00-00-68-4c-e0-00
	Capabilities: [178 v1] Transaction Processing Hints
		No steering table available
	Capabilities: [204 v1] Latency Tolerance Reporting
		Max snoop latency: 3145728ns
		Max no snoop latency: 3145728ns
	Capabilities: [20c v1] L1 PM Substates
		L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
			  PortCommonModeRestoreTime=150us PortTPowerOnTime=150us
		L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
			   T_CommonMode=0us LTR1.2_Threshold=317440ns
		L1SubCtl2: T_PwrOn=150us
	Capabilities: [21c v1] Vendor Specific Information: ID=0002 Rev=4 Len=100 <?>
	Kernel driver in use: r8169
	Kernel modules: r8169

In BIOS I’ve tried to enable/disable Wifi card but then it’s not recognized

and `dmesg` producing 10 errors per second:

dietpi@pc:~$ sudo dmesg
[  114.419707] pcieport 0000:00:1c.3:   device [8086:7abb] error status/mask=00000001/00002000
[  114.419711] pcieport 0000:00:1c.3:    [ 0] RxErr                  (First)
[  114.423062] pcieport 0000:00:1c.3: AER: Corrected error received: 0000:00:1c.3
[  114.423084] pcieport 0000:00:1c.3: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[  114.423088] pcieport 0000:00:1c.3:   device [8086:7abb] error status/mask=00000001/00002000
[  114.423092] pcieport 0000:00:1c.3:    [ 0] RxErr                  (First)
[  114.428475] pcieport 0000:00:1c.3: AER: Corrected error received: 0000:00:1c.3
[  114.428499] pcieport 0000:00:1c.3: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[  114.428503] pcieport 0000:00:1c.3:   device [8086:7abb] error status/mask=00000001/00002000
[  114.428507] pcieport 0000:00:1c.3:    [ 0] RxErr                  (First)
[  114.431079] pcieport 0000:00:1c.3: AER: Corrected error received: 0000:00:1c.3
[  114.431103] pcieport 0000:00:1c.3: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[  114.431106] pcieport 0000:00:1c.3:   device [8086:7abb] error status/mask=00000001/00002000
[  114.431111] pcieport 0000:00:1c.3:    [ 0] RxErr                  (First)
[  114.432096] pcieport 0000:00:1c.3: AER: Multiple Corrected error received: 0000:00:1c.3
[  114.432124] pcieport 0000:00:1c.3: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[  114.432128] pcieport 0000:00:1c.3:   device [8086:7abb] error status/mask=00000001/00002000
[  114.432132] pcieport 0000:00:1c.3:    [ 0] RxErr                  (First)
[  114.437767] pcieport 0000:00:1c.3: AER: Corrected error received: 0000:00:1c.3
[  114.437789] pcieport 0000:00:1c.3: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[  114.437793] pcieport 0000:00:1c.3:   device [8086:7abb] error status/mask=00000001/00002000
[  114.437798] pcieport 0000:00:1c.3:    [ 0] RxErr                  (First)
[  114.440400] pcieport 0000:00:1c.3: AER: Corrected error received: 0000:00:1c.3
[  114.440422] pcieport 0000:00:1c.3: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[  114.440426] pcieport 0000:00:1c.3:   device [8086:7abb] error status/mask=00000001/00002000
[  114.440430] pcieport 0000:00:1c.3:    [ 0] RxErr                  (First)
[  114.449552] pcieport 0000:00:1c.3: AER: Corrected error received: 0000:00:1c.3
[  114.449574] pcieport 0000:00:1c.3: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[  114.449577] pcieport 0000:00:1c.3:   device [8086:7abb] error status/mask=00000001/00002000
[  114.449582] pcieport 0000:00:1c.3:    [ 0] RxErr                  (First)
[  114.453974] pcieport 0000:00:1c.3: AER: Corrected error received: 0000:00:1c.3

Today later I will try to update drivers from here: Realtek PCIe FE / GBE / 2.5G / 5G Ethernet Family Controller Software - REALTEK

Is it possible that M.2 slot Key E accepts Wifi/bluetooth cards only??

according to manual:

tjw · December 19, 2023, 3:18am

maybe the red index pin on the idc connector goes to the arrow on the PCB?

xnd · December 22, 2023, 9:00pm

@tjw - yes I noticed it, I re-plugged it correctly.

Finally when I disabled in BIOS built-in Realtek LAN Controller under “Onboard devices” then connected M.2 to 2.5GbE Adapter was working, but not 100% correctly (see dmesg errors at the end of this post).

And power consumption was slightly higher ( 2.5 — 3.5 W ^{with enabled ASPM and only LAN plugged})

# dietpi@pc:~$ lspci -knn | grep Eth --color
03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)

But still with current drivers (r8169) and not with r8125.

I have tried to install drivers from Realtek website (Realtek PCIe FE / GBE / 2.5G / 5G Ethernet Family Controller Software - REALTEK )

# Description                                             Version    Update Time  File Size
2.5G/5G Ethernet LINUX driver r8125 for kernel up to 6.4  9.012.03   2023/10/31   112 KB

I was not successful with installing drivers according to their README.

error when running autorun.sh as root: No rule to make target 'clean'. Stop

I have googled it and tried several solutions:

(Realtek rtl8125) 2.5GBe ethernet not working on AMD B550 mobo - Support & Help Requests - Ubuntu MATE Community
https://www.reddit.com/r/Proxmox/comments/mzbmma/aorus_b550_realtek_r8125_25gb_linux_moduledriver/

I have tried also DKMS version: GitHub - awesometic/realtek-r8125-dkms: A DKMS package for easy use of Realtek r8125 driver, which supports 2.5 GbE.

but again, I was not successful
I got 0 errors, but “Module build for kernel … was skipped since kernel headers … do not seem to be installed”:

dietpi@pc:~$ sudo dpkg -i realtek-r8125-dkms_9.012.03-2_amd64.deb 
(Reading database ... 54969 files and directories currently installed.)
Preparing to unpack realtek-r8125-dkms_9.012.03-2_amd64.deb ...
Deleting module realtek-r8125-9.012.03 completely from the DKMS tree.
Unpacking realtek-r8125-dkms (9.012.03-2) over (9.012.03-2) ...
Setting up realtek-r8125-dkms (9.012.03-2) ...
Loading new realtek-r8125-9.012.03 DKMS files...
Deprecated feature: REMAKE_INITRD (/usr/src/realtek-r8125-9.012.03/dkms.conf)
Building for 6.1.0-16-amd64
Building for architecture amd64
Module build for kernel 6.1.0-16-amd64 was skipped since the
kernel headers for this kernel do not seem to be installed.

and I insalled the kernel headers before

I gave up. I’m not expert to GNU/Linux / Debian. I spent 1/2 day with that.

I have consulted my steps and errors also with ChatGPT. It gave a nice and very accurate and meaningful output in most cases. Not bad.
- here is my chat with ChatGPT: https://chat.openai.com/share/5d03a1a1-0df4-4593-8fdf-b2f2e9fb5c53
- note it was my first usage of ChatGPT

More outputs

ASPM is working:

# dietpi@pc:~$ sudo lspci -vv | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM )'
00:1c.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #1 (rev 11) (prog-if 00 [Normal decode])
		LnkCap:	Port #1, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk-
00:1c.2 PCI bridge: Intel Corporation Device 7aba (rev 11) (prog-if 00 [Normal decode])
		LnkCap:	Port #3, Speed 8GT/s, Width x1, ASPM L1, Exit Latency L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled+ CommClk+
pcilib: sysfs_read_vpd: read failed: No such device
00:1c.3 PCI bridge: Intel Corporation Device 7abb (rev 11) (prog-if 00 [Normal decode])
		LnkCap:	Port #4, Speed 8GT/s, Width x1, ASPM L1, Exit Latency L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
		LnkCap:	Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+

dmesg errors (every few seconds, sometimes several logs per 1 second appeared):

command used: sudo dmesg -w:

...

[  699.292150] pcieport 0000:00:1c.3: AER: Corrected error received: 0000:00:1c.3
[  699.292166] pcieport 0000:00:1c.3: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[  699.292170] pcieport 0000:00:1c.3:   device [8086:7abb] error status/mask=00000001/00002000
[  699.292175] pcieport 0000:00:1c.3:    [ 0] RxErr                  (First)

[  703.292471] pcieport 0000:00:1c.3: AER: Multiple Corrected error received: 0000:00:1c.3
[  703.292656] pcieport 0000:00:1c.3: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[  703.292660] pcieport 0000:00:1c.3:   device [8086:7abb] error status/mask=00000001/00002000
[  703.292665] pcieport 0000:00:1c.3:    [ 0] RxErr                  (First)
[  704.295127] pcieport 0000:00:1c.3: AER: Multiple Corrected error received: 0000:00:1c.3
[  704.295323] pcieport 0000:00:1c.3: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[  704.295327] pcieport 0000:00:1c.3:   device [8086:7abb] error status/mask=00000001/00002000
[  704.295331] pcieport 0000:00:1c.3:    [ 0] RxErr                  (First)

[  706.065033] pcieport 0000:00:1c.3: AER: Corrected error received: 0000:00:1c.3
[  706.065049] pcieport 0000:00:1c.3: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[  706.065054] pcieport 0000:00:1c.3:   device [8086:7abb] error status/mask=00000001/00002000
[  706.065058] pcieport 0000:00:1c.3:    [ 0] RxErr                  (First)

lsmod:

dietpi@pc:~$ sudo lsmod | grep r8
r8169                  94208  0
mdio_devres            16384  1 r8169
libphy                180224  3 r8169,mdio_devres,realtek

here shoud be r8125 ideally

My next plan: tomorrow I will try M.2 to 6x SATA adapter

with ASM1166 chip: https://www.amazon.de/Express-Converter-Downstream-Thunderbolt-Expansion/dp/B0B7CNL4QD

DaddyPig · December 23, 2023, 9:36am

Hi,
I’m new here, but I’ve read some posts from time to time and am a big fan of the youtube channel.

Are you really sure that you installed the correct version of the headers, because it sounds a lot like they’re not.

Regarding the messages from AER, they are pretty much harmless as long as it says “corrected”. You can get rid of them by using boot parameter pci=nommconf. This will switch from memory mapped PCI configuration registers to I/O based.

xnd · December 23, 2023, 2:21pm

I installed the same version of headers as my current kernel. Using command sudo apt-get install linux-headers-$(uname -r).

Now I tried to boot live USB with Ubuntu 23.10 (Kernel: 6.5.0-9-generic) and that RTL8125 / r8125 driver can be installed directly from packages - via apt.

See output of `apt-cache show r8125-dkms`:

ubuntu@ubuntu:~/Downloads$ apt-cache show r8125-dkms 
Package: r8125-dkms
Architecture: all
Version: 9.011.00-4
Priority: optional
Section: multiverse/kernel
Source: r8125
Origin: Ubuntu
Maintainer: Ubuntu Developers <[email protected]>
Original-Maintainer: Hideki Yamane <[email protected]>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 840
Depends: dkms (>= 2.1.0.0)
Filename: pool/multiverse/r/r8125/r8125-dkms_9.011.00-4_all.deb
Size: 102430
MD5sum: 14e9a601a002666f287a028f0cb84395
SHA1: 24c70dd02c4300613eb1a27c5a494d412da47db8
SHA256: 442915a5fd9ad192b1a0ff4fd49b5c3e4c10f02eb6557601633bd2a1ffcfac55
SHA512: 71048ac0b08a98cbf709e60f6ee56f757d2b93a0c6ef222891dd568e94ccec04cebe1cfbd91d085660799045375673332f16b3cb92106d5f70566c3dc08bc0d2
Homepage: https://www.realtek.com/en/component/zoo/category/network-interface-controllers-10-100-1000m-gigabit-ethernet-pci-express-software
Description-en: dkms source for the r8125 network driver
 r8125 is the Linux 2.5G Ethernet device driver released by RealTek for their
 network controller.
 .
 This package provides the dkms source code for the r8125 kernel modules.
 Kernel source or headers are required to compile these modules.
Description-md5: 21605b7f80ee7a56a99a0ef7f36c7598

but still in `dmesg` there are dozens of errors per second.

Output when I tried to install it from .deb package - newest version from https://github.com/awesometic/realtek-r8125-dkms

ubuntu@ubuntu:~$ sudo lsmod | grep r8
r8169                 114688  0
ubuntu@ubuntu:~$ cd Downloads/

ubuntu@ubuntu:~/Downloads$ sudo dpkg -i realtek-r8125-dkms*.deb
Selecting previously unselected package realtek-r8125-dkms.
(Reading database ... 257984 files and directories currently installed.)
Preparing to unpack realtek-r8125-dkms_9.012.03-2_amd64.deb ...
Unpacking realtek-r8125-dkms (9.012.03-2) ...
Setting up realtek-r8125-dkms (9.012.03-2) ...
Loading new realtek-r8125-9.012.03 DKMS files...
Deprecated feature: REMAKE_INITRD (/usr/src/realtek-r8125-9.012.03/dkms.conf)
Building for 6.5.0-9-generic 6.5.0-14-generic
Building for architecture x86_64
Building initial module for 6.5.0-9-generic
Deprecated feature: REMAKE_INITRD (/var/lib/dkms/realtek-r8125/9.012.03/source/dkms.conf)
Done.
Deprecated feature: REMAKE_INITRD (/var/lib/dkms/realtek-r8125/9.012.03/source/dkms.conf)
Deprecated feature: REMAKE_INITRD (/var/lib/dkms/realtek-r8125/9.012.03/source/dkms.conf)

r8125.ko.zst:
Running module version sanity check.
 - Original module
 - Installation
   - Installing to /lib/modules/6.5.0-9-generic/updates/dkms/
depmod...
Building initial module for 6.5.0-14-generic
Deprecated feature: REMAKE_INITRD (/var/lib/dkms/realtek-r8125/9.012.03/source/dkms.conf)
Done.
Deprecated feature: REMAKE_INITRD (/var/lib/dkms/realtek-r8125/9.012.03/source/dkms.conf)
Deprecated feature: REMAKE_INITRD (/var/lib/dkms/realtek-r8125/9.012.03/source/dkms.conf)

r8125.ko.zst:
Running module version sanity check.
 - Original module
 - Installation
   - Installing to /lib/modules/6.5.0-14-generic/updates/dkms/
depmod...

ubuntu@ubuntu:~/Downloads$ sudo lsmod | grep r8
r8169                 114688  0

ubuntu@ubuntu:~/Downloads$ sudo dkms status
Deprecated feature: REMAKE_INITRD (/var/lib/dkms/realtek-r8125/9.012.03/source/dkms.conf)
Deprecated feature: REMAKE_INITRD (/var/lib/dkms/realtek-r8125/9.012.03/source/dkms.conf)
r8125/9.011.00, 6.5.0-14-generic, x86_64: installed (WARNING! Diff between built and installed module!)
r8125/9.011.00, 6.5.0-9-generic, x86_64: installed (WARNING! Diff between built and installed module!)
realtek-r8125/9.012.03, 6.5.0-14-generic, x86_64: installed
realtek-r8125/9.012.03, 6.5.0-9-generic, x86_64: installed

DaddyPig · December 23, 2023, 2:23pm

if those AER errors appear at such high frequency it could really become a problem, just because the log is being spammed.
please try with boot parameter pci=nommconf in grub.

xnd · December 23, 2023, 9:34pm

I have tried M.2 to 6x SATA adapter and I upgraded firmware according to this post Recommended controllers for Unraid - Page 8 - Storage Devices and Controllers - Unraid

I have this version: Asmedia ASM-2116/116x Sata 6G Controller Firmware Version NC

and with enabled powersave --auto-tune and script to enable ASPM on all available PCI devices

idle 13—14W (with 1x SATA SSD connected)
- note: before without this adapter, when SATA SSD was plugged directly into motherboard idle power consumption was only 2—3W
no errors/warnings in dmesg

powertop report: JS Bin - Collaborative JavaScript Debugging

logs (powertop, lspci, neofetch)

dietpi@pc:~$ sudo lspci -vv | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM )'
00:01.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 (rev 05) (prog-if 00 [Normal decode])
		LnkCap:	Port #2, Speed 32GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <4us, L1 <16us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
00:1c.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #1 (rev 11) (prog-if 00 [Normal decode])
		LnkCap:	Port #1, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk-
00:1c.2 PCI bridge: Intel Corporation Device 7aba (rev 11) (prog-if 00 [Normal decode])
		LnkCap:	Port #3, Speed 8GT/s, Width x1, ASPM L1, Exit Latency L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
01:00.0 SATA controller: ASMedia Technology Inc. ASM1166 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0])
		LnkCap:	Port #0, Speed 8GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <4us, L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+

neofetch:

       _,met$$$$$gg.          dietpi@pc 
    ,g$$$$$$$$$$$$$$$P.       --------- 
  ,g$$P"     """Y$$.".        OS: Debian GNU/Linux 12 (bookworm) x86_64 
 ,$$P'              `$$$.     Kernel: 6.1.0-16-amd64 
',$$P       ,ggs.     `$$b:   Uptime: 19 mins 
`d$$'     ,$P"'   .    $$$    Packages: 256 (dpkg) 
 $$P      d$'     ,    $$P    Shell: bash 5.2.15 
 $$:      $$.   -    ,d$$'    Terminal: /dev/pts/0 
 $$;      Y$b._   _,d$P'      CPU: 13th Gen Intel i3-13100 (8) @ 4.400GHz 
 Y$$.    `.`"Y$$$$P"'         GPU: Intel Alder Lake-S GT1 [UHD Graphics 730] 
 `$$b      "-.__              Memory: 163MiB / 31681MiB 
  `Y$$
   `Y$$.                                              
     `$$b.                                            
       `Y$$b.
          `"Y$b._
              `"""`

powertop:

PowerTOP 2.14     Overview   Idle stats   Frequency stats   Device stats   Tunables   WakeUp                            


           Pkg(OS)  |            Core(OS) |            CPU(OS) 0   CPU(OS) 1
Powered On  0.0%    | POLL        0.0%    | POLL        0.0%    0.0 ms  0.0%    0.1 ms
C1_ACPI     0.4%    | C1_ACPI     0.3%    | C1_ACPI     0.1%    0.4 ms  0.5%    1.1 ms
C2_ACPI     0.3%    | C2_ACPI     0.6%    | C2_ACPI     1.2%    2.1 ms  0.1%    2.1 ms
C3_ACPI    99.2%    | C3_ACPI    99.0%    | C3_ACPI    98.6%   67.3 ms 99.4%  160.7 ms
RC6pp       0.0%    |                     |

                    |            Core(OS) |            CPU(OS) 2   CPU(OS) 3
                    | POLL        0.0%    | POLL        0.0%    0.2 ms  0.0%    0.1 ms
                    | C1_ACPI     0.2%    | C1_ACPI     0.1%    1.5 ms  0.4%    0.8 ms
                    | C2_ACPI     0.2%    | C2_ACPI     0.1%    2.2 ms  0.2%    3.3 ms
                    | C3_ACPI    99.2%    | C3_ACPI    99.0%  145.1 ms 99.4%  250.6 ms
                    |                     |

                    |            Core(OS) |            CPU(OS) 4   CPU(OS) 5
                    | POLL        0.0%    | POLL        0.0%    0.1 ms  0.0%    0.1 ms
                    | C1_ACPI     0.5%    | C1_ACPI     0.7%    1.0 ms  0.3%    0.6 ms
                    | C2_ACPI     0.1%    | C2_ACPI     0.1%    3.4 ms  0.0%    0.0 ms
                    | C3_ACPI    99.4%    | C3_ACPI    99.1%  151.2 ms 99.7%  326.0 ms
                    |                     |

                    |            Core(OS) |            CPU(OS) 6   CPU(OS) 7
                    | POLL        0.0%    | POLL        0.0%    0.0 ms  0.0%    0.0 ms
                    | C1_ACPI     0.5%    | C1_ACPI     0.1%    2.3 ms  0.9%    0.8 ms
                    | C2_ACPI     0.2%    | C2_ACPI     0.2%    2.8 ms  0.1%    3.6 ms
                    | C3_ACPI    99.2%    | C3_ACPI    99.4%  257.2 ms 99.0%  160.0 ms
                    |                     |

                    |             GPU     |
                    | Powered On  0.0%    |
                    | RC6       100.0%    |
                    | RC6p        0.0%    |
                    | RC6pp       0.0%    |
                    |                     |


PowerTOP 2.14     Overview   Idle stats   Frequency stats   Device stats   Tunables   WakeUp                            


              Usage     Device name
              0.4%        CPU misc
              0.4%        CPU core
            100.0%        PCI Device: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
            100.0%        PCI Device: Intel Corporation Platform Monitoring Technology
            100.0%        PCI Device: Intel Corporation Device 7aba
            100.0%        PCI Device: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1
            100.0%        PCI Device: Intel Corporation Alder Lake Innovation Platform Framework Processor Participant
            100.0%        PCI Device: ASMedia Technology Inc. ASM1166 Serial ATA Controller
             65.6%        PCI Device: Intel Corporation Alder Lake-S GT1 [UHD Graphics 730]
             11.0 pkts/s  Network interface: eth0 (r8169)
              0.0%        runtime-ACPI0007:15
              0.0%        USB device: xHCI Host Controller
              0.0%        runtime-INT33A1:00
              0.0%        PCI Device: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode]
              0.0%        PCI Device: Intel Corporation Alder Lake-S PCH PCI Express Root Port #1
              0.0%        PCI Device: Intel Corporation Device 7a87
              0.0%        PCI Device: Intel Corporation Alder Lake-S PCH HECI Controller #1
              0.0%        PCI Device: Intel Corporation Alder Lake-S PCH SPI Controller
              0.0%        PCI Device: Intel Corporation Device 4630
              0.0%        PCI Device: Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #0
              0.0%        PCI Device: Intel Corporation Alder Lake-S PCH Shared SRAM
              0.0%        PCI Device: Intel Corporation Alder Lake-S PCH USB 3.2 Gen 2x2 XHCI Controller
              0.0%        PCI Device: Intel Corporation Alder Lake-S PCH SMBus Controller
              0.0%        runtime-ACPI0007:31
              0.0%        USB device: xHCI Host Controller
              0.0%        runtime-PNP0C14:00
              0.0%        runtime-alarmtimer.0.auto
              0.0%        runtime-ACPI0007:0c
              0.0%        runtime-ACPI0007:33
              0.0%        runtime-reg-dummy
              0.0%        runtime-ACPI0007:3a
              0.0%        runtime-iTCO_wdt
              0.0%        runtime-ACPI0007:23
              0.0%        runtime-ACPI0007:2a
              0.0%        runtime-PNP0C0B:03
              0.0%        runtime-ACPI0007:13
              0.0%        runtime-ACPI0007:1a
              0.0%        runtime-ACPI0007:0a
              0.0%        runtime-ACPI0007:1f

nutral · January 5, 2024, 12:27pm

Did you test anything running that actually has server applications running off of the ssd/m2 slot? I’ve noticed i drop from 20 to 14 if i just stop my containers.

haven’t really had the time to go into it, and it’s running a lot of things, else i would have tried remove the m2 ssd and just run everything off of a bigger sata ssd.

xnd · January 9, 2024, 8:25pm

I’m still not using it regularly. Because I’m waiting for some parts for passive cooling.

BTW there are 2 new BIOSes since I last was checking that:

see new BIOS info:

xnd · March 2, 2024, 10:23am

Again, new BIOS 3011 released yesterday with note:

Improve system performance

xnd · March 10, 2024, 12:45pm

Again, new BIOS 3012

Summary


"• Update the Intel microcode to version 0x123, allowing for disabling CEP for improved performance on Intel 14th Gen series processors.
*Disabling CEP for performance enhancement may lead to increased power consumption and temperature. Ensure adequate CPU cooling or use a higher-spec cooling solution;
**Intel CEP function varies by the CPU model. Check Intel specification sheet or contact technical support for details.
Updating this BIOS will simultaneously update the corresponding Intel ME to version 16.1.30.2307v4. Please note after you update this BIOS, the ME version remains the updated one even if you roll back to an older BIOS later."

xnd · April 2, 2024, 11:34am

Again, new BIOS: 3206

Pro H610T D4 BIOS 3206 (2024-04-02)

Summary

Version 3206 | 8.97 MB | 2024/04/02

“Further optimized CEP settings when disabled
Updating this BIOS will simultaneously update the corresponding Intel ME to version 16.1.30.2307v4. Please note after you update this BIOS, the ME version remains the updated one even if you roll back to an older BIOS later.”

xnd · May 3, 2024, 1:42pm

Again, new BIOS: 3210

Pro H610T D4 BIOS 3210 (2024-04-24)

Summary

Version 3210 | 8.97 MB | 2024/04/24

“1. The update introduces the Intel Baseline Profile option, allowing users to revert to Intel factory default settings for basic functionality, lower power limits, and improving stability in certain games.
Updating this BIOS will simultaneously update the corresponding Intel ME to version 16.1.30.2307v4. Please note after you update this BIOS, the ME version remains the updated one even if you roll back to an older BIOS later.”

nutral · May 5, 2024, 8:20am

So i had an issue again with my zfs pool. The same errors making me think it is an power issue as i’ve had it before from enabling power management options (that have been off)

Installed a new power supply instead of the pico PSU. Don’t have the errors, but i do still get the errors if i enable power management, so i guess it will stay off.
Idle usage has gone from average 30W to 31,5W, so not huge. I did specifically chose this bequiet 550W 12M because it has been tested to be pretty efficient.