@wendell 's dual Xeon 8380’s did 3.8 minutes on the older Bladebit. >=1.1.0 added some optimizations that push times down into 3.4’s (across NUMA). Faster effective times are possible by NUMA isolating a pair provided you have >420GB per socket. I’ve been meaning to try that here…
2 Likes
Yeah so when doing ramplot you really want to run it on all the available threads and maybe leave 2 threads idle so max threads - 2, numa nodes should be left on either home snoop or cluster on die setting but my testing has shown that home snoop gives best result for this workload, numa nodes should not be interleaved.
Bladebit should be run with -w to enable warm start for testing if running only one plot, if running many in a single test run it can be omitted. Bladebit will do its best to bind itself on specific numa nodes based on the physical cpu that its running on unless -m or –no-cpu-affinity flags are set.
@wendell Can we please get a single plot run with -w flag specified running on all 256 threads if running on dual 7773x system or 128 threads if running on single one. Thanks a bunch.
Dual affinity pined processes on dual CPUs make no sense - you still need expensive 420 Gb of RAM per process especially if you want to populate all 12 channels you must buy 768, and you need also to Rsync it out fast enough which also takes resources and must be either affinity pinned as well or it will be slowing down one CPU to rsync out plots form both CPUs. On what CPU will network card be affiliated 25Gb/s no less? On what CPU will be SSD that accepts 100Gb every 4 minutes per CPU? Essentially it’s the performance per socket what matters.
In the new generation I think either 96 core or 48 core high speed will take the crown.
By the way @wendell if you know guys from Phoronix (cause you know everyone) - they have Chia VDF benchmark, which is Chia farming. People do it on Raspberry Pi - it eludes me what is to test there and who cares. Tell them this is what they should be benchmarking!
Theres 1.5tb in the genoa system 
512gb per socket is same cost whether 1s or 2s system
Genoa is the first platform ive seen that can saturate 200gbe with no tuning out of the box and suuupet low cpu usage
2 Likes
[w@fedora build-release]$ ./bladebit -w --threads 256 --count 5 --farmer-key ad65cab5777776b47a54cc2947e2089c8743d0374dcbca44ed98baee55d045fd2a1c9a7b4fd6dfdd71701648bfd67340 --pool-key 8f4b9db4409333ece47f2024e33946d6d312cc0207b704aae63f2fa38eecf0fb8dc658a9fdd41ff1bcf6aa9bda016fb9 --verbose ramplot …/foo
Bladebit Chia Plotter
Version : 2.0.1-dev
Git Commit : 9fac46aff0476e829d476412de18497a3a2f7ed8
Compiled With: gcc 12.2.1
[Global Plotting Config]
Will create 5 plots.
Thread count : 256
Warm start enabled : true
NUMA disabled : false
CPU affinity disabled : false
Farmer public key : ad65cab5777776b47a54cc2947e2089c8743d0374dcbca44ed98baee55d045fd2a1c9a7b4fd6dfdd71701648bfd67340
Pool public key : 8f4b9db4409333ece47f2024e33946d6d312cc0207b704aae63f2fa38eecf0fb8dc658a9fdd41ff1bcf6aa9bda016fb9
Output path : …/foo
System Memory: 492/503 GiB.
Memory required: 416 GiB.
Allocating buffers.
Generating plot 1 / 5: 4c0491c124c114459b5faa1781b0dd892261c09812ddd2ca72d3c373e8867a28
Running Phase 1
Generating F1…
Finished F1 generation in 0.61 seconds.
Sorting F1…
Finished F1 sort in 3.92 seconds.
Forward propagating to table 2…
Pairing L/R groups…
Finished pairing L/R groups in 2.5710 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 2.8850 seconds.
Sorting entries…
Finished sorting in 18.50 seconds.
Finished forward propagating table 2 in 24.13 seconds.
Forward propagating to table 3…
Pairing L/R groups…
Finished pairing L/R groups in 2.5500 seconds. Created 4294864024 pairs.
Average of 236.1349 pairs per group.
Computing Fx…
Finished computing Fx in 2.9710 seconds.
Sorting entries…
Finished sorting in 14.29 seconds.
Finished forward propagating table 3 in 19.98 seconds.
Forward propagating to table 4…
Pairing L/R groups…
Finished pairing L/R groups in 2.5240 seconds. Created 4294900187 pairs.
Average of 236.1369 pairs per group.
Computing Fx…
Finished computing Fx in 3.1980 seconds.
Sorting entries…
Finished sorting in 14.26 seconds.
Finished forward propagating table 4 in 20.14 seconds.
Forward propagating to table 5…
Pairing L/R groups…
Finished pairing L/R groups in 2.5510 seconds. Created 4294813187 pairs.
Average of 236.1322 pairs per group.
Computing Fx…
Finished computing Fx in 3.1960 seconds.
Sorting entries…
Finished sorting in 14.36 seconds.
Finished forward propagating table 5 in 20.27 seconds.
Forward propagating to table 6…
Pairing L/R groups…
Finished pairing L/R groups in 2.5540 seconds. Created 4294685921 pairs.
Average of 236.1252 pairs per group.
Computing Fx…
Finished computing Fx in 3.0500 seconds.
Sorting entries…
Finished sorting in 18.42 seconds.
Finished forward propagating table 6 in 24.19 seconds.
Forward propagating to table 7…
Pairing L/R groups…
Finished pairing L/R groups in 2.5500 seconds. Created 4294480894 pairs.
Average of 236.1139 pairs per group.
Computing Fx…
Finished computing Fx in 2.9170 seconds.
Finished forward propagating table 7 in 5.63 seconds.
Finished Phase 1 in 118.87 seconds.
Running Phase 2
Prunning table 6…
Finished prunning table 6 in 0.20 seconds.
Prunning table 5…
Finished prunning table 5 in 10.41 seconds.
Prunning table 4…
Finished prunning table 4 in 9.92 seconds.
Prunning table 3…
Finished prunning table 3 in 9.70 seconds.
Prunning table 2…
Finished prunning table 2 in 9.64 seconds.
Finished Phase 2 in 40.03 seconds.
Running Phase 3
Compressing tables 1 and 2…
Finished compressing tables 1 and 2 in 39.83 seconds
Table 1 now has 3429383129 / 4294967296 entries ( 79.85% ).
Compressing tables 2 and 3…
Finished compressing tables 2 and 3 in 41.07 seconds
Table 2 now has 3439837008 / 4294864024 entries ( 80.09% ).
Compressing tables 3 and 4…
Finished compressing tables 3 and 4 in 41.36 seconds
Table 3 now has 3466042927 / 4294900187 entries ( 80.70% ).
Compressing tables 4 and 5…
Finished compressing tables 4 and 5 in 42.09 seconds
Table 4 now has 3532847598 / 4294813187 entries ( 82.26% ).
Compressing tables 5 and 6…
Finished compressing tables 5 and 6 in 44.25 seconds
Table 5 now has 3713394526 / 4294685921 entries ( 86.46% ).
Compressing tables 6 and 7…
Finished compressing tables 6 and 7 in 24.64 seconds
Table 6 now has 4294480894 / 4294480894 entries ( 100.00% ).
Finished Phase 3 in 233.25 seconds.
Running Phase 4
Writing P7.
Finished writing P7 in 0.23 seconds.
Writing C1 table.
Finished writing C1 table in 0.01 seconds.
Writing C2 table.
Finished writing C2 table in 0.00 seconds.
Writing C3 table.
Finished writing C3 table in 0.14 seconds.
Finished Phase 4 in 0.38 seconds.
Finished plotting in 392.52 seconds (6.54 minutes).
Generating plot 2 / 5: 8af5e36e39b425f8d9814bbf16bab99c1b663c9271188d989a8413077affab4c
Running Phase 1
Generating F1…
Finished F1 generation in 0.63 seconds.
Sorting F1…
Finished F1 sort in 6.59 seconds.
Forward propagating to table 2…
Pairing L/R groups…
Finished pairing L/R groups in 3.5990 seconds. Created 4294965059 pairs.
Average of 236.1405 pairs per group.
Computing Fx…
Finished computing Fx in 4.6850 seconds.
Waiting for last plot to finish being written to disk…
Previous plot …/foo/plot-k32-2022-11-18-08-27-4c0491c124c114459b5faa1781b0dd892261c09812ddd2ca72d3c373e8867a28.plot.tmp finished writing to disk:
Table 1 pointer : 4096 ( 0x0000000000001000 )
Table 2 pointer : 14839459840 ( 0x0000000374803000 )
Table 3 pointer : 28822200320 ( 0x00000006b5efe000 )
Table 4 pointer : 42911465472 ( 0x00000009fdb90000 )
Table 5 pointer : 57272291328 ( 0x0000000d55b1d000 )
Table 6 pointer : 72367026176 ( 0x00000010d9693000 )
Table 7 pointer : 89823846400 ( 0x00000014e9eb2000 )
C1 table pointer : 107538587648 ( 0x0000001909cca000 )
C2 table pointer : 107540307968 ( 0x0000001909e6e000 )
C3 table pointer : 107540312064 ( 0x0000001909e6f000 )
Sorting entries…
Finished sorting in 18.46 seconds.
Finished forward propagating table 2 in 49.97 seconds.
Forward propagating to table 3…
Pairing L/R groups…
Finished pairing L/R groups in 2.5520 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 2.9760 seconds.
Sorting entries…
Finished sorting in 14.47 seconds.
Finished forward propagating table 3 in 20.16 seconds.
Forward propagating to table 4…
Pairing L/R groups…
Finished pairing L/R groups in 2.5550 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 3.1990 seconds.
Sorting entries…
Finished sorting in 14.24 seconds.
Finished forward propagating table 4 in 20.16 seconds.
Forward propagating to table 5…
Pairing L/R groups…
Finished pairing L/R groups in 2.5470 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 3.1960 seconds.
Sorting entries…
Finished sorting in 14.28 seconds.
Finished forward propagating table 5 in 20.18 seconds.
Forward propagating to table 6…
Pairing L/R groups…
Finished pairing L/R groups in 2.5490 seconds. Created 4294897990 pairs.
Average of 236.1368 pairs per group.
Computing Fx…
Finished computing Fx in 3.0530 seconds.
Sorting entries…
Finished sorting in 18.42 seconds.
Finished forward propagating table 6 in 24.19 seconds.
Forward propagating to table 7…
Pairing L/R groups…
Finished pairing L/R groups in 2.5540 seconds. Created 4294789040 pairs.
Average of 236.1308 pairs per group.
Computing Fx…
Finished computing Fx in 2.9160 seconds.
Finished forward propagating table 7 in 5.63 seconds.
Finished Phase 1 in 147.52 seconds.
Running Phase 2
Prunning table 6…
Finished prunning table 6 in 0.20 seconds.
Prunning table 5…
Finished prunning table 5 in 10.42 seconds.
Prunning table 4…
Finished prunning table 4 in 9.92 seconds.
Prunning table 3…
Finished prunning table 3 in 9.71 seconds.
Prunning table 2…
Finished prunning table 2 in 9.64 seconds.
Finished Phase 2 in 40.04 seconds.
Running Phase 3
Compressing tables 1 and 2…
Finished compressing tables 1 and 2 in 39.87 seconds
Table 1 now has 3429397181 / 4294965059 entries ( 79.85% ).
Compressing tables 2 and 3…
Finished compressing tables 2 and 3 in 41.05 seconds
Table 2 now has 3439935979 / 4294967296 entries ( 80.09% ).
Compressing tables 3 and 4…
Finished compressing tables 3 and 4 in 41.20 seconds
Table 3 now has 3466150286 / 4294967296 entries ( 80.70% ).
Compressing tables 4 and 5…
Finished compressing tables 4 and 5 in 42.07 seconds
Table 4 now has 3533027689 / 4294967296 entries ( 82.26% ).
Compressing tables 5 and 6…
Finished compressing tables 5 and 6 in 44.11 seconds
Table 5 now has 3713630918 / 4294897990 entries ( 86.47% ).
Compressing tables 6 and 7…
Finished compressing tables 6 and 7 in 24.84 seconds
Table 6 now has 4294789040 / 4294789040 entries ( 100.00% ).
Finished Phase 3 in 233.16 seconds.
Running Phase 4
Writing P7.
Finished writing P7 in 0.23 seconds.
Writing C1 table.
Finished writing C1 table in 0.00 seconds.
Writing C2 table.
Finished writing C2 table in 0.00 seconds.
Writing C3 table.
Finished writing C3 table in 0.14 seconds.
Finished Phase 4 in 0.37 seconds.
Finished plotting in 421.09 seconds (7.02 minutes).
Generating plot 3 / 5: 9d86f6eb3023cc7b26e88fb93de93ba7280319d62ef20f60b02433f327b06af5
Running Phase 1
Generating F1…
Finished F1 generation in 0.63 seconds.
Sorting F1…
Finished F1 sort in 6.55 seconds.
Forward propagating to table 2…
Pairing L/R groups…
Finished pairing L/R groups in 3.6190 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 4.6750 seconds.
Waiting for last plot to finish being written to disk…
Previous plot …/foo/plot-k32-2022-11-18-08-33-8af5e36e39b425f8d9814bbf16bab99c1b663c9271188d989a8413077affab4c.plot.tmp finished writing to disk:
Table 1 pointer : 4096 ( 0x0000000000001000 )
Table 2 pointer : 14839521280 ( 0x0000000374812000 )
Table 3 pointer : 28822667264 ( 0x00000006b5f70000 )
Table 4 pointer : 42912374784 ( 0x00000009fdc6e000 )
Table 5 pointer : 57273933824 ( 0x0000000d55cae000 )
Table 6 pointer : 72369635328 ( 0x00000010d9910000 )
Table 7 pointer : 89827704832 ( 0x00000014ea260000 )
C1 table pointer : 107543711744 ( 0x000000190a1ad000 )
C2 table pointer : 107545432064 ( 0x000000190a351000 )
C3 table pointer : 107545436160 ( 0x000000190a352000 )
Sorting entries…
Finished sorting in 18.51 seconds.
Finished forward propagating table 2 in 50.14 seconds.
Forward propagating to table 3…
Pairing L/R groups…
Finished pairing L/R groups in 2.5470 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 2.9760 seconds.
Sorting entries…
Finished sorting in 14.92 seconds.
Finished forward propagating table 3 in 20.60 seconds.
Forward propagating to table 4…
Pairing L/R groups…
Finished pairing L/R groups in 2.5490 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 3.2060 seconds.
Sorting entries…
Finished sorting in 14.55 seconds.
Finished forward propagating table 4 in 20.47 seconds.
Forward propagating to table 5…
Pairing L/R groups…
Finished pairing L/R groups in 2.5430 seconds. Created 4294897714 pairs.
Average of 236.1368 pairs per group.
Computing Fx…
Finished computing Fx in 3.1940 seconds.
Sorting entries…
Finished sorting in 14.20 seconds.
Finished forward propagating table 5 in 20.10 seconds.
Forward propagating to table 6…
Pairing L/R groups…
Finished pairing L/R groups in 2.5570 seconds. Created 4294778528 pairs.
Average of 236.1302 pairs per group.
Computing Fx…
Finished computing Fx in 3.0490 seconds.
Sorting entries…
Finished sorting in 18.46 seconds.
Finished forward propagating table 6 in 24.23 seconds.
Forward propagating to table 7…
Pairing L/R groups…
Finished pairing L/R groups in 2.5480 seconds. Created 4294605882 pairs.
Average of 236.1208 pairs per group.
Computing Fx…
Finished computing Fx in 2.9160 seconds.
Finished forward propagating table 7 in 5.63 seconds.
Finished Phase 1 in 148.34 seconds.
Running Phase 2
Prunning table 6…
Finished prunning table 6 in 0.20 seconds.
Prunning table 5…
Finished prunning table 5 in 10.38 seconds.
Prunning table 4…
Finished prunning table 4 in 9.90 seconds.
Prunning table 3…
Finished prunning table 3 in 9.69 seconds.
Prunning table 2…
Finished prunning table 2 in 10.93 seconds.
Finished Phase 2 in 41.25 seconds.
Running Phase 3
Compressing tables 1 and 2…
Finished compressing tables 1 and 2 in 39.74 seconds
Table 1 now has 3429364385 / 4294967296 entries ( 79.85% ).
Compressing tables 2 and 3…
Finished compressing tables 2 and 3 in 41.09 seconds
Table 2 now has 3439841154 / 4294967296 entries ( 80.09% ).
Compressing tables 3 and 4…
Finished compressing tables 3 and 4 in 41.31 seconds
Table 3 now has 3466083954 / 4294967296 entries ( 80.70% ).
Compressing tables 4 and 5…
Finished compressing tables 4 and 5 in 42.09 seconds
Table 4 now has 3532879869 / 4294897714 entries ( 82.26% ).
Compressing tables 5 and 6…
Finished compressing tables 5 and 6 in 44.22 seconds
Table 5 now has 3713464656 / 4294778528 entries ( 86.46% ).
Compressing tables 6 and 7…
Finished compressing tables 6 and 7 in 24.56 seconds
Table 6 now has 4294605882 / 4294605882 entries ( 100.00% ).
Finished Phase 3 in 233.01 seconds.
Running Phase 4
Writing P7.
Finished writing P7 in 0.22 seconds.
Writing C1 table.
Finished writing C1 table in 0.01 seconds.
Writing C2 table.
Finished writing C2 table in 0.00 seconds.
Writing C3 table.
Finished writing C3 table in 0.12 seconds.
Finished Phase 4 in 0.36 seconds.
Finished plotting in 422.96 seconds (7.05 minutes).
Generating plot 4 / 5: 46e8fc219cf19b0e3df327536b21fe1cd7f920b5d302c2a7985a3f679aedb234
Running Phase 1
Generating F1…
Finished F1 generation in 0.61 seconds.
Sorting F1…
Finished F1 sort in 6.46 seconds.
Forward propagating to table 2…
Pairing L/R groups…
Finished pairing L/R groups in 3.6460 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 4.6840 seconds.
Waiting for last plot to finish being written to disk…
Previous plot …/foo/plot-k32-2022-11-18-08-41-9d86f6eb3023cc7b26e88fb93de93ba7280319d62ef20f60b02433f327b06af5.plot.tmp finished writing to disk:
Table 1 pointer : 4096 ( 0x0000000000001000 )
Table 2 pointer : 14839382016 ( 0x00000003747f0000 )
Table 3 pointer : 28822138880 ( 0x00000006b5eef000 )
Table 4 pointer : 42911571968 ( 0x00000009fdbaa000 )
Table 5 pointer : 57272524800 ( 0x0000000d55b56000 )
Table 6 pointer : 72367550464 ( 0x00000010d9713000 )
Table 7 pointer : 89824878592 ( 0x00000014e9fae000 )
C1 table pointer : 107540131840 ( 0x0000001909e43000 )
C2 table pointer : 107541852160 ( 0x0000001909fe7000 )
C3 table pointer : 107541856256 ( 0x0000001909fe8000 )
Sorting entries…
Finished sorting in 18.51 seconds.
Finished forward propagating table 2 in 50.27 seconds.
Forward propagating to table 3…
Pairing L/R groups…
Finished pairing L/R groups in 2.5290 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 2.9820 seconds.
Sorting entries…
Finished sorting in 14.25 seconds.
Finished forward propagating table 3 in 19.93 seconds.
Forward propagating to table 4…
Pairing L/R groups…
Finished pairing L/R groups in 2.5530 seconds. Created 4294947702 pairs.
Average of 236.1396 pairs per group.
Computing Fx…
Finished computing Fx in 3.2030 seconds.
Sorting entries…
Finished sorting in 14.23 seconds.
Finished forward propagating table 4 in 20.15 seconds.
Forward propagating to table 5…
Pairing L/R groups…
Finished pairing L/R groups in 2.5560 seconds. Created 4294951995 pairs.
Average of 236.1398 pairs per group.
Computing Fx…
Finished computing Fx in 3.2020 seconds.
Sorting entries…
Finished sorting in 14.22 seconds.
Finished forward propagating table 5 in 20.15 seconds.
Forward propagating to table 6…
Pairing L/R groups…
Finished pairing L/R groups in 2.5520 seconds. Created 4294894532 pairs.
Average of 236.1366 pairs per group.
Computing Fx…
Finished computing Fx in 3.0470 seconds.
Sorting entries…
Finished sorting in 18.43 seconds.
Finished forward propagating table 6 in 24.19 seconds.
Forward propagating to table 7…
Pairing L/R groups…
Finished pairing L/R groups in 2.5520 seconds. Created 4294792060 pairs.
Average of 236.1310 pairs per group.
Computing Fx…
Finished computing Fx in 2.9260 seconds.
Finished forward propagating table 7 in 5.64 seconds.
Finished Phase 1 in 147.40 seconds.
Running Phase 2
Prunning table 6…
Finished prunning table 6 in 0.20 seconds.
Prunning table 5…
Finished prunning table 5 in 10.42 seconds.
Prunning table 4…
Finished prunning table 4 in 9.91 seconds.
Prunning table 3…
Finished prunning table 3 in 9.69 seconds.
Prunning table 2…
Finished prunning table 2 in 9.65 seconds.
Finished Phase 2 in 40.02 seconds.
Running Phase 3
Compressing tables 1 and 2…
Finished compressing tables 1 and 2 in 39.86 seconds
Table 1 now has 3429430620 / 4294967296 entries ( 79.85% ).
Compressing tables 2 and 3…
Finished compressing tables 2 and 3 in 41.02 seconds
Table 2 now has 3439928277 / 4294967296 entries ( 80.09% ).
Compressing tables 3 and 4…
Finished compressing tables 3 and 4 in 41.29 seconds
Table 3 now has 3466118696 / 4294947702 entries ( 80.70% ).
Compressing tables 4 and 5…
Finished compressing tables 4 and 5 in 41.16 seconds
Table 4 now has 3532954610 / 4294951995 entries ( 82.26% ).
Compressing tables 5 and 6…
Finished compressing tables 5 and 6 in 44.16 seconds
Table 5 now has 3713616831 / 4294894532 entries ( 86.47% ).
Compressing tables 6 and 7…
Finished compressing tables 6 and 7 in 24.31 seconds
Table 6 now has 4294792060 / 4294792060 entries ( 100.00% ).
Finished Phase 3 in 231.80 seconds.
Running Phase 4
Writing P7.
Finished writing P7 in 0.21 seconds.
Writing C1 table.
Finished writing C1 table in 0.00 seconds.
Writing C2 table.
Finished writing C2 table in 0.00 seconds.
Writing C3 table.
Finished writing C3 table in 0.11 seconds.
Finished Phase 4 in 0.32 seconds.
Finished plotting in 419.54 seconds (6.99 minutes).
Generating plot 5 / 5: 17cd7e60e84a7b2c1408312cd3850ecb193269732c67eec89765074fb3b388e4
Running Phase 1
Generating F1…
Finished F1 generation in 0.61 seconds.
Sorting F1…
Finished F1 sort in 3.92 seconds.
Forward propagating to table 2…
Pairing L/R groups…
Finished pairing L/R groups in 2.5530 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 2.8970 seconds.
Waiting for last plot to finish being written to disk…
Previous plot …/foo/plot-k32-2022-11-18-08-48-46e8fc219cf19b0e3df327536b21fe1cd7f920b5d302c2a7985a3f679aedb234.plot.tmp finished writing to disk:
Table 1 pointer : 4096 ( 0x0000000000001000 )
Table 2 pointer : 14839664640 ( 0x0000000374835000 )
Table 3 pointer : 28822777856 ( 0x00000006b5f8b000 )
Table 4 pointer : 42912350208 ( 0x00000009fdc68000 )
Table 5 pointer : 57273610240 ( 0x0000000d55c5f000 )
Table 6 pointer : 72369250304 ( 0x00000010d98b2000 )
Table 7 pointer : 89827336192 ( 0x00000014ea206000 )
C1 table pointer : 107543359488 ( 0x000000190a157000 )
C2 table pointer : 107545079808 ( 0x000000190a2fb000 )
C3 table pointer : 107545083904 ( 0x000000190a2fc000 )
Sorting entries…
Finished sorting in 18.43 seconds.
Finished forward propagating table 2 in 80.93 seconds.
Forward propagating to table 3…
Pairing L/R groups…
Finished pairing L/R groups in 2.5570 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 2.9790 seconds.
Sorting entries…
Finished sorting in 14.22 seconds.
Finished forward propagating table 3 in 19.92 seconds.
Forward propagating to table 4…
Pairing L/R groups…
Finished pairing L/R groups in 2.5510 seconds. Created 4294838050 pairs.
Average of 236.1335 pairs per group.
Computing Fx…
Finished computing Fx in 3.2080 seconds.
Sorting entries…
Finished sorting in 14.23 seconds.
Finished forward propagating table 4 in 20.15 seconds.
Forward propagating to table 5…
Pairing L/R groups…
Finished pairing L/R groups in 2.5560 seconds. Created 4294671755 pairs.
Average of 236.1244 pairs per group.
Computing Fx…
Finished computing Fx in 3.1960 seconds.
Sorting entries…
Finished sorting in 14.22 seconds.
Finished forward propagating table 5 in 20.13 seconds.
Forward propagating to table 6…
Pairing L/R groups…
Finished pairing L/R groups in 2.5520 seconds. Created 4294332828 pairs.
Average of 236.1057 pairs per group.
Computing Fx…
Finished computing Fx in 3.0500 seconds.
Sorting entries…
Finished sorting in 18.43 seconds.
Finished forward propagating table 6 in 24.20 seconds.
Forward propagating to table 7…
Pairing L/R groups…
Finished pairing L/R groups in 2.5520 seconds. Created 4293698703 pairs.
Average of 236.0709 pairs per group.
Computing Fx…
Finished computing Fx in 2.9210 seconds.
Finished forward propagating table 7 in 5.64 seconds.
Finished Phase 1 in 175.50 seconds.
Running Phase 2
Prunning table 6…
Finished prunning table 6 in 0.21 seconds.
Prunning table 5…
Finished prunning table 5 in 10.37 seconds.
Prunning table 4…
Finished prunning table 4 in 9.88 seconds.
Prunning table 3…
Finished prunning table 3 in 9.68 seconds.
Prunning table 2…
Finished prunning table 2 in 11.53 seconds.
Finished Phase 2 in 41.83 seconds.
Running Phase 3
Compressing tables 1 and 2…
Finished compressing tables 1 and 2 in 39.80 seconds
Table 1 now has 3429370796 / 4294967296 entries ( 79.85% ).
Compressing tables 2 and 3…
Finished compressing tables 2 and 3 in 40.97 seconds
Table 2 now has 3439874542 / 4294967296 entries ( 80.09% ).
Compressing tables 3 and 4…
Finished compressing tables 3 and 4 in 41.20 seconds
Table 3 now has 3465945706 / 4294838050 entries ( 80.70% ).
Compressing tables 4 and 5…
Finished compressing tables 4 and 5 in 42.14 seconds
Table 4 now has 3532642332 / 4294671755 entries ( 82.26% ).
Compressing tables 5 and 6…
Finished compressing tables 5 and 6 in 44.20 seconds
Table 5 now has 3712970225 / 4294332828 entries ( 86.46% ).
Compressing tables 6 and 7…
Finished compressing tables 6 and 7 in 24.55 seconds
Table 6 now has 4293698703 / 4293698703 entries ( 100.00% ).
Finished Phase 3 in 232.85 seconds.
Running Phase 4
Writing P7.
Finished writing P7 in 0.22 seconds.
Writing C1 table.
Finished writing C1 table in 0.01 seconds.
Writing C2 table.
Finished writing C2 table in 0.00 seconds.
Writing C3 table.
Finished writing C3 table in 0.14 seconds.
Finished Phase 4 in 0.37 seconds.
Writing final plot tables to disk
2 Likes
Oh wow, I did not expect that at all, I have recently run it on 192 threads on m6a.metal (2 x 7R13) AWS instance and got time of 233 seconds (without copy) 265.17 secs with copy. I wander if the plotter is maybe broken in 2.x.x version.
For comparison my R930’s (4 x e7-8890v4) do plots in 253 seconds no copy and 258 seconds with copy.
Thanks a bunch for this test man.
I should maybe check tdp settings etc if this is below expectations incase the last person using this used ctdp to do funky things
That’s the thing I think everyone is trying to gauge exactly that, to me the numbers seam to be too high judging by other benchmarks that I run in the past. Also refusing to believe that 6 year old platform with 96c/192t (r930) is performing better then dual 7773x’s. But its also possible that bladebit is broken in its current iteration.
I have also run tests on an Intel platform in AWS m6i.metal (2 x 8375C) it did a plot in 202 sec
It works faster on Intels and also there is Arm version of Bladebit. Though I didn’t see any benchmark of it. I know Gigabyte has Arm 2U server, but I bet it’s super rare.
According to this people turn off two CCDs of 64 core Epyc and get better results… EPYC 7002 series optimization issue · Discussion #228 · Chia-Network/bladebit · GitHub
I thought that extra cache would alleviate the issue with with having 8 CCD’s but I was wrong, maybe bladebit is just not taking advantage of that much cache.
Yeah they do have Ampere Altra boards and servers available but these are prohibitively expensive and hard to source, def not as easy as going to ebay and grabbing 2 7773x’s and plopping them in any random Supermicro,Asus,Asrock board you can get your hands on.
It does when you consider that the NUMA hop adds a ~30% performance hit vs. NUMA isolated. In that respect, I’d argue that a second CPU that was only perf scaling by 70% makes no sense.
That’s not a hard thing to do. A single NVMe SSD on each socket works fine, especially for plot output where BB is just laying down a single file sequentially.
You don’t need 25Gb/s to offload completed plots that fast.
1 Like
I thought the same - that’s why I was looking someone to test it in the first place.
Yeah they do have Ampere Altra boards and servers available but these are prohibitively expensive
Not true - the price is more or less like 64 core Epyc
I can run more tests, and spool up the 8380 system as it was really nice.
The 8380 qs cpus pop up on ebay every now and then for around 1500 which is a steal. 8368 is prolly better tho.
My 8380 system is 2 cpus and configured to let the cpus run up to 400w all the time with beefy cooling.
1 Like
512GB of ram costs around 4k and separate system is till faster.Also P series CPUs are significantly cheaper. Dual CPU dual thread systems don’t make sense financially.
So what is the point of dual CPU system if you double everything anyway.
10Gbit moves one plot in ~2 minutes. Dual CPU dual thread sub 4 minutes plotter will generate plots faster that they will be Rsynced out via 10G
That would be great! Also it’s very interesting to see how it scales with dual CPUs on Intels.
Where do you have a link ?
What do you think about 8373C did you have any exposure to it in the wild prices range 1500-2000 which in my mind is a bargain for 36c 300w 3rd gen.
1 Like