Chia Plotting on Sapphire Rapids (Supermicro X13 BigTwin)

Just for the lulz/giggles @malventano asked “how many plots per day?” looks like 360 with zero tuning right out of the box. Per node. so 720 per 2u server. :smiley:

Bladebit Chia Plotter
Version : 2.0.1-dev
Git Commit : 9fac46aff0476e829d476412de18497a3a2f7ed8
Compiled With: gcc 11.3.0

[Global Plotting Config]
Will create 2 plots.
Thread count : 88
Warm start enabled : true
NUMA disabled : false
CPU affinity disabled : false
Farmer public key : 90cda70cb1c63c183e80bcfe974933a90e5c90ac29ac71e87846a3e494c1dc71b939966a0ede2afd91ccec60720aaac4
Pool contract address : xch16rm6mkk6400083rhnndl3gneyd5guuv7e8fukaqugnndd3vg50hsw4xggk
Output path : /mnt/f

System Memory: 337/503 GiB.
Memory required: 416 GiB.
Warning: Not enough memory available. Buffer allocation may fail.
Allocating buffers.

Generating plot 1 / 2: 76638be84bbc57f1ef44aa15636560fcf1810f7e37bdea1e118b93a692b13e67

Running Phase 1
Generating F1…
Finished F1 generation in 0.99 seconds.
Sorting F1…
Finished F1 sort in 6.09 seconds.
Forward propagating to table 2…
Pairing L/R groups…
Finished pairing L/R groups in 8.1620 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 8.0660 seconds.
Sorting entries…
Finished sorting in 14.64 seconds.
Finished forward propagating table 2 in 31.11 seconds.
Forward propagating to table 3…
Pairing L/R groups…
Finished pairing L/R groups in 7.9990 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 7.9880 seconds.
Sorting entries…
Finished sorting in 12.11 seconds.
Finished forward propagating table 3 in 28.34 seconds.
Forward propagating to table 4…
Pairing L/R groups…
Finished pairing L/R groups in 5.5140 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 4.9180 seconds.
Sorting entries…
Finished sorting in 10.67 seconds.
Finished forward propagating table 4 in 21.33 seconds.
Forward propagating to table 5…
Pairing L/R groups…
Finished pairing L/R groups in 5.4720 seconds. Created 4294916965 pairs.
Average of 236.1379 pairs per group.
Computing Fx…
Finished computing Fx in 4.9020 seconds.
Sorting entries…
Finished sorting in 10.33 seconds.
Finished forward propagating table 5 in 20.94 seconds.
Forward propagating to table 6…
Pairing L/R groups…
Finished pairing L/R groups in 5.4540 seconds. Created 4294926517 pairs.
Average of 236.1384 pairs per group.
Computing Fx…
Finished computing Fx in 4.6980 seconds.
Sorting entries…
Finished sorting in 9.53 seconds.
Finished forward propagating table 6 in 19.91 seconds.
Forward propagating to table 7…
Pairing L/R groups…
Finished pairing L/R groups in 5.4540 seconds. Created 4294854673 pairs.
Average of 236.1344 pairs per group.
Computing Fx…
Finished computing Fx in 4.2730 seconds.
Finished forward propagating table 7 in 9.96 seconds.
Finished Phase 1 in 138.69 seconds.
Running Phase 2
Prunning table 6…
Finished prunning table 6 in 0.22 seconds.
Prunning table 5…
Finished prunning table 5 in 5.73 seconds.
Prunning table 4…
Finished prunning table 4 in 5.54 seconds.
Prunning table 3…
Finished prunning table 3 in 5.28 seconds.
Prunning table 2…
Finished prunning table 2 in 5.27 seconds.
Finished Phase 2 in 22.20 seconds.
Running Phase 3
Compressing tables 1 and 2…
Finished compressing tables 1 and 2 in 13.69 seconds
Table 1 now has 3429400578 / 4294967296 entries ( 79.85% ).
Compressing tables 2 and 3…
Finished compressing tables 2 and 3 in 13.86 seconds
Table 2 now has 3439859599 / 4294967296 entries ( 80.09% ).
Compressing tables 3 and 4…
Finished compressing tables 3 and 4 in 13.75 seconds
Table 3 now has 3466062680 / 4294967296 entries ( 80.70% ).
Compressing tables 4 and 5…
Finished compressing tables 4 and 5 in 13.67 seconds
Table 4 now has 3532934376 / 4294916965 entries ( 82.26% ).
Compressing tables 5 and 6…
Finished compressing tables 5 and 6 in 15.31 seconds
Table 5 now has 3713647068 / 4294926517 entries ( 86.47% ).
Compressing tables 6 and 7…
Finished compressing tables 6 and 7 in 17.96 seconds
Table 6 now has 4294854673 / 4294854673 entries ( 100.00% ).
Finished Phase 3 in 88.24 seconds.
Running Phase 4
Writing P7.
Finished writing P7 in 0.21 seconds.
Writing C1 table.
Finished writing C1 table in 0.00 seconds.
Writing C2 table.
Finished writing C2 table in 0.00 seconds.
Writing C3 table.
Finished writing C3 table in 0.16 seconds.
Finished Phase 4 in 0.37 seconds.
Finished plotting in 249.51 seconds (4.16 minutes).
Generating plot 2 / 2: 90d3d014dd7849b531198eb313e9edbc40449f504785edad04f0408ca18bbb2e

Running Phase 1
Generating F1…
Finished F1 generation in 0.60 seconds.
Sorting F1…
Finished F1 sort in 4.69 seconds.
Forward propagating to table 2…
Pairing L/R groups…
Finished pairing L/R groups in 5.8300 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 4.3350 seconds.
Waiting for last plot to finish being written to disk…

Previous plot /mnt/f/plot-k32-2023-01-13-01-17-76638be84bbc57f1ef44aa15636560fcf1810f7e37bdea1e118b93a692b13e67.plot.tmp finished writing to disk:
Table 1 pointer : 4096 ( 0x0000000000001000 )
Table 2 pointer : 14839541760 ( 0x0000000374817000 )
Table 3 pointer : 28822372352 ( 0x00000006b5f28000 )
Table 4 pointer : 42911719424 ( 0x00000009fdbce000 )
Table 5 pointer : 57272897536 ( 0x0000000d55bb1000 )
Table 6 pointer : 72368664576 ( 0x00000010d9823000 )
Table 7 pointer : 89827008512 ( 0x00000014ea1b6000 )
C1 table pointer : 107543293952 ( 0x000000190a147000 )
C2 table pointer : 107545014272 ( 0x000000190a2eb000 )
C3 table pointer : 107545018368 ( 0x000000190a2ec000 )

Sorting entries…
Finished sorting in 10.07 seconds.
Finished forward propagating table 2 in 20.49 seconds.
Forward propagating to table 3…
Pairing L/R groups…
Finished pairing L/R groups in 5.5600 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 4.4630 seconds.
Sorting entries…
Finished sorting in 10.59 seconds.
Finished forward propagating table 3 in 20.85 seconds.
Forward propagating to table 4…
Pairing L/R groups…
Finished pairing L/R groups in 5.4370 seconds. Created 4294936069 pairs.
Average of 236.1389 pairs per group.
Computing Fx…
Finished computing Fx in 4.8830 seconds.
Sorting entries…
Finished sorting in 10.29 seconds.
Finished forward propagating table 4 in 20.84 seconds.
Forward propagating to table 5…
Pairing L/R groups…
Finished pairing L/R groups in 5.4610 seconds. Created 4294932522 pairs.
Average of 236.1387 pairs per group.
Computing Fx…
Finished computing Fx in 4.8780 seconds.
Sorting entries…
Finished sorting in 10.88 seconds.
Finished forward propagating table 5 in 21.45 seconds.
Forward propagating to table 6…
Pairing L/R groups…
Finished pairing L/R groups in 5.5270 seconds. Created 4294957243 pairs.
Average of 236.1401 pairs per group.
Computing Fx…
Finished computing Fx in 4.7380 seconds.
Sorting entries…
Finished sorting in 10.11 seconds.
Finished forward propagating table 6 in 20.61 seconds.
Forward propagating to table 7…
Pairing L/R groups…
Finished pairing L/R groups in 5.5550 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 4.3050 seconds.
Finished forward propagating table 7 in 10.09 seconds.
Finished Phase 1 in 119.64 seconds.
Running Phase 2
Prunning table 6…
Finished prunning table 6 in 0.22 seconds.
Prunning table 5…
Finished prunning table 5 in 5.83 seconds.
Prunning table 4…
Finished prunning table 4 in 5.42 seconds.
Prunning table 3…
Finished prunning table 3 in 5.44 seconds.
Prunning table 2…
Finished prunning table 2 in 5.35 seconds.
Finished Phase 2 in 22.43 seconds.
Running Phase 3
Compressing tables 1 and 2…
Finished compressing tables 1 and 2 in 13.10 seconds
Table 1 now has 3429392100 / 4294967296 entries ( 79.85% ).
Compressing tables 2 and 3…
Finished compressing tables 2 and 3 in 14.09 seconds
Table 2 now has 3439918620 / 4294967296 entries ( 80.09% ).
Compressing tables 3 and 4…
Finished compressing tables 3 and 4 in 13.81 seconds
Table 3 now has 3466104172 / 4294936069 entries ( 80.70% ).
Compressing tables 4 and 5…
Finished compressing tables 4 and 5 in 14.49 seconds
Table 4 now has 3532992538 / 4294932522 entries ( 82.26% ).
Compressing tables 5 and 6…
Finished compressing tables 5 and 6 in 14.49 seconds
Table 5 now has 3713645789 / 4294957243 entries ( 86.47% ).
Compressing tables 6 and 7…
Finished compressing tables 6 and 7 in 16.61 seconds
Table 6 now has 4294967296 / 4294967296 entries ( 100.00% ).
Finished Phase 3 in 86.59 seconds.
Running Phase 4
Writing P7.
Finished writing P7 in 0.21 seconds.
Writing C1 table.
Finished writing C1 table in 0.00 seconds.
Writing C2 table.
Finished writing C2 table in 0.00 seconds.
Writing C3 table.
Finished writing C3 table in 0.16 seconds.
Finished Phase 4 in 0.37 seconds.
Writing final plot tables to disk

Plot /mnt/f/plot-k32-2023-01-13-01-22-90d3d014dd7849b531198eb313e9edbc40449f504785edad04f0408ca18bbb2e.plot finished writing to disk:
Table 1 pointer : 4096 ( 0x0000000000001000 )
Table 2 pointer : 14839496704 ( 0x000000037480c000 )
Table 3 pointer : 28822568960 ( 0x00000006b5f58000 )
Table 4 pointer : 42912083968 ( 0x00000009fdc27000 )
Table 5 pointer : 57273503744 ( 0x0000000d55c45000 )
Table 6 pointer : 72369262592 ( 0x00000010d98b5000 )
Table 7 pointer : 89828052992 ( 0x00000014ea2b5000 )
C1 table pointer : 107544793088 ( 0x000000190a2b5000 )
C2 table pointer : 107546513408 ( 0x000000190a459000 )
C3 table pointer : 107546517504 ( 0x000000190a45a000 )

Finished writing tables to disk in 9.65 seconds.
Finished plotting in 238.67 seconds (3.98 minutes).

that’s with just 88 threads.

let’s do 176 threads! (both sockets)

Bladebit Chia Plotter
Version : 2.0.1-dev
Git Commit : 9fac46aff0476e829d476412de18497a3a2f7ed8
Compiled With: gcc 11.3.0

[Global Plotting Config]
Will create 2 plots.
Thread count : 176
Warm start enabled : true
NUMA disabled : false
CPU affinity disabled : false
Farmer public key : 90cda70cb1c63c183e80bcfe974933a90e5c90ac29ac71e87846a3e494c1dc71b939966a0ede2afd91ccec60720aaac4
Pool contract address : xch16rm6mkk6400083rhnndl3gneyd5guuv7e8fukaqugnndd3vg50hsw4xggk
Output path : /mnt/f

System Memory: 497/503 GiB.
Memory required: 416 GiB.
Allocating buffers.

Generating plot 1 / 2: 75f1d8782ffc0487f490b35a6209417cfd1b115571f6cb56667f5436907726c7

Running Phase 1
Generating F1…
Finished F1 generation in 0.44 seconds.
Sorting F1…
Finished F1 sort in 2.56 seconds.
Forward propagating to table 2…
Pairing L/R groups…
Finished pairing L/R groups in 4.7510 seconds. Created 4294883580 pairs.
Average of 236.1360 pairs per group.
Computing Fx…
Finished computing Fx in 2.7990 seconds.
Sorting entries…
Finished sorting in 5.98 seconds.
Finished forward propagating table 2 in 13.66 seconds.
Forward propagating to table 3…
Pairing L/R groups…
Finished pairing L/R groups in 4.6730 seconds. Created 4294721574 pairs.
Average of 236.1271 pairs per group.
Computing Fx…
Finished computing Fx in 3.0130 seconds.
Sorting entries…
Finished sorting in 6.46 seconds.
Finished forward propagating table 3 in 14.27 seconds.
Forward propagating to table 4…
Pairing L/R groups…
Finished pairing L/R groups in 4.7690 seconds. Created 4294486209 pairs.
Average of 236.1142 pairs per group.
Computing Fx…
Finished computing Fx in 3.3850 seconds.
Sorting entries…
Finished sorting in 6.42 seconds.
Finished forward propagating table 4 in 14.75 seconds.
Forward propagating to table 5…
Pairing L/R groups…
Finished pairing L/R groups in 4.6660 seconds. Created 4294156584 pairs.
Average of 236.0961 pairs per group.
Computing Fx…
Finished computing Fx in 3.2880 seconds.
Sorting entries…
Finished sorting in 6.42 seconds.
Finished forward propagating table 5 in 14.50 seconds.
Forward propagating to table 6…
Pairing L/R groups…
Finished pairing L/R groups in 4.6840 seconds. Created 4293401058 pairs.
Average of 236.0545 pairs per group.
Computing Fx…
Finished computing Fx in 2.9920 seconds.
Sorting entries…
Finished sorting in 6.01 seconds.
Finished forward propagating table 6 in 13.81 seconds.
Forward propagating to table 7…
Pairing L/R groups…
Finished pairing L/R groups in 4.7210 seconds. Created 4291939831 pairs.
Average of 235.9742 pairs per group.
Computing Fx…
Finished computing Fx in 2.9210 seconds.
Finished forward propagating table 7 in 7.77 seconds.
Finished Phase 1 in 81.78 seconds.
Running Phase 2
Prunning table 6…
Finished prunning table 6 in 0.12 seconds.
Prunning table 5…
Finished prunning table 5 in 2.96 seconds.
Prunning table 4…
Finished prunning table 4 in 2.82 seconds.
Prunning table 3…
Finished prunning table 3 in 2.77 seconds.
Prunning table 2…
Finished prunning table 2 in 2.75 seconds.
Finished Phase 2 in 11.54 seconds.
Running Phase 3
Compressing tables 1 and 2…
Finished compressing tables 1 and 2 in 8.96 seconds
Table 1 now has 3429226582 / 4294883580 entries ( 79.84% ).
Compressing tables 2 and 3…
Finished compressing tables 2 and 3 in 9.58 seconds
Table 2 now has 3439540511 / 4294721574 entries ( 80.09% ).
Compressing tables 3 and 4…
Finished compressing tables 3 and 4 in 9.57 seconds
Table 3 now has 3465514896 / 4294486209 entries ( 80.70% ).
Compressing tables 4 and 5…
Finished compressing tables 4 and 5 in 9.95 seconds
Table 4 now has 3531997597 / 4294156584 entries ( 82.25% ).
Compressing tables 5 and 6…
Finished compressing tables 5 and 6 in 10.25 seconds
Table 5 now has 3711978945 / 4293401058 entries ( 86.46% ).
Compressing tables 6 and 7…
Finished compressing tables 6 and 7 in 10.71 seconds
Table 6 now has 4291939831 / 4291939831 entries ( 100.00% ).
Finished Phase 3 in 59.02 seconds.
Running Phase 4
Writing P7.
Finished writing P7 in 0.18 seconds.
Writing C1 table.
Finished writing C1 table in 0.00 seconds.
Writing C2 table.
Finished writing C2 table in 0.00 seconds.
Writing C3 table.
Finished writing C3 table in 0.15 seconds.
Finished Phase 4 in 0.33 seconds.
Finished plotting in 152.68 seconds (2.54 minutes).
Generating plot 2 / 2: a60b5e4c165c3c5d32a0c92f0fab5ace000db609d555180ecbf9ddccc1d4931b

Generating plot 2 / 2: a60b5e4c165c3c5d32a0c92f0fab5ace000db609d555180ecbf9ddccc1d4931b

Running Phase 1
Generating F1…
Finished F1 generation in 0.48 seconds.
Sorting F1…
Finished F1 sort in 3.33 seconds.
Forward propagating to table 2…
Pairing L/R groups…
Finished pairing L/R groups in 4.8770 seconds. Created 4294942369 pairs.
Average of 236.1393 pairs per group.
Computing Fx…
Finished computing Fx in 3.0240 seconds.
Waiting for last plot to finish being written to disk…

Previous plot /mnt/f/plot-k32-2023-01-13-02-10-75f1d8782ffc0487f490b35a6209417cfd1b115571f6cb56667f5436907726c7.plot.tmp finished writing to disk:
Table 1 pointer : 4096 ( 0x0000000000001000 )
Table 2 pointer : 14838788096 ( 0x000000037475f000 )
Table 3 pointer : 28820328448 ( 0x00000006b5d35000 )
Table 4 pointer : 42907447296 ( 0x00000009fd7bb000 )
Table 5 pointer : 57264820224 ( 0x0000000d553fd000 )
Table 6 pointer : 72353800192 ( 0x00000010d89f6000 )
Table 7 pointer : 89800290304 ( 0x00000014e883b000 )
C1 table pointer : 107504545792 ( 0x0000001907c53000 )
C2 table pointer : 107506266112 ( 0x0000001907df7000 )
C3 table pointer : 107506270208 ( 0x0000001907df8000 )

Sorting entries…
Finished sorting in 5.97 seconds.
Finished forward propagating table 2 in 14.05 seconds.
Forward propagating to table 3…
Pairing L/R groups…
Finished pairing L/R groups in 4.6450 seconds. Created 4294967296 pairs.
Average of 236.1406 pairs per group.
Computing Fx…
Finished computing Fx in 3.0360 seconds.
Sorting entries…
Finished sorting in 6.45 seconds.
Finished forward propagating table 3 in 14.26 seconds.
Forward propagating to table 4…
Pairing L/R groups…
Finished pairing L/R groups in 4.6820 seconds. Created 4294939320 pairs.
Average of 236.1391 pairs per group.
Computing Fx…
Finished computing Fx in 3.3000 seconds.
Sorting entries…
Finished sorting in 6.43 seconds.
Finished forward propagating table 4 in 14.54 seconds.
Forward propagating to table 5…
Pairing L/R groups…
Finished pairing L/R groups in 4.7500 seconds. Created 4294911176 pairs.
Average of 236.1375 pairs per group.
Computing Fx…
Finished computing Fx in 3.3760 seconds.
Sorting entries…
Finished sorting in 6.44 seconds.
Finished forward propagating table 5 in 14.73 seconds.
Forward propagating to table 6…
Pairing L/R groups…
Finished pairing L/R groups in 4.7240 seconds. Created 4294795814 pairs.
Average of 236.1312 pairs per group.
Computing Fx…
Finished computing Fx in 2.9870 seconds.
Sorting entries…
Finished sorting in 5.99 seconds.
Finished forward propagating table 6 in 13.84 seconds.
Forward propagating to table 7…
Pairing L/R groups…
Finished pairing L/R groups in 4.6990 seconds. Created 4294652648 pairs.
Average of 236.1233 pairs per group.
Computing Fx…
Finished computing Fx in 2.9260 seconds.
Finished forward propagating table 7 in 7.75 seconds.
Finished Phase 1 in 82.99 seconds.
Running Phase 2
Prunning table 6…
Finished prunning table 6 in 0.12 seconds.
Prunning table 5…
Finished prunning table 5 in 2.96 seconds.
Prunning table 4…
Finished prunning table 4 in 2.82 seconds.
Prunning table 3…
Finished prunning table 3 in 2.77 seconds.
Prunning table 2…
Finished prunning table 2 in 2.75 seconds.
Finished Phase 2 in 11.55 seconds.
Running Phase 3
Compressing tables 1 and 2…
Finished compressing tables 1 and 2 in 8.99 seconds
Table 1 now has 3429382397 / 4294942369 entries ( 79.85% ).
Compressing tables 2 and 3…
Finished compressing tables 2 and 3 in 9.93 seconds
Table 2 now has 3439892749 / 4294967296 entries ( 80.09% ).
Compressing tables 3 and 4…
Finished compressing tables 3 and 4 in 9.82 seconds
Table 3 now has 3466083419 / 4294939320 entries ( 80.70% ).
Compressing tables 4 and 5…
Finished compressing tables 4 and 5 in 9.77 seconds
Table 4 now has 3532937771 / 4294911176 entries ( 82.26% ).
Compressing tables 5 and 6…
Finished compressing tables 5 and 6 in 10.47 seconds
Table 5 now has 3713522875 / 4294795814 entries ( 86.47% ).
Compressing tables 6 and 7…
Finished compressing tables 6 and 7 in 10.72 seconds
Table 6 now has 4294652648 / 4294652648 entries ( 100.00% ).
Finished Phase 3 in 59.70 seconds.
Running Phase 4
Writing P7.
Finished writing P7 in 0.18 seconds.
Writing C1 table.
Finished writing C1 table in 0.00 seconds.
Writing C2 table.
Finished writing C2 table in 0.00 seconds.
Writing C3 table.
Finished writing C3 table in 0.12 seconds.
Finished Phase 4 in 0.30 seconds.
Writing final plot tables to disk

Plot /mnt/f/plot-k32-2023-01-13-02-12-a60b5e4c165c3c5d32a0c92f0fab5ace000db609d555180ecbf9ddccc1d4931b.plot finished writing to disk:
Table 1 pointer : 4096 ( 0x0000000000001000 )
Table 2 pointer : 14839459840 ( 0x0000000374803000 )
Table 3 pointer : 28822429696 ( 0x00000006b5f36000 )
Table 4 pointer : 42911862784 ( 0x00000009fdbf1000 )
Table 5 pointer : 57273057280 ( 0x0000000d55bd8000 )
Table 6 pointer : 72368316416 ( 0x00000010d97ce000 )
Table 7 pointer : 89825837056 ( 0x00000014ea098000 )
C1 table pointer : 107541286912 ( 0x0000001909f5d000 )
C2 table pointer : 107543007232 ( 0x000000190a101000 )
C3 table pointer : 107543011328 ( 0x000000190a102000 )

Finished writing tables to disk in 9.49 seconds.
Finished plotting in 164.03 seconds (2.73 minutes).

2 Likes

That’s the fastest plot creation time anyone has seen to date. It shattered my previous record on dual 8368Q’s:

Finished plotting in 185.26 seconds (3.09 minutes).

Nothing from AMD had beaten that Ice Lake time to date. I’d imagine it is going to be even harder with that Sapphire Rapids result. The workload is doing heavy in-memory sorting of ~416GB of data, so it stresses both compute and memory simultaneously, and AMD doesn’t seem to have fully caught up to Intel there despite the higher core counts.