So, from time to time I still go to reedit, and I noticed a user posting their Arch desktop and using cat file | head -n N instead of head -n N file in their shell.
Naturally I wanted to establish dominance point out the unnecessary overhead in the fellow Arch user’s shell usage.
I disengaged after the last comment, thinking to myself:
pff, speed and Python lol
But, in the back of my mind I thought. What if I am wrong. Is Python faster than head?
Let’s find out!
So I wrote as simple, yet usable, head Python script as I could. Please, tho, if it can be optimized TELL ME. And decided to put it up against head and for memery sed. I used perf stat to gather performance counter information. I ran the same command 100 times, just to be sure.
But, first the python script:
import sys
if len(sys.argv) < 2:
sys.exit(f"Usage: {sys.argv[0]} file n-lines")
with open(sys.argv[1], "r") as f:
for i in range(0, int(sys.argv[2])):
sys.stdout.write(f.readline())
head
Performance counter stats for 'head -n 5 /home/red/.vimrc' (100 runs):
0.62 msec task-clock:u # 0.683 CPUs utilized ( +- 1.94% )
0 context-switches:u # 0.000 K/sec
0 cpu-migrations:u # 0.000 K/sec
72 page-faults:u # 0.116 M/sec ( +- 0.30% )
381,228 cycles:u # 0.615 GHz ( +- 2.75% ) (85.93%)
1,146,039 stalled-cycles-frontend:u # 300.62% frontend cycles idle ( +- 0.86% )
1,105,819 stalled-cycles-backend:u # 290.07% backend cycles idle ( +- 0.80% )
362,696 instructions:u # 0.95 insn per cycle
# 3.16 stalled cycles per insn ( +- 0.01% )
80,459 branches:u # 129.899 M/sec ( +- 0.00% )
<not counted> branch-misses:u ( +- 19.72% ) (14.07%)
0.0009063 +- 0.0000171 seconds time elapsed ( +- 1.89% )
0.0009063 seconds time elapsed
sed
Performance counter stats for 'sed 5q /home/red/.vimrc' (100 runs):
2.67 msec task-clock:u # 0.905 CPUs utilized ( +- 2.37% )
0 context-switches:u # 0.000 K/sec
0 cpu-migrations:u # 0.000 K/sec
120 page-faults:u # 0.045 M/sec ( +- 0.15% )
3,989,722 cycles:u # 1.496 GHz ( +- 3.78% ) (53.98%)
3,232,280 stalled-cycles-frontend:u # 81.02% frontend cycles idle ( +- 1.58% ) (99.20%)
2,678,686 stalled-cycles-backend:u # 67.14% backend cycles idle ( +- 1.33% )
8,086,924 instructions:u # 2.03 insn per cycle
# 0.40 stalled cycles per insn ( +- 0.00% )
1,854,166 branches:u # 695.078 M/sec ( +- 0.02% )
46,710 branch-misses:u # 2.52% of all branches ( +- 4.79% ) (46.82%)
0.0029479 +- 0.0000682 seconds time elapsed ( +- 2.31% )
0.0029479 seconds time elapsed
python
Performance counter stats for 'python head.py /home/red/.vimrc 5' (100 runs):
14.26 msec task-clock:u # 0.978 CPUs utilized ( +- 1.17% )
0 context-switches:u # 0.000 K/sec
0 cpu-migrations:u # 0.000 K/sec
812 page-faults:u # 0.057 M/sec ( +- 0.02% )
37,703,676 cycles:u # 2.643 GHz ( +- 0.21% ) (78.83%)
26,023,964 stalled-cycles-frontend:u # 69.02% frontend cycles idle ( +- 0.39% ) (78.87%)
20,863,888 stalled-cycles-backend:u # 55.34% backend cycles idle ( +- 0.45% ) (59.70%)
46,541,575 instructions:u # 1.23 insn per cycle
# 0.56 stalled cycles per insn ( +- 0.24% ) (81.26%)
10,037,122 branches:u # 703.695 M/sec ( +- 0.15% ) (94.64%)
439,273 branch-misses:u # 4.38% of all branches ( +- 0.23% ) (87.95%)
0.014579 +- 0.000171 seconds time elapsed ( +- 1.18% )
0.014579 seconds time elapsed
Results
Execution time:
| head | sed | py |
|---|---|---|
| 100% | 325.27% | 1608.63% |
| 0.0009063 s | 0.0029479 s | 0.014579 s |
1608.63%
Also look at the CPU cores utilization.
- ~300% for head
- ~70% for sed
- ~60% for python
head seems to be multi-threaded. Unlike sed. Idk if I can do actual multi-threading in python, coz global interpret lock, or whatever.
Conclusions
-
headis da wei ( ͡° ͜ʖ ͡°) -
sedis a meme made by ((( cat-v ))) and ((( Luke Smith ))) spergs - python is python (also could be that my code shit, U TELL ME)
Addendum
The “lab” environment:
- cpu:
Intel(R) Xeon(R) CPU E31240 @ 3.30GHz - kernel:
5.0.6-arch1-1-ARCH - head version:
head (GNU coreutils) 8.31 - sed version:
sed (GNU sed) 4.7 - python version:
Python 3.7.3
Quirks:
- head.py is in /tmp, so it’s in RAM
@AnotherDev @MFZuul @SesameStreetThug @whoeverelseinterestedinthis
