Return to Level1Techs.com

The absolute state of programming. And the sed 11q meme

So, from time to time I still go to reedit, and I noticed a user posting their Arch desktop and using cat file | head -n N instead of head -n N file in their shell.

Naturally I wanted to establish dominance point out the unnecessary overhead in the fellow Arch user’s shell usage.

image

I disengaged after the last comment, thinking to myself:

pff, speed and Python lol

But, in the back of my mind I thought. What if I am wrong. Is Python faster than head?

image

Let’s find out!


So I wrote as simple, yet usable, head Python script as I could. Please, tho, if it can be optimized TELL ME. And decided to put it up against head and for memery sed. I used perf stat to gather performance counter information. I ran the same command 100 times, just to be sure.

But, first the python script:

import sys

if len(sys.argv) < 2:
    sys.exit(f"Usage: {sys.argv[0]} file n-lines")

with open(sys.argv[1], "r") as f:
    for i in range(0, int(sys.argv[2])):
        sys.stdout.write(f.readline())

head

 Performance counter stats for 'head -n 5 /home/red/.vimrc' (100 runs):

              0.62 msec task-clock:u              #    0.683 CPUs utilized            ( +-  1.94% )
                 0      context-switches:u        #    0.000 K/sec                  
                 0      cpu-migrations:u          #    0.000 K/sec                  
                72      page-faults:u             #    0.116 M/sec                    ( +-  0.30% )
           381,228      cycles:u                  #    0.615 GHz                      ( +-  2.75% )  (85.93%)
         1,146,039      stalled-cycles-frontend:u #  300.62% frontend cycles idle     ( +-  0.86% )
         1,105,819      stalled-cycles-backend:u  #  290.07% backend cycles idle      ( +-  0.80% )
           362,696      instructions:u            #    0.95  insn per cycle         
                                                  #    3.16  stalled cycles per insn  ( +-  0.01% )
            80,459      branches:u                #  129.899 M/sec                    ( +-  0.00% )
     <not counted>      branch-misses:u                                               ( +- 19.72% )  (14.07%)

         0.0009063 +- 0.0000171 seconds time elapsed  ( +-  1.89% )

0.0009063 seconds time elapsed

sed

 Performance counter stats for 'sed 5q /home/red/.vimrc' (100 runs):

              2.67 msec task-clock:u              #    0.905 CPUs utilized            ( +-  2.37% )
                 0      context-switches:u        #    0.000 K/sec                  
                 0      cpu-migrations:u          #    0.000 K/sec                  
               120      page-faults:u             #    0.045 M/sec                    ( +-  0.15% )
         3,989,722      cycles:u                  #    1.496 GHz                      ( +-  3.78% )  (53.98%)
         3,232,280      stalled-cycles-frontend:u #   81.02% frontend cycles idle     ( +-  1.58% )  (99.20%)
         2,678,686      stalled-cycles-backend:u  #   67.14% backend cycles idle      ( +-  1.33% )
         8,086,924      instructions:u            #    2.03  insn per cycle         
                                                  #    0.40  stalled cycles per insn  ( +-  0.00% )
         1,854,166      branches:u                #  695.078 M/sec                    ( +-  0.02% )
            46,710      branch-misses:u           #    2.52% of all branches          ( +-  4.79% )  (46.82%)

         0.0029479 +- 0.0000682 seconds time elapsed  ( +-  2.31% )

0.0029479 seconds time elapsed

python

 Performance counter stats for 'python head.py /home/red/.vimrc 5' (100 runs):

             14.26 msec task-clock:u              #    0.978 CPUs utilized            ( +-  1.17% )
                 0      context-switches:u        #    0.000 K/sec                  
                 0      cpu-migrations:u          #    0.000 K/sec                  
               812      page-faults:u             #    0.057 M/sec                    ( +-  0.02% )
        37,703,676      cycles:u                  #    2.643 GHz                      ( +-  0.21% )  (78.83%)
        26,023,964      stalled-cycles-frontend:u #   69.02% frontend cycles idle     ( +-  0.39% )  (78.87%)
        20,863,888      stalled-cycles-backend:u  #   55.34% backend cycles idle      ( +-  0.45% )  (59.70%)
        46,541,575      instructions:u            #    1.23  insn per cycle         
                                                  #    0.56  stalled cycles per insn  ( +-  0.24% )  (81.26%)
        10,037,122      branches:u                #  703.695 M/sec                    ( +-  0.15% )  (94.64%)
           439,273      branch-misses:u           #    4.38% of all branches          ( +-  0.23% )  (87.95%)

          0.014579 +- 0.000171 seconds time elapsed  ( +-  1.18% )

0.014579 seconds time elapsed

Results

Execution time:

head sed py
100% 325.27% 1608.63%
0.0009063 s 0.0029479 s 0.014579 s

1608.63%

Also look at the CPU cores utilization.

  • ~300% for head
  • ~70% for sed
  • ~60% for python

head seems to be multi-threaded. Unlike sed. Idk if I can do actual multi-threading in python, coz global interpret lock, or whatever.

Conclusions

  • head is da wei ( ͡° ͜ʖ ͡°)
  • sed is a meme made by ((( cat-v ))) and ((( Luke Smith ))) spergs
  • python is python (also could be that my code shit, U TELL ME)

Addendum

The “lab” environment:

  • cpu: Intel(R) Xeon(R) CPU E31240 @ 3.30GHz
  • kernel: 5.0.6-arch1-1-ARCH
  • head version: head (GNU coreutils) 8.31
  • sed version: sed (GNU sed) 4.7
  • python version: Python 3.7.3

Quirks:

  • head.py is in /tmp, so it’s in RAM

@AnotherDev @MFZuul @SesameStreetThug @whoeverelseinterestedinthis

12 Likes

I just realized how sperg this whole post is.

5 Likes

Only if you went and link said post to said reeditor so you could affirm your dominance.

5 Likes

> buys xeon
> installs arch ftw
> tests head

8 Likes

> arguing on reeddit

that’s a shit tier thing to do

3 Likes

Also, shouldn’t you have tested with sed 6q? I know sed 11q is the alternative to default head which prints 10 lines so I think sed doesn’t print the last line

1 Like

Well then it looks like you’re just testing your theories in arch btw.

your whole desktop is a meme

4 Likes

image

2 Likes

my whole life is a meme

3 Likes

This is one of those things I always knew was true, but now I have the numbers.

4 Likes

You are the meme, you were always the meme. I can’t believe I forgot. I was the one who made navygeks.txt and I forgot you were the meme the whole time.

2 Likes

honkhonk

1 Like

This. Post this reply as you’re taking his girlfriend and sister back to your place.

6 Likes

That guy should thank gek now for all the fractions of a second he will save.

2 Likes

people who don’t care about

ship sub 60 fps games on modern hardware

4 Likes

Not everyone is making games or developing software.

Sure, that’s not the point, the point is that hardware fucking peaked and even hits the limit of current tech, while software stagnated.

Give a programmer better hardware and he’ll write slower code

> Coz it doesn’t matter dude, it’s just a fraction of a second, silicon is CHEAP

2 Likes

Well thats your point. My point was you went full sperg to shave fractions because a guy who clearly doesnt do software development said something that triggered you.

for starters it didn’t trigger me, in fact I got curios and doubted myself about python being slow

what if they’re right?

I tested it, and no, it’s slow.

Why do I care about it? I care about it coz most software I use these days feels sluggish and not responsive coz software is built on top of software, that is built on top of software, that is built on top of software, and so on, and so on.

Each ((( fraction of a second ))) adds up in each loop, in each layer. All that shit adds up to you puking and ditching that VR set, coz the fucking latency is too big.

2 Likes

Python is super fast, its your xeon thats slow.

2 Likes