Grep questions

so im starting to learn bash and im looking at some scripts that already exist at my job and im trying to replicate what this script is doing in that it gives some cpu output.

i can obviously look up the tags for top thru “man top” which i will do now but im curious as to exactly how these are working with grep. i can read it, and get that it’s looking for Cpu or average but im curious how i can implement this to show WA and ID (iowait and idle) stats as well.

top -b -d1 -n1 | grep Cpu
top -b -d1 -n1 | grep average

Grep is a powerful tool, but it’s essentially just a filter process. Grep can’t provide you any info that it isn’t passed to it.

What’s happening here is that ‘top’ (a process monitoring tool) is being run with the “-b -d1 -n1” flags, the output is sent (via a process called ‘piping’, which is the name of the | character) to grep, which excludes anything that doesn’t match “Cpu” or “average”.

It should be noted that grep is case-sensitive unless the -i flag is used.

For disk statistics, you’ll need to invoke something that reports the stat you need, like ‘iostat’ or ‘iotop’. Grep will work the same way, regardless of which tool it takes input from.

2 Likes

To build upon @imhigh.today’s answer imagine you asked a friend to read a newspaper line by line and find any line that had the word dog in it and to cut it out and give it to you.

That’s what grep does on a very basic level.

So here’s a real world example to help you understand. I wanted to get a list of every kernel I have used on my Arch Desktop. Arch stores all of the package managers pacman's action in a log file /var/log/pacman.log.

Now, I could open it up in a GUI text editor, run the find function, find every instance of the upgrade, and copy it into a different file. Problem with this is that my pacman.log file is long. Like, 24529 lines long. That would take forever!

Instead let’s use our grep command. Previous knowledge tells me that when I upgrade to a new kernel on Arch, it is called “linux” so let’s start with that.

grep linux /var/log/pacman.conf

Look at the results here.

That’s still really long and includes all sorts of stuff that I don’t want. Let’s refine it a bit more.

Thankfully, we don’t have to look very far. The 3rd line looks like this:

[2017-10-16 07:39] [ALPM] upgraded linux (4.13.5-1 -> 4.13.6-1)

Okay the keywords “upgraded linux” seems like what I want. Because it’s multiple words separated by a space, I need to use quotes or it won’t work the way we expect it too.

grep "upgraded linux" /var/log/pacman.log

Results here.

Well that’s closer, but that is also including the firmware and api headers. There might be a better way to filter this, but if you notice, the lines I want look just like this upgraded linux (... so there is a space after linux while the other lines have a hypen after linux. So let’s just grep that.

grep "upgraded linux " /var/log/pacman.log

Ah! Finally the information we wanted!

Results here.

1 Like

So you want to show iowait and idle stats. As @imhigh.today mentioned, iostat and iotop are probably good programs to start with. I’m more familiar with iotop so let’s use that.

If we run iotop by itself you’ll notice it looks a lot like top. We get a similar interface that behaves in a similar fashion. So let’s try piping (remember, that’s the | character) it into grep and looking for “idle”

# iotop | grep idle

Just a blank line. I guess we’ll need to figure out something else. Let’s check man iotop to see if there’s anything that can help us.

Oh look! There is a -b switch that runs it in batch mode. From previous experience, I know that batch mode usually outputs things in a way that we can manipulate. So let’s try that to see what it does.

# iotop -b

Okay, so it’s printing out information and it just keeps doing it. Really we only want to do this once so we get a current snapshot of the processes. Let’s go back to the man page. (man iotop)

I see -n says “Set the number of iterations before quitting (never quit by default). This is most useful in non-interactive mode.” Which seems like what we want. Let’s test it out.

# iotop -b -n 1

Hey there we go, a stream of information that prints out to our screen. Perfect for grep! Let’s try to grep “idle” again.

# iotop -b -n 1 | grep idle

And there we go, a list of all idle I/O processes.

Hopefully you can use this and find a program that lists whatever stats you’re looking for and use grep to extract the information.