Shell script to check top memory & cpu consuming process in Linux

Earlier I had written an article with the commands to check memory usage per process in Linux. Now what if you wish to check top cpu and memory utilization process, so in this article we will go one step ahead and write a shell script to check top CPU consuming process and top Memory consuming process in Linux

 

Tools to check top CPU consuming process

If you just wish to monitor top CPU consuming process then there are various utilities such as top, vmstat, sar which can help you check high cpu usage process and  give you runtime CPU utilization status in detail.

Advertisement

 

top - check high CPU utilization process

top is a very useful tool for system administrators to monitor system resources. To check top CPU consuming process execute top and then press "shift + p" which will sort and list the high CPU usage process in Linux. As you see here currently

top - 20:32:26 up  9:31,  3 users,  load average: 0.60, 0.17, 0.30
Tasks: 120 total,   3 running, 117 sleeping,   0 stopped,   0 zombie
%Cpu(s): 27.4 us, 33.6 sy,  0.0 ni, 23.4 id, 13.7 wa,  0.0 hi,  2.0 si,  0.0 st
KiB Mem :  3880016 total,  2932020 free,   399832 used,   548164 buff/cache
KiB Swap:  1048572 total,  1048572 free,        0 used.  3268824 avail Mem
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
13681 root      20   0    7312    100      0 R  99.7  0.0   0:06.42 stress
13683 root      20   0  269460 128468    124 R  99.3  3.3   0:06.41 stress
13684 root      20   0  108000    616    520 R  51.0  0.0   0:01.54 dd
  427 root       0 -20       0      0      0 S   2.0  0.0   0:00.81 kworker/0:1H

As you see currently stress and dd are our top CPU consuming process

 

sar- check CPU usage

sar is another powerful monitoring tool which is used by most professionals (my personal favourite). I have written another article with all the system resources you can monitor using sar.

To check CPU usage using sar you can use -u. For example to check CPU usage for two intervals with a gap of 1 second

# sar -u 1 2
Linux 3.10.0-1062.4.1.el7.x86_64 (server1.example.com)  01/10/2020      _x86_64_        (4 CPU)

08:45:20 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
08:45:21 PM     all     26.87      0.00     36.43     12.92      0.00     23.77
08:45:22 PM     all     27.46      0.00     35.49     13.99      0.00     23.06
Average:        all     27.17      0.00     35.96     13.45      0.00     23.42

Similarly you can use -P to check CPU usage per core. For example to check CPU usage per core for 2 interval with 1 second gap use below command. As you see CPU0 is most used while CPU 1-3 are mostly idle. You can also assign a service to a specific core or distribute CPU and other resources to services by creating cgroups

Advertisement
# sar -P ALL 1 2
Linux 3.10.0-1062.4.1.el7.x86_64 (server1.example.com)  01/10/2020      _x86_64_        (4 CPU)

08:54:56 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
08:54:57 PM     all     27.32      0.00     35.31     14.43      0.00     22.94
08:54:57 PM       0      0.00      0.00      7.37      0.00      0.00     92.63
08:54:57 PM       1      1.09      0.00     39.13     59.78      0.00      0.00
08:54:57 PM       2      5.05      0.00     94.95      0.00      0.00      0.00
08:54:57 PM       3    100.00      0.00      0.00      0.00      0.00      0.00

08:54:57 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
08:54:58 PM     all     26.99      0.00     36.76     13.62      0.00     22.62
08:54:58 PM       0      0.00      0.00      9.28      0.00      0.00     90.72
08:54:58 PM       1      0.00      0.00     42.39     57.61      0.00      0.00
08:54:58 PM       2      6.93      0.00     93.07      0.00      0.00      0.00
08:54:58 PM       3    100.00      0.00      0.00      0.00      0.00      0.00

Average:        CPU     %user     %nice   %system   %iowait    %steal     %idle
Average:        all     27.16      0.00     36.04     14.03      0.00     22.78
Average:          0      0.00      0.00      8.33      0.00      0.00     91.67
Average:          1      0.54      0.00     40.76     58.70      0.00      0.00
Average:          2      6.00      0.00     94.00      0.00      0.00      0.00
Average:          3    100.00      0.00      0.00      0.00      0.00      0.00

To check CPU usage per CPU core you can use sar -P <CPU Core> for example to check CPU0 use sar -P 0

 

Tools to check top Memory consuming process

Similar to the above chapter, same tools can also be used to monitor top memory consuming process. I will give some examples using top and sar commands

 

top - check high memory utilization process

You can execute top without any arguments on the terminal and then press "shift + m" to sort and print current high memory utilization process in Linux.

top - 21:05:23 up 10:04,  3 users,  load average: 3.13, 3.05, 2.73
Tasks: 121 total,   5 running, 116 sleeping,   0 stopped,   0 zombie
%Cpu(s): 27.5 us, 34.1 sy,  0.0 ni, 22.8 id, 14.0 wa,  0.0 hi,  1.6 si,  0.0 st
KiB Mem :  3880016 total,  2457248 free,   178352 used,  1244416 buff/cache
KiB Swap:  1048572 total,  1048572 free,        0 used.  3462736 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
13683 root      20   0  269460  54740    124 R 100.0  1.4  30:47.68 stress
  857 root      20   0  358748  29472   7048 S   0.0  0.8   0:00.63 firewalld
 1193 root      20   0  574200  17512   6136 S   0.0  0.5   0:05.59 tuned
  841 polkitd   20   0  613016  12952   4892 S   0.0  0.3   0:00.07 polkitd

Here currently as you see the memory usage is not very high and only few process are consuming memory where stress is the leading scorer.

 

sar - check memory usage

with sar use -r to check memory usage in Linux. For example here I will print memory usage for 2 intervals with 1 second gap.

# sar -r 1 2
Linux 3.10.0-1062.4.1.el7.x86_64 (server1.example.com)  01/10/2020      _x86_64_        (4 CPU)

09:11:20 PM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
09:11:21 PM   2274656   1605360     41.38     39156   1150276    552848     11.22   1202516    234216         0
09:11:22 PM   2337276   1542740     39.76     39156   1150276    552848     11.22   1155908    234216         0
Average:      2305966   1574050     40.57     39156   1150276    552848     11.22   1179212    234216         0

 

There are many other tools such as vmstat, collectl, nmon etc. But since our main intention of this article is to check top CPU and memory consuming process using shell script, we will jump to our sample shell script.

 

Command to check top CPU consuming process

We will use ps to check top CPU consuming process in Linux.

# ps -eocomm,pcpu | egrep -v '(0.0)|(%CPU)'
kworker/0:1H     0.3
stress          99.9
stress          46.7
stress          99.9
kworker/u8:2     0.3
kworker/u8:1     0.3

We can utilise this command in a shell script to check top CPU consuming process.

Command to check top Memory consuming process

Here also we will use ps command to check top memory consuming process with a little change compared to above command

# ps -eocomm,pmem | egrep -v '(0.0)|(%MEM)'
systemd          0.1
polkitd          0.1
NetworkManager   0.1
tuned            0.2
java            82.1
metricbeat       1.4
rsyslogd         0.1
iscsid           0.2

Here,

  • The -e option specifies to collect data on all processes, not just this session's tasks.
  • The -o option specifies an output format.
  • The comm, pcpu and pmem words specify reporting the command name, percentage of CPU and percentage of memory respectively.
  • This ps command generates a line with the command name and current percentage of CPU and memory usage for each running process.
  • These lines are filtered with grep to remove lines where there was no CPU usage (%CPU is 0.0) with the COMMAND %CPU header and no MEM usage (%MEM is 0.0) with the COMMAND %MEM header.

 

Shell script to check top CPU consuming process for an hour

Now we can utilise this command in a shell script to get top memory and cpu consuming process in Linux. The below sample script collects and prints top cpu consuming process for an hour.
I have added a trap for Ctrl + C so that if end user does not wish to wait for an hour and wants to check high cpu utilization process then he can press Ctrl + C to send an interrupt.

#!/bin/bash
#Name: monitor_cpu_usage.sh
#Description: Script to check top cpu consuming process for 1 hour

#Change the SECS to total seconds to monitor CPU usage.
#UNIT_TIME is the interval in seconds between each sampling


function report_utilisation {
  # Process collected data
  echo
  echo "CPU eaters :"

  cat /tmp/cpu_usage.$$ | 
awk '
{ process[$1]+=$2; }
END{
  for(i in process)
  {
    printf("%-20s %s\n",i, process[i]) ;
  }

   }' | sort -nrk 2 | head

#Remove the temporary log file
rm /tmp/cpu_usage.$$
exit 0
}

trap 'report_utilisation' INT

SECS=3600
UNIT_TIME=10

STEPS=$(( $SECS / $UNIT_TIME ))

echo "Watching CPU usage... ;"

# Collect data in temp file
for((i=0;i<$STEPS;i++)); do
    ps -eocomm,pcpu | egrep -v '(0.0)|(%CPU)' >> /tmp/cpu_usage.$$
    sleep $UNIT_TIME
done

report_utilisation

Provide executable permission to the script

# chmod u+x /tmp/monitor_cpu_usage.sh

Now execute the script. This gives us a cumulative value of the overall CPU usage for the time the script was running

# /tmp/monitor_cpu_usage.sh
Watching CPU usage...
^C
CPU eaters :
stress               235.1
java                 15.7
metricbeat           3.1
sshd                 0.4
systemd-journal      0.1
systemd              0.1
rcu_sched            0.1
kworker/u8:0         0.1
kworker/0:1H         0.1

 

Shell script to check top CPU consuming process for an hour

We can use the same script by modifying certain variables and commands to check top memory consuming process:

# cat /tmp/monitor_mem_usage.sh
#!/bin/bash
#Name: monitor_mem_usage.sh
#Description: Script to check top memory consuming process for 1 hour

#Change the SECS to total seconds to monitor CPU usage.
#UNIT_TIME is the interval in seconds between each sampling


function report_utilisation {
  # Process collected data
  echo
  echo "Memory eaters :"

  cat /tmp/mem_usage.$$ | 
awk '
{ process[$1]+=$2; }
END{
  for(i in process)
  {
    printf("%-20s %s\n",i, process[i]) ;
  }

   }' | sort -nrk 2 | head

#Remove the temporary log file
rm /tmp/mem_usage.$$
exit 0
}

trap 'report_utilisation' INT

SECS=3600
UNIT_TIME=10

STEPS=$(( $SECS / $UNIT_TIME ))

echo "Watching Memory usage... ;"

# Collect data in temp file
for((i=0;i<$STEPS;i++)); do 
    ps -eocomm,pmem | egrep -v '(0.0)|(%MEM)' >> /tmp/mem_usage.$$
    sleep $UNIT_TIME
done

report_utilisation

Provide executable permission to the script

# chmod u+x /tmp/monitor_mem_usage.sh

Now we will run the script for few minutes and then press ctrl + C to send an interrupt

# /tmp/monitor_mem_usage.sh
Watching Memory usage...
^C
Memory eaters :
java                 82.1
stress               1.6
metricbeat           1.5
tuned                0.2
iscsid               0.2
systemd              0.1
rsyslogd             0.1
polkitd              0.1
NetworkManager       0.1

 

How this script works?

  • Based on UNIT_TIME variable, the ps -eocomm,pcpu or ps -eocomm,pmem command generates a report on the system activity at that time and stores in the temporary file
  • The temporary file is named /tmp/cpu_usage.$$. Here, $$ is a script variable that holds the process ID (PID) of the current script. For example, if the script's PID is 1345, the temporary file will be named /tmp/cpu_usage.1345
  • The statistics file will be ready after one hour and will contain 60 sets of entries, corresponding to the system status at each minute.
  • The awk script sums the total CPU and memory usage for each process into an associative array named process. This array uses the process name as array index.
  • Finally, awk sorts the result with a numeric reverse sort according to the total CPU and Memory usage and uses head to limit the report to the top 10 usage entries.

 

Related Searches: linux command to find top 5 memory consuming process. how to check high cpu usage process in linux. top 10 memory utilization process in linux. how to check top memory consuming process in linux. top memory usage process in linux. monitor process cpu usage linux. top cpu consuming process in linux. how to check high memory usage process in linux. command to check high cpu utilization in linux. linux cpu usage.

Didn't find what you were looking for? Perform a quick search across GoLinuxCloud

If my articles on GoLinuxCloud has helped you, kindly consider buying me a coffee as a token of appreciation.

Buy GoLinuxCloud a Coffee

For any other feedbacks or questions you can either use the comments section or contact me form.

Thank You for your support!!

14 thoughts on “Shell script to check top memory & cpu consuming process in Linux”

  1. Hello, thank for these two good scripts.

    This line needs a backslash added to it:

    printf("%-20s %sn",i, process[i]) 

    to look like this

    printf("%-20s %s\n",i, process[i])
    Reply
    • Thank you for highlighting this, I use HTML page to write the code so while converting to visual, the backslash was getting removed (no idea why). Have fixed it now.

      Reply
  2. Superb scripts to use. thanks

    Can we make a script output every hourly based in horizontal format ?

    Watching Memory usage... ;
    ^C
    Memory eaters :  
    sshd                 0.6               0.9
    polkitd              0.6              1.2
    NetworkManager       0.4    5.6
    systemd-journal      0.2        3.0
    Reply
    • Thanks for your feedback. You can add a higher sleep value based on your requirement. The timer in the for loop needs to be adjusted accordingly. Although if you have such hourly job then it would be better to use cronjob and just write a script to collect the CPU and Memory usage.

      Reply
      • Thanks for the response, i can run hourly in cronjob but need the value to be in horizontal format as shown above. is that possible and for that what i need to do please help.

        Intension to show which service through out the day consuming i need to give report to management

        Reply
        • It will be a tricky implementation because I don't do any operation on the process name here. You would have to sort the result based on process name i.e. assuming at 9 AM java was consuming high CPU but at 11AM they are consuming very less CPU so you have to map both and then place them in the same line.

          You could store the list of process with their CPU usage and at the next hour match the process name and append the CPU usage.

          Reply
  3. yes this would be wise You could store the list of process with their CPU usage and at the next hour match the process name and append the CPU usage. Can I merge 2 output and get consolidate report for the day

    actually i don't mind consumption high or low just i need to know how was process in hour interval when it got peak & low through a day.

    Could you please help to achieve this , I am struggling for quite long

    Thanks

    Reply
    • Execute the script via cron job.
      You can modify the sleep timer to execute the script for a shorter time, for example 30 seconds and then store the output in a file suppose file1.
      Then after an hour cron job will re-trigger the script, and this time the output must be saved to a different file, suppose file2 (You can achieve this by using mktemp to create a new file for each iteration
      Use printf("%-20s %.1f\n",i, process[i]) ; in the script to have the output in decimal format as it makes easier to have tab separated values.
      In the end you can sort the files using awk

      For example here I have two files:

      ~]# cat /tmp/file
      dockerd              26.9
      httpd                3.4
      containerd           13.9
      libvirtd             33.09
      sssd_nss             14.05
      firewalld            9.17
      tuned                122.48
      iscsid               72.19
      polkitd              71.45
      systemd              11.27
      
       ~]# cat /tmp/file1
      dockerd              6.9
      httpd                3.9
      containerd           3.9
      libvirtd             3.0
      sssd_nss             2.7
      firewalld            2.7
      tuned                2.4
      iscsid               2.1
      polkitd              1.5
      systemd              1.2

      Then I use awk to sort the file based on first column and then arrange the remaining columns:

      ~]# awk 'NR==FNR{a[$1]=$2;next}{print $0"\t",a[$1]}' /tmp/file1 /tmp/file
      dockerd              26.9        6.9
      httpd                3.4         3.9
      containerd           13.9        3.9
      libvirtd             33.09       3.0
      sssd_nss             14.05       2.7
      firewalld            9.17        2.7
      tuned                122.48      2.4
      iscsid               72.19       2.1
      polkitd              71.45       1.5
      systemd              11.27       1.2

      Although creating so many files per hour may not be a good idea. You may save the output in a single file with --START-- and --END-- line which can help you cut and sort each section.

      Reply
  4. Hi Admin ,

    I have this files output which i am trying to get but its not working as excepting. could you please help to get

    vi file1.txt
    Encl   Drawer   Sensor Name                                     Value    Status
    -------------------------------------------------------------------------------
    0      N/A      CPU Temperature-Ctlr B                          64 C     OK    
    0      N/A      Capacitor Pack Temperature-Ctlr B               41 C    OK    
    0      N/A      Expander Temperature-Ctlr B                     66 C     OK    
    0      N/A      Disk Controller Temperature-Ctlr B              60 C     OK    
    0      N/A      Host Left IOC Temperature-Ctlr B                51 C     OK    
    0      N/A      CPU Temperature-Ctlr A                          61 C     OK    
    
    vi file
    
    Encl   Drawer   Sensor Name                                     Value    Status		
    ----------------------------------------------------------------------------------
    0      N/A      CPU Temperature-Ctlr B                          84 C     OK       
    0      N/A      Capacitor Pack Temperature-Ctlr B               81 C    OK       
    0      N/A      Expander Temperature-Ctlr B                     86 C     OK       
    0      N/A      Disk Controller Temperature-Ctlr B              80 C     OK       
    0      N/A      Host Left IOC Temperature-Ctlr B                81 C     OK       
    0      N/A      CPU Temperature-Ctlr A                          81 C     OK       

    3 columns are static data I need 4 & 5 column to be added ahead

    Reply
    • This will be complicated as your each column contains whitespace so awk or cut would treat them as an individual column. For example here is a sorted output of one of your files

      Encl    Drawer  Sensor          Name            Value   Status
      0       N/A     CPU             Temperature-Ctlr B 64 C OK
      0       N/A     Capacitor Pack  Temperature-Ctlr B 41 C OK
      0       N/A     Expander        Temperature-Ctlr B 66 C OK
      0       N/A     Disk Controller Temperature-Ctlr B 60 C OK
      0       N/A     Host Left IOC   Temperature-Ctlr B 51 C OK
      0       N/A     CPU             Temperature-Ctlr A 61 C OK

      But your column 3 contains "Disk Controller" so Disk would be added in 3rd column while Controller will be considered for 4th column. The same goes for your "Value field". I would suggest to use CSV file instead of a text file wherein csv can handle the columns using semicolon which can make it easier to process the data. Also you should avoid using underlines after the heading or else you would have to addd extra handling to remove and then re-add that later.

      Reply
      • Thanks a lot for your swift response . indeed i was facing those challenges what you said now what i was trying to do with the column sensor name , value and status excluding 1st 2 columns & underline as well.

        Really appreciate your response, no one have yet comeup with the solution to required output. would really help if you guide me with awk or sort to combine also will it be feasible to check multiple hourly comparison

        Thanks looking ahead for support

        Reply

Leave a Comment