Linux AWK command examples

In this article, we will show you some practical examples on how to use AWK on a Linux VPS.

Introduction

AWK is named after the names of its authors: Alfred Aho, Peter Weinberger and Brian Kernighan. AWK is a very useful scripting language for word processing. This language is executed in the interpreter. It allows the user to process some input, define variables, use boolean operators, strings and numeric functions, retrieve data, and generate formatted reports. The AWK syntax is very similar to the C language and is the direct predecessor of Perl. All AWK scripts can be converted to Perl scripts using the A2P utility.

Prerequisites

The AWK interpreter is a standard tool found on every Linux distribution. The gawk package contains an open source version of the AWK, and depending on the Linux distribution, it can be installed from a source file or using the gawk or mawk packages included with the specific Linux distribution.

Installation

Log in to the server via SSH as a superuser

ssh [email protected]_Address

To install the AWK command line utility on CentOS / Fedora or any other RPM based Linux distribution, run the following command:

yum install gawk

On Ubuntu / Debian, you need to invoke this command to install Gawk:

apt-get install gawk

AWK command examples

Simple awk commands can be easily run from the command line, while more complex tasks must be written as awk scripts to a file. Some useful examples of awk commands and executable scripts are listed below.

You can use the AWK command to print only specific columns from the input field. For example, using the command below you can find out the list of IP addresses that are connected to the server:

netstat -anp|grep tcp|awk '{print $5}'| cut -d : -f1 | sort | uniq -c | sort -n

This is very useful if you are investigating whether your server is under a DoS or DDoS attack.

In the following example, we use AWK to search for a specific pattern in specific columns and do some action based on the result:

exim -bpr | grep frozen | awk {'print $3'} | xargs exim -Mrm

The above command will remove all frozen email messages from Exim’s mail queue.

AWK is often used to perform useful and practical text processing and manipulation. For example, we can use AWK to remove duplicates in a text file without sorting:

awk '!x[$0]++' file-with-duplicates > new-file-without-duplicates

The following command will print five random numbers from 0 to 999:

awk 'BEGIN { for (i = 1; i <= 5; i++) print int(1000 * rand()) }'

Use the following command to count the number of lines in the file named “sample_file”:

awk 'END { print NR }' sample_file

The following command will print all lines in sample_file that contain lines beginning with ‘A’ or ‘a’ followed by ‘re’:

awk '/[Aa]re/{print}' /opt/sample_file

You can use AWK command for more complex operations. If your website is rather slow, you can use the following command to check if there is any problem with your disk I / O (and / or network, in some rare cases):

tac /proc/stat | awk '/^btime/ {up=systime()-$2;print "up " up/86400 "d"}; /^cpu / {print "user " $2/up "%, nice " $3/up "%, sys " $4/up "%, idle " $5/up "%, iowait " $6/up "%, steal " $9/up "%niowait/used " $6 / ($2+$3+$4) ", steal/used " $9 / ($2+$3+$4) }'

IOWAIT refers to how long processes are blocked from busy I / O, mostly disk storage or possibly networking. STEAL means how long processes are blocked by CPU Time slice luck on the server. The higher iowait for the user’s cpu time (= USER + NICE + SYSTEM) shows busy I / O, the higher steal viewed shows busy cpu.

The following script uses a simple awk command that searches the input file ‘/ etc / passwd’ and provides output with the username followed by the date and time of the last login:

vi login-check
#!/bin/bash

for user in `awk -F: '{print $1}' /etc/passwd`
do
echo -n "$user: "
finger $user | grep Last
if [ $? != 0 ]; then
echo
fi
done

Make the script executable:

chmod 755 login-check

Execute the script:

./login-check

You should be able to see the user accounts available on the server, and then the date and time of the last login of each user.

Output

There are some new languages ​​like Perl and Python that can be used in place of AWK, but using AWK has a number of advantages as:

  • AWK is very easy to learn.
  • AWK can be used to solve certain types of problems faster and create more efficient scripts than other tools / languages.
  • AWK comes in very handy when dealing with large files like logs etc. because with the AWK command / script you can create a filtered and readable report.
Sidebar