File: awk.html

package info (click to toggle)
abs-guide 10-4
links: PTS, VCS
area: main
in suites: bookworm, bullseye, forky, sid, trixie
size: 6,952 kB
sloc: sh: 14,129; makefile: 81
file content (587 lines) | stat: -rw-r--r-- 10,541 bytes
parent folder | download | duplicates (3)
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML
><HEAD
><TITLE
>Awk</TITLE
><META
NAME="GENERATOR"
CONTENT="Modular DocBook HTML Stylesheet Version 1.76b+
"><LINK
REL="HOME"
TITLE="Advanced Bash-Scripting Guide"
HREF="index.html"><LINK
REL="UP"
TITLE="A Sed and Awk Micro-Primer"
HREF="sedawk.html"><LINK
REL="PREVIOUS"
TITLE="A Sed and Awk Micro-Primer"
HREF="sedawk.html"><LINK
REL="NEXT"
TITLE="Parsing and Managing Pathnames"
HREF="pathmanagement.html"><META
HTTP-EQUIV="Content-Style-Type"
CONTENT="text/css"><LINK
REL="stylesheet"
HREF="common/kde-common.css"
TYPE="text/css"><META
HTTP-EQUIV="Content-Type"
CONTENT="text/html; charset=iso-8859-1"><META
HTTP-EQUIV="Content-Language"
CONTENT="en"><LINK
REL="stylesheet"
HREF="common/kde-localised.css"
TYPE="text/css"
TITLE="KDE-English"><LINK
REL="stylesheet"
HREF="common/kde-default.css"
TYPE="text/css"
TITLE="KDE-Default"></HEAD
><BODY
CLASS="SECT1"
BGCOLOR="#FFFFFF"
TEXT="#000000"
LINK="#AA0000"
VLINK="#AA0055"
ALINK="#AA0000"
STYLE="font-family: sans-serif;"
><DIV
CLASS="NAVHEADER"
><TABLE
SUMMARY="Header navigation table"
WIDTH="100%"
BORDER="0"
CELLPADDING="0"
CELLSPACING="0"
><TR
><TH
COLSPAN="3"
ALIGN="center"
>Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting</TH
></TR
><TR
><TD
WIDTH="10%"
ALIGN="left"
VALIGN="bottom"
><A
HREF="sedawk.html"
ACCESSKEY="P"
>Prev</A
></TD
><TD
WIDTH="80%"
ALIGN="center"
VALIGN="bottom"
>Appendix C. A Sed and Awk Micro-Primer</TD
><TD
WIDTH="10%"
ALIGN="right"
VALIGN="bottom"
><A
HREF="pathmanagement.html"
ACCESSKEY="N"
>Next</A
></TD
></TR
></TABLE
><HR
ALIGN="LEFT"
WIDTH="100%"></DIV
><DIV
CLASS="SECT1"
><H1
CLASS="SECT1"
><A
NAME="AWK"
></A
>C.2. Awk</H1
><P
><A
NAME="AWKREF"
></A
></P
><P
><I
CLASS="FIRSTTERM"
>Awk</I
>
       <A
NAME="AEN23443"
HREF="#FTN.AEN23443"
>[1]</A
>
       is a full-featured text processing language with a syntax
       reminiscent of <I
CLASS="FIRSTTERM"
>C</I
>.  While it possesses an
       extensive set of operators and capabilities, we will cover only
       a few of these here - the ones most useful in shell scripts.</P
><P
>Awk breaks each line of input passed to it into
       <A
NAME="FIELDREF2"
></A
>
       <A
HREF="special-chars.html#FIELDREF"
>fields</A
>. By default, a field
       is a string of consecutive characters delimited by <A
HREF="special-chars.html#WHITESPACEREF"
>whitespace</A
>, though there are options
       for changing this. Awk parses and operates on each separate
       field. This makes it ideal for handling structured text files
       -- especially tables -- data organized into consistent chunks,
       such as rows and columns.</P
><P
><A
HREF="variables.html#SNGLQUO"
>Strong quoting</A
> and <A
HREF="special-chars.html#CODEBLOCKREF"
>curly brackets</A
> enclose blocks of
       awk code within a shell script.</P
><P
><TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><PRE
CLASS="PROGRAMLISTING"
>   1&nbsp;# $1 is field #1, $2 is field #2, etc.
   2&nbsp;
   3&nbsp;echo one two | awk '{print $1}'
   4&nbsp;# one
   5&nbsp;
   6&nbsp;echo one two | awk '{print $2}'
   7&nbsp;# two
   8&nbsp;
   9&nbsp;# But what is field #0 ($0)?
  10&nbsp;echo one two | awk '{print $0}'
  11&nbsp;# one two
  12&nbsp;# All the fields!
  13&nbsp;
  14&nbsp;
  15&nbsp;awk '{print $3}' $filename
  16&nbsp;# Prints field #3 of file $filename to stdout.
  17&nbsp;
  18&nbsp;awk '{print $1 $5 $6}' $filename
  19&nbsp;# Prints fields #1, #5, and #6 of file $filename.
  20&nbsp;
  21&nbsp;awk '{print $0}' $filename
  22&nbsp;# Prints the entire file!
  23&nbsp;# Same effect as:   cat $filename . . . or . . . sed '' $filename</PRE
></TD
></TR
></TABLE
></P
><P
>We have just seen the awk <I
CLASS="FIRSTTERM"
>print</I
> command
       in action. The only other feature of awk we need to deal with
       here is variables. Awk handles variables similarly to shell
       scripts, though a bit more flexibly.</P
><P
><TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><PRE
CLASS="PROGRAMLISTING"
>   1&nbsp;{ total += ${column_number} }</PRE
></TD
></TR
></TABLE
>
       This adds the value of <TT
CLASS="PARAMETER"
><I
>column_number</I
></TT
> to
       the running total of <TT
CLASS="PARAMETER"
><I
>total</I
></TT
>&#62;. Finally, to print
       <SPAN
CLASS="QUOTE"
>"total"</SPAN
>, there is an <B
CLASS="COMMAND"
>END</B
> command
       block, executed after the script has processed all its input.
       <TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><PRE
CLASS="PROGRAMLISTING"
>   1&nbsp;END { print total }</PRE
></TD
></TR
></TABLE
></P
><P
>Corresponding to the <B
CLASS="COMMAND"
>END</B
>, there is a
       <B
CLASS="COMMAND"
>BEGIN</B
>, for a code block to be performed before awk
       starts processing its input.</P
><P
>The following example illustrates how <B
CLASS="COMMAND"
>awk</B
> can
       add text-parsing tools to a shell script.</P
><DIV
CLASS="EXAMPLE"
><HR><A
NAME="LETTERCOUNT2"
></A
><P
><B
>Example C-1. Counting Letter Occurrences</B
></P
><TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><PRE
CLASS="PROGRAMLISTING"
>   1&nbsp;#! /bin/sh
   2&nbsp;# letter-count2.sh: Counting letter occurrences in a text file.
   3&nbsp;#
   4&nbsp;# Script by nyal [nyal@voila.fr].
   5&nbsp;# Used in ABS Guide with permission.
   6&nbsp;# Recommented and reformatted by ABS Guide author.
   7&nbsp;# Version 1.1: Modified to work with gawk 3.1.3.
   8&nbsp;#              (Will still work with earlier versions.)
   9&nbsp;
  10&nbsp;
  11&nbsp;INIT_TAB_AWK=""
  12&nbsp;# Parameter to initialize awk script.
  13&nbsp;count_case=0
  14&nbsp;FILE_PARSE=$1
  15&nbsp;
  16&nbsp;E_PARAMERR=85
  17&nbsp;
  18&nbsp;usage()
  19&nbsp;{
  20&nbsp;    echo "Usage: letter-count.sh file letters" 2&#62;&#38;1
  21&nbsp;    # For example:   ./letter-count2.sh filename.txt a b c
  22&nbsp;    exit $E_PARAMERR  # Too few arguments passed to script.
  23&nbsp;}
  24&nbsp;
  25&nbsp;if [ ! -f "$1" ] ; then
  26&nbsp;    echo "$1: No such file." 2&#62;&#38;1
  27&nbsp;    usage                 # Print usage message and exit.
  28&nbsp;fi 
  29&nbsp;
  30&nbsp;if [ -z "$2" ] ; then
  31&nbsp;    echo "$2: No letters specified." 2&#62;&#38;1
  32&nbsp;    usage
  33&nbsp;fi 
  34&nbsp;
  35&nbsp;shift                      # Letters specified.
  36&nbsp;for letter in `echo $@`    # For each one . . .
  37&nbsp;  do
  38&nbsp;  INIT_TAB_AWK="$INIT_TAB_AWK tab_search[${count_case}] = \
  39&nbsp;  \"$letter\"; final_tab[${count_case}] = 0; " 
  40&nbsp;  # Pass as parameter to awk script below.
  41&nbsp;  count_case=`expr $count_case + 1`
  42&nbsp;done
  43&nbsp;
  44&nbsp;# DEBUG:
  45&nbsp;# echo $INIT_TAB_AWK;
  46&nbsp;
  47&nbsp;cat $FILE_PARSE |
  48&nbsp;# Pipe the target file to the following awk script.
  49&nbsp;
  50&nbsp;# ---------------------------------------------------------------------
  51&nbsp;# Earlier version of script:
  52&nbsp;# awk -v tab_search=0 -v final_tab=0 -v tab=0 -v \
  53&nbsp;# nb_letter=0 -v chara=0 -v chara2=0 \
  54&nbsp;
  55&nbsp;awk \
  56&nbsp;"BEGIN { $INIT_TAB_AWK } \
  57&nbsp;{ split(\$0, tab, \"\"); \
  58&nbsp;for (chara in tab) \
  59&nbsp;{ for (chara2 in tab_search) \
  60&nbsp;{ if (tab_search[chara2] == tab[chara]) { final_tab[chara2]++ } } } } \
  61&nbsp;END { for (chara in final_tab) \
  62&nbsp;{ print tab_search[chara] \" =&#62; \" final_tab[chara] } }"
  63&nbsp;# ---------------------------------------------------------------------
  64&nbsp;#  Nothing all that complicated, just . . .
  65&nbsp;#+ for-loops, if-tests, and a couple of specialized functions.
  66&nbsp;
  67&nbsp;exit $?
  68&nbsp;
  69&nbsp;# Compare this script to letter-count.sh.</PRE
></TD
></TR
></TABLE
><HR></DIV
><P
>For simpler examples of awk within shell scripts, see:
       <OL
TYPE="1"
><LI
><P
><A
HREF="internal.html#EX44"
>Example 15-14</A
></P
></LI
><LI
><P
><A
HREF="redircb.html#REDIR4"
>Example 20-8</A
></P
></LI
><LI
><P
><A
HREF="filearchiv.html#STRIPC"
>Example 16-32</A
></P
></LI
><LI
><P
><A
HREF="wrapper.html#COLTOTALER"
>Example 36-5</A
></P
></LI
><LI
><P
><A
HREF="ivr.html#COLTOTALER2"
>Example 28-2</A
></P
></LI
><LI
><P
><A
HREF="internal.html#COLTOTALER3"
>Example 15-20</A
></P
></LI
><LI
><P
><A
HREF="procref1.html#PIDID"
>Example 29-3</A
></P
></LI
><LI
><P
><A
HREF="procref1.html#CONSTAT"
>Example 29-4</A
></P
></LI
><LI
><P
><A
HREF="loops.html#FILEINFO"
>Example 11-3</A
></P
></LI
><LI
><P
><A
HREF="extmisc.html#BLOTOUT"
>Example 16-61</A
></P
></LI
><LI
><P
><A
HREF="randomvar.html#SEEDINGRANDOM"
>Example 9-16</A
></P
></LI
><LI
><P
><A
HREF="moreadv.html#IDELETE"
>Example 16-4</A
></P
></LI
><LI
><P
><A
HREF="manipulatingvars.html#SUBSTRINGEX"
>Example 10-6</A
></P
></LI
><LI
><P
><A
HREF="assortedtips.html#SUMPRODUCT"
>Example 36-19</A
></P
></LI
><LI
><P
><A
HREF="loops.html#USERLIST"
>Example 11-9</A
></P
></LI
><LI
><P
><A
HREF="wrapper.html#PRASC"
>Example 36-4</A
></P
></LI
><LI
><P
><A
HREF="mathc.html#HYPOT"
>Example 16-53</A
></P
></LI
><LI
><P
><A
HREF="asciitable.html#ASCII3SH"
>Example T-3</A
></P
></LI
></OL
>
      </P
><P
>That's all the awk we'll cover here, folks, but there's lots
       more to learn. See the appropriate references in the <A
HREF="biblio.html"
><I
>Bibliography</I
></A
>.</P
></DIV
><H3
CLASS="FOOTNOTES"
>Notes</H3
><TABLE
BORDER="0"
CLASS="FOOTNOTES"
WIDTH="100%"
><TR
><TD
ALIGN="LEFT"
VALIGN="TOP"
WIDTH="5%"
><A
NAME="FTN.AEN23443"
HREF="awk.html#AEN23443"
>[1]</A
></TD
><TD
ALIGN="LEFT"
VALIGN="TOP"
WIDTH="95%"
><P
>Its name derives from the initials of its authors,
       <B
CLASS="COMMAND"
>A</B
>ho, <B
CLASS="COMMAND"
>W</B
>einberg, and
       <B
CLASS="COMMAND"
>K</B
>ernighan.</P
></TD
></TR
></TABLE
><DIV
CLASS="NAVFOOTER"
><HR
ALIGN="LEFT"
WIDTH="100%"><TABLE
SUMMARY="Footer navigation table"
WIDTH="100%"
BORDER="0"
CELLPADDING="0"
CELLSPACING="0"
><TR
><TD
WIDTH="33%"
ALIGN="left"
VALIGN="top"
><A
HREF="sedawk.html"
ACCESSKEY="P"
>Prev</A
></TD
><TD
WIDTH="34%"
ALIGN="center"
VALIGN="top"
><A
HREF="index.html"
ACCESSKEY="H"
>Home</A
></TD
><TD
WIDTH="33%"
ALIGN="right"
VALIGN="top"
><A
HREF="pathmanagement.html"
ACCESSKEY="N"
>Next</A
></TD
></TR
><TR
><TD
WIDTH="33%"
ALIGN="left"
VALIGN="top"
>A Sed and Awk Micro-Primer</TD
><TD
WIDTH="34%"
ALIGN="center"
VALIGN="top"
><A
HREF="sedawk.html"
ACCESSKEY="U"
>Up</A
></TD
><TD
WIDTH="33%"
ALIGN="right"
VALIGN="top"
>Parsing and Managing Pathnames</TD
></TR
></TABLE
></DIV
></BODY
></HTML
>