Shell Features

Introduction

This document provides a more detailed introduction to the Linux/Unix shell, in particular csh and tcsh.

Piping and redirection

Linux is very flexible with regard to where input to commands comes from and where the output goes to.  By default the standard input is the keyboard and the standard output is the terminal.  For example, if you type

date

into an xterm then the output of this command (the date and time) will be sent to the standard output i.e. the xterm.  We can use the output redirection operator > to send this output to a file i.e.

date > datefile

If you now type

more datefile

then the file will be displayed and should contain the output of the date command.

Input redirection can be used in a similar way.  To demonstrate this we'll use the wc program which count lines and characters.  Unless you tell it otherwise wc takes its input from the standard input (the keyboard).  Try the following

wc < datefile

The results of the word count should be displayed on the screen.  A good way to think about what is happening is that the input redirection is effectively typing the contents of the file on the keyboard and the wc command doesn't know any different.  You can also combine input and output redirection like this

wc < datefile > datecount

This does the word count as before and then redirects the output to the named file.

If you try to redirect to a file that already exists then you will get an error message.  If you do not want this then you can use >! to automatically overwrite the file.

Summary of redirection operators

Operator(s) Behaviour
> Standard output redirection
>! Standard output redirection and automatically overwite the file if it exists.
>& Output the standard error as well as the standard output
>&! Output the standard error as well as the standard output with automatic file overwriting.
>> Redirect standard output and append to file.  An error is generated if the file does not exist. (Assuming variable noclobber is set)
>>! Redirect standard output and append to file.  No error is generated if the file does not exist.
>>& Redirect standard output and standard error and append to file.  An error is generated if the file does not exist. (Assuming variable noclobber is set)
>>&! Redirect standard output and and standard error and append to file.  No error is generated if the file does not exist.
< Redirect standard input
<< word Redirect from standard input until exact match for word is encountered

Piping

Piping is a very simple concept, but very powerful.  The basics concept is that you can create a `pipeline' where the output of one command becomes the input of the next command.  The following example illustrates this feature.

ypcat hosts | grep "tinky-winky"

The output of the ypcat command (a list of everything registered on the network) becomes the standard input of the grep command which searches through that for all lines with the string "tinky-winky" on.  Without a pipe you would have to do this

ypcat hosts > tempfile
grep "tinky-winky" tempfile
rm tempfile

You can use pipes for anything where you need to pass standard output to standard input, and you can use as many times as you need on one line.  Here are a few more examples of using pipes

zcat myfile.ps.gz | psnup -2 | lpr -Plw-g10a
psnup -2 myfile.ps | lpr -Plw-g10a
du -k | sort -rn | more
rup | awk '{print $1,$(NF-1)}' | sort -rn +1 -2

Command Substitution

This section uses Solaris specific example code...to be updated

Command substitution allows you to use the results of the command within another command.  We can use the command substitution operators (the forward quote character `) to run a command and then insert the results back into the command line.  In this example, the shell runs the hostname command (which just returns the name of the machine you are using) and inserts it back into the command line for the locate command you use.

locate `hostname`

If you were creating a command which made a compressed backup of some files and used the date and time as the filename then you can use the command substitution operator as follows.

zip backup`date +%h%d-%H:%M`.zip *.java
backupAug08-13:19.zip

Combining methods

This example shows how the techniques described can built up to form a complex command.  The aim is to create a list of everything on the network in a given room.  Try typing each of the steps to see the results.

We'll start by using the ypcat command to display the complete list.

ypcat hosts

We need to extract the machine name from that 2nd field of that list.  To do this we use a program called awk.

ypcat hosts | awk '{print $2}'

The list that's printed contains a lot of duplicates so we use sort -u to remove them

ypcat hosts | awk '{print $2}' | sort -u

To get the location we need to run each of those names through the locate command.  To do this we use the command substitution operator

locate `ypcat hosts | awk '{print $2}' | sort -u`

We then need to use grep to test whether each of the lines contains the room name.  We'll use G10 in this step but convert it to an argument later.

locate `ypcat hosts | awk '{print $2}' | sort -u` | grep -i g10

We can use awk again to strip off all the stuff we don't want and print the final list.

locate `ypcat hosts | awk '{print $2}' | sort -u` | grep -i g10 | awk '{print $1}'

Aliases

Aliases allow you to give a different name to a command or group of commands. This is a very powerful feature. At the simplest level you can do things like this.

alias n "netscape &"

This creates an alias so that typing n will run netscape as a background process. You are not limited to this though, anything can be aliased. e.g.

alias dir "ls -l | dir"
alias usage "du -k | sort -rn | more"

You can also supply an argument to an alias. For example

alias dictionary "grep -i \!* /bham/ums/dict/oxadln.dict"

searches throught the word file for the given word.

Take another look at our machine location lister.  If we want to convert it into an argumented alias we have a few problems.

alias lister "locate `ypcat hosts | awk '{print $2}' | sort -u` | grep -i \!* | awk '{print $1}'"   WRONG

The first problem is that the command substitution (in red) would be expanded at the time of definition rather than at the time of execution.  To stop this behaviour we can use single quote characters.  Anything in single quotes is 'protected' from the shell and will not be substituted.  So we can change the line to this

alias lister 'locate `ypcat hosts | awk '{print $2}' | sort -u` | grep -i \!* | awk '{print $1}''   WRONG

At first glance you may think this solves our problem, however the shell forbids sets of nested single quotes (in red).  In order to solve this we need to escape the quotes using backslash characters.  The line becomes

alias lister 'locate `ypcat hosts | awk \{print\ \$2\} | sort -u` | grep -i \!* | awk \{print\ \$1\}'   CORRECT

Aliases are useful for simpler problems but they are quite limited when it comes to solving more complex problems.  To solve such problems we need to use shell scripts.