Hints and Tips for general shell script programming
WARNING: this will fail if the user is playing with $0
For example using a symbolic or hard link with a unexpected name.
# Simplest...
#
PROGNAME=`type $0 | awk '{print $3}'` # search for executable on path
PROGNAME=`basename $PROGNAME` # base name of program
# Advanced...
# Script name, in what directory, and in what directory is user
# running the script from.
#
# Discover where the shell script resides
ORIGDIR=`pwd` # original directory (builtin)
PROGNAME=`type $0 | awk '{print $3}'` # search for executable on path
PROGDIR=`dirname $PROGNAME` # extract directory of program
PROGNAME=`basename $PROGNAME` # base name of program
cd $PROGDIR # go to that directory
PROGDIR=`pwd` # remove any symbolic link parts
cd $ORIGDIR # return to original directory (opt)
Results...
$ORIGDIR -- where the users was when called
$PROGDIR -- script directory location (and now current directory)
$PROGNAME -- This scripts executable name
Shell Script Option Handling
PROGNAME=`basename $0` # Or the above script locator
Usage() {
echo >&2 "$PROGNAME:" "$@"
echo >&2 "Usage: $PROGNAME [options] [file...]"
exit 10
}
#!/bin/sh
#
# script [options] args...
#
# The frist 'Usage()' code that follows these comments locates the script on
# disk, and then reads and output these comments as the documentation for this
# script. That is the script commands and usage documention are in the same
# file, making it self documenting, via options.
#
###
#
# programmers only docs, whcih are not output by Usage()
#
PROGNAME=`type $0 | awk '{print $3}'` # search for executable on path
PROGDIR=`dirname $PROGNAME` # extract directory of program
PROGNAME=`basename $PROGNAME` # base name of program
Usage() { # output the script comments as docs
echo >&2 "$PROGNAME:" "$@"
sed >&2 -n '/^###/q; /^#/!q; s/^#//; s/^ //; 3s/^/Usage: /; 2,$ p'
"$PROGDIR/$PROGNAME"
exit 10;
# Generalized Otion handling.
#
while [ $# -gt 0 ]; do
case "$1" in
# Standard help option.
--help|--doc*) Usage ;;
# Simple flag option
-d) DEBUG=true ;;
# Word flag option
-debug) DEBUG=true ;;
# Simple option and argument EG: -n name
-n) shift; name="$1" ;;
# Option and argument joined EG: -Jname
-J) name=expr "$1" : '-.(.*)'` ||
Usage "Option \"$1\" missing required value" ;;
# Joined OR unjoined Argument EG: -Nname or -N name
-N*) Name=`expr "$1" : '-.(..*)'` || { shift; Name="$1"; } ;;
# As last, but if -b0 is an posibility then you need to use this instead.
# Otherwise it will think the value is unjoined when it isn't Arrrgghhhh...
-b*) bits=`expr "$1" : '-.(..*)'`
[ "$bits" ] || { shift; bits="$1"; }
;;
# Numbers with type checks EG: -{width}x{height}
-[0-9]*x[0-9]*)
w=`expr "$1" : '-([0-9]*)x'` || Usage "Bed Geometry"
h=`expr "$1" : '-[0-9]*x([0-9]*)$'` || Usage "Bad Geometry"
[ "$width" -eq 0 -o "$height" -eq 0 ] && Usage "Zero Geometry"
geometry="${w}x${h}"
;;
# Number with type check EG: -{size}
-[0-9]*)
size=`expr "$1" : '-([0-9]*)$'` || Usage
[ "$size" -eq 0 ] && Usage
Width=$size; Height=$size
;;
# Generalised Argument Save EG: -G value => $opt_G
-*) var=`expr "$1" : '-(.).*'`
case "$var" in
[a-zA-Z]) arg=`expr "$1" : '-.(..*)'` || { shift; arg="$1"; }
eval "opt_$var"="\"$arg\"" ;;
*) Usage "Bad Non-Letter option \"$1\"" ;;
esac ;;
-) break ;; # STDIN, end of user options
--) shift; break ;; # end of user options
-*) Usage "Unknown option \"$1\"" ;;
*) break ;; # end of user options
esac
shift # next option
done
# Handle normal arguments now...
Files=${1+"$@"}
# or
[ $# -gt 0 ] && Usage "To many Arguments"
Variable testing under shell (rules of thumb)
Boolean Variable tests…
true="true"; false=""
[ "$var" ] && echo var is true
[ -z "$var" ] && echo var is false
Test of variables containing ANY string (see PROBLEM CASES below)
option=-xyzzy
[ "X$option" != X ] && echo option is defined
[ "X$a" = "X$b" ] && echo "$a equals $b"
Rules of thumb…
* Always quote all variables being tested
* Only use the boolean type test (above) when
all possible values are defineately known.
* Prepend X (or other alphanumberic) to unknown strings being compared
* Don’t use ! if you can avoid it.
PROBLEM CASES…
These cases cause the “[…]” tests to fail badly!
[ $var ] but var="a = b"
[ "$var" ] but var=-t (actually any option starting with '-')
[ "$a" = "$b" ] but a='(' and b=')'
This is why you must use the string test above (with ‘X’ prefixes).
NOTE test ! … differs from UNIX to UNIX
Under bash it is the NOT of the next 3 arguments following
Under solaris “sh” it seems to handled with precedence.
EG: test “” -a ! “” is false as you would expect
BUT test ! “” -a “” is true for BASH and false for Solaris SH
Argument Sorting By Shells
All Shells normally sort the output of any commands with meta characters.
Also “ls” will sort its arguments again internally, while “echo” will not.
EG compare the output of these commands
ls -d b* a*
echo b* a*
echo [ba]*
The first sorts the output so a’s are before b’s, the second does not.
However the shell sorts the arguments of the third so a’s are again first.
However the “{}” meta characters do NOT sort the results…
echo {b,a}*
will output b’s first then a’s.
This can be useful when the order of the arguments is important.
This has been tested and works as described in csh, tcsh, bash, and zsh
Argument handling…
If you care for the possibility that there aren’t any arguments.
“$@” -> “” (one empty argument)
${1+”$@”} -> nothing at all
This is not a problem with newer modern shells, only very old Bourne shells.
WARNING: Command line shell argument start at 0!
sh -c ‘echo $*’ 1 2 3 4 5
or sh -c ‘echo $*’ `seq 5`
Will print
2 3 4 5
Note that the first arguement is missing. That is because it was assigned to
argument $0 and not $1 or the $@ and $* variables!
To use all the arguments on the shell command line, use either
sh -c ‘echo $*’ junk `seq 5`
or
sh -c ‘echo $0 $*’ `seq 5`
the later is not recomended as if no arguments are provided $0 defaults
to “sh” instead of the empty string!
This is a posix compliant feature, $0 being the program name when a shell
script is called. the “-c” command line script just layers on top of this.
this is the case with solaris bourne sh, bash, ksh, and zsh. csh and tcsh
shell does not have an equivelence.
See “find” and its “-exec” option, (see below), for a useful example of this.
I also use it in csh aliases to provide a default argument.
EG: alias xvd ‘sh -c ”’cd ${1:-.}; xv -vsmap &”’ junk !:* &’
Auto change shells on brain dead systems (Ultrix)
#!/bin/sh - # # Check the type of shell that is running (For Ultrix) [ "X$1" != 'X-sh5' -a -f /bin/sh5 ] && exec /bin/sh5 -$- "$0" -sh5 "$@" [ "X$1" = 'X-sh5' ] && shift #
Getting environment from csh
env - DISPLAY=$DISPLAY HOME=$HOME TERM=dumb
csh -cf 'set prompt="> ";
source .cshrc; source .login;
echo "#------ENVIRONMENT------"
env' |
sed -n '/#------ENVIRONMENT------/,$p'
Inserting a AWK or PERL script inside a SHELL script…
Example…
nawk '# output to mailx commands!
/^[^ ]/ { recipent=$0; }
/^ / { print $0 > "|mailx -s \"This is the test\" " recipent; }
/^$/ { close( "|mailx -s \"This is the test\" " recipent );
' list.txt
Note the whole script is inside single quotes on the nawk command line!
ASIDE: old versions of awk must have something on the first line thus the
addition of the # comment to keep it happy! Perl needs no such comment but
does require a -e option to execute a command line argument.
To insert a external shell variable into the script you need to close
the single quotes, output variable and re-open the single quotes. Also
the variable sould be in double quotes so as to prvent any insertion of
space characters
Example inserting a $prefix shell variable into a awk string.
...
{ print "'"$prefix"'" $0; }
...
Also to insert a single quote into the script you have to also exit the
wrapping single quotes and supply it outside those quotes
...
{ print "I just can'''t do that!"; }
...
CAUTION: Watch for single quotes inside any COMMENTS which is in the script!
Comments are within the single quotes so are also scanned for thos quotes.
CSH SCRIPTS: If you must write a csh script you will need to escape the new
line at the end of every line, even though single quotes are being used, which
requires to backslashes.. Also watch out for history escapes `!’ which work
inside single quotes!
Is COMMAND available
There is two techniques available to test if a command is available.
The first is to use the `status’ return of the “type” or “which” command
of the shell you are using. The Second is to examine the output of that
command itself.
Using status return.
This is a simple method of testing the existance of a command
but DOES NOT WORK ON ALL SYSTEMS! The problem is that old shells
(For Example: SunOS bourne-sh) always returns a true status weather
the command is present or not!
Bourne Shell
if type COMMAND 2>&1; then
# COMMAND is available
fi
C-Shell
Warning: The “which” command in C shell is not a builtin but a external
script which sources your .cshrc (YUCK). As such the bourne shell alias
is prefered which will only search your current command PATH.
if ( ! $?tcsh ) then
alias which 'sh -c "type !:1 2>&1"'
endif
if ( -x "`which COMMAND >/dev/null`" ) then
# COMMAND Available
endif
TC-Shell
Tcsh 6.06 also does not return the correct status in its which
command. Use it like the csh which above.
WARNING: this method will also test positive for :-
subroutines, bash aliases, and probably other non-command definitions.
Examine output
The other am more reliable method is to examine the output of the
“type” or “which” command to looks for the string “not found” (See
below). This is more complex than the above status check but should
work on ALL unix machines regardless of age.
Bourne Shell
cmd_found() {
case "`type $1 2>&1`" in *'not found'*) return 1 ;; esac; return 0
}
...
if cmd_found COMMAND; then
# COMMAND is available
fi
C-Shell & Tcsh (See notes below)
if ( ! $?tcsh ) then
alias which 'sh -c "type !:1 2>&1"'
endif
...
if ( "`which less`" !~ *'not found'* ) then
# COMMAND Available
endif
NOTES for “which/type” commands :-
The above methods look for the specific string “not found”
This is important as the sh type command and tcsh which command
produce different output, and this may also vary from bourne shell
to bourne shell or other shell types.
Csh — “which” is an unreliable shell script!
fudge it into a shell script “type” command.
See the “Which Problem” below.
Tcsh
> which less
/opt/bin/less
> which junk
junk: Command not found.
> which which
which: shell built-in command.
> alias a alias
> which a
a: aliased to alias
Solaris Bourne shell
> type less
less is /opt/bin/less
> type junk
junk not found
> type type
type is a shell builtin
> func(){ echo ha; }
> type func
func is a function
func(){
echo ha
}
Solaris Ksh
As per Sh, but the actual function definition is NOT listed
Bash
> type less
less is /opt/bin/less
> type junk
bash: type: junk: not found
> type type
type is a shell builtin
> func(){ echo ha; }
> type func
func is a function
func ()
{
echo ha
}
NOTE: bash also has a type -t which responds with a single word “file”,
“alias”, “function”, “builtin”, “keyword”, or nothing if command does not
exist. A -p will print the disk file name, or nothing. A -a prints all the
places that have that name.
From the results above, only the appearence of “not found” in the false
statement is consistant. But only when the result is not a bourne shell
function, which presumably replaces the real-command of that name.
The Expanded Bourne shell form, without using the “cmd_found” function is as
follows, But is is a low simpler and easier to read if you use the funtion.
If command present
if expr match “`type COMMAND 2>&1`” ‘.*not found’ == 0 >/dev/null; then
# COMMAND is available
fi
and its inverse (not present)
if expr match “`type COMMAND 2>&1`” ‘.*not found’ >/dev/null; then
# Comand is NOT present
fi
finally only using built-in commands…
case “`type COMMAND`” in
*’not found’*) # Command not found ;;
*) # Command found ;;
esac
Functional forms
cmd_found() {
expr match “`type $1 2>&1`” ‘.*not found’ == 0 >/dev/null
}
OR
cmd_found() {
case “`type $1 2>&1`” in *’not found’*) return 1 ;; esac; return 0
}
Which problem!
The which command is a csh script that specifically reads the .cshrc
file to find out about aliases. To avoid having .cshrc do lots of odd
things to your script, use the following form.
set program = `/bin/env HOME= /usr/ucb/which $program`
This is NOT a problem in Tcsh, where “which” is a built-in.
The best solution it to replace “which” with the bourne shell “type” command
in C shells but leave it alone for TC shells.
if ( ! $?tcsh ) then
alias which ‘sh -c “type !:1 2>&1” | sed “s/.* is //”‘
endif
set program = `which $program`
——————————————————————————-
builtin cat command
This cat only uses the shell builtins! As such can be used on a machine
which has no access to shared libraries and nothing but the statically
linked bourne sh can run.
shcat() {
while test $# -ge 1; do
while read i; do
echo “$i”
done < $1
shift
done
}
Of course the real cat command is only as big as it is, to protect the
user from himself and to provide a huge number of options.
PS: If the ls command is also not available then you can use
echo *
to do a directory listing using only builtins. Though this will not tell
you what files are executables or sub-directories.
——————————————————————————-
Note `cmd` does save the newline characters within the output
::::> m=`mount`;
::::> echo $m
/ /usr /home
::::> echo “$m”
/
/usr
/home
In other words outside quotes newlines in the input are treated purely
as white space between arguments and thus ignored. Inside quotes
newlines are retained as newlines.
——————————————————————————-
One line if-then-else shell command
cmd1 && cmd2 || cmd3
This hoever will execute cmd3 if cmd2 fails! If cmd2 never fails — fine
——————————————————————————-
The useless use of ‘cat’!
You often see in shell scripts…
cat input_file | some_command
The cat is usless as it is exactly the same as
some_command < input_file
without needing to fork the extra “cat” process or creating a pipeline.
However it is sometimes usefull to do anyway to make the code more readable.
Particularly in long pipelines.
——————————————————————————-
Cat with here file and pipe to another command (Shell Syntax Example)
cat <
…..
E0F
while read line; do
…
done
This is also a “Useless use of ‘cat’!”. The equivelent without cat is…
while read line; do
…
done <
…..
EOF
Of course the “cat” could be a “sed” “awk” or other filter of the here file
data before you feed it into the while loop.
Also in the second method, the here file is then at the end, which can be
confusing for later code readers as their is no indication at the start of the
loop you are reading from a here file and not from the stdandard input of the
script.
Because of this I usally prefer the cat, unless the while-loop is very very
short.
WARNING: See the “non-posix shell bug” below
——————————————————————————-
Non-Posix shell bug
foo=bar
while read line
do
foo=bletch
done < /etc/passwd
echo “foo = $foo”
In some shells redirecting a file into a control structure will use a
sub-shell for that structure. In the above this will result in “foo =
bar” instead of “foo = bletch” assigned in the while loop.
The POSIX standard forbids this behaviour.
——————————————————————————-
echo without a return (bsd & sysV)
For Bourne Shell
# echo without a return (bsd & sysV)
if [ “X`echo -n`” = “X-n” ]; then
echo_n() { echo ${1+”$@”}”c”; }
else
echo_n() { echo -n ${1+”$@”}; }
fi
For Csh or Tcsh (which you shouldn’t use for scripts anyway)
# echo without a return (bsd & sysV)
if ( “X`echo -n`” == ‘X-n’ ) then
alias echo_n ‘echo !:* “c”‘
else
alias echo_n ‘echo -n !:* ‘
endif
——————————————————————————-
Columizing data
pr
To convert a file (or part of a file) to columns.
Warning: this program will truncate data to fit into space available!
Example file: numbers 1 – 10, one number per line
It is recomended that the -# be the first option.
Down the columns (like ls)
pr -3 -t -l3
| `– number of columns
`—– lines per column (very important to give correctly)
if output width is a problem add -w80
EG: seq -f %02g 10 | pr -3 -t -l3 -w30
Outputs: 01 04 07
02 05 08
03 06 09
10
Or on most linux machines..
column -x -c30
`— width of output line
Across
paste – – –
Number of – given determines number of columns
EG: seq 10 | paste – – –
Outputs: 01 02 03
04 05 06
07 08 09
10
OR (with posible data truncation to fit ouptput line width)
pr -t -l1 -3
column
On linux to just put the data into columns use “column”
CAUTION: any meta-characters found will may column silently ABORT,
truncating the file being processed!
Example:
seq -f %03g 5 5 500 | column
This can also format text tables using “-t” function of column to preserve
the line by line structure of the input file.
column -s: -t /etc/group
OR a more complex example…
( echo “PERM LINKS OWNER GROUP SIZE MONTH DAY HH:MM NAME”;
ls -l | tail -n+2;
) | column -t
Warnings:
* It also does not work as well as should
EG: compare the output of “ls” and “ls | column”
* Tabs are used. Use the “expand” filter to remove them before using
the “column” filter.
You can also using a non-standard perl module (from CPAN)
seq -f %03g 100 |
perl -MArray::PrintCols -e ‘@a=<>; chomp @a; print_cols @a’
Otherwise manually handle columns in perl
seq -f %03g 100 |
perl -e ‘ @a=<>; chomp @a;
my ( $c, $i ) = (10,10);
foreach ( @a ) {
print(“n”),$i=$c unless $i;
printf ” %4s”, $_; $i–;
} print “n”; ‘
——————————————————————————-
Suppressing the shell background fork message (*csh)
The trick is to redirect the standard error output of the _shell_ itself
Korn (with loss of standard error)
( command & ) 2>/dev/null
csh/tcsh (this appears to work)
( command & )
bourne shell does not give a message about background forks ever.
——————————————————————————-
Auto Background a shell script
#!/bin/csh -f
if ( $1:q != ‘…’ ) then
( $0 ‘…’ $*:q & )
exit 0
endif
shift
…rest of script to run in background…
OR
#!/bin/sh
foreground stuff
( background stuff
) &
exit
——————————————————————————-
Sort a shell array
=======8< ——–
i=0
for n ;
do
 a[$((i++))]=$n ;
done
# sort array
for (( i=0; $i<(${#a[@]}-1); i++ )) ;
do
 for (( j=i+1; $j<(${#a[@]}); j++ )) ;
 do
   if [[ ${a[$j]} < ${a[$i]} ]] ;
   then
     t=${a[$j]};
     a[$j]=${a[$i]};
     a[$i]=$t;
   fi
 done
done
# launch xmms with sorted list
xmms -e “${a[@]}”
=======8<——–
——————————————————————————-
Command timeout
Run a command but kill it if it runs for too long. This prevents scripts
hanging on a command, especially network related commands, like nslookup.
Simple C solution…
Look at ~/store/c/programs/timeout.c which will timeout a command
simply and easilly. A lot better than the complex solutions below.
—- Attempt 1 —-
Runs the command in the background and waits for it to
complete. A sleep command is also run to provide a timeout.
Works but full timeout period is always waited before exiting
TIMEOUT=60 # timelimit for command
( command_which_can_hang &
sleep $TIMEOUT; kill $! 2>/dev/null
)
This does not work. We will wait on the sleep for the full TIMEOUT period
regardless of how fast the command completes. If the command is known to
never finish then the above will be fine!
—- Attempt 2 —–
Works for modern day shells…
do_quota() {
# lookup the users disk quota but with a timeout
quota -v “$@” &
# now ensure that the above command does not run too long
cmd_pid=$!
( sleep $QUOTA_TIMEOUT
echo >&2 “Quota Timeout”
kill -9 $cmd_pid 2>/dev/null
) &
kill_pid=$!
wait $cmd_pid
kill $kill_pid 2>/dev/null
}
The problem here is that this works fine but the sleep will still continue
for the full timeout period. Basically the parent has no simple way of
determining the PID of the sleep, to abort it, if it was not needed. This is
not a problem if the sleep itself is not too long, so it doesn’t stick around
for hour.
The sub-shell however is killed, so the kill command is not run if the command
completes before the timeout
Unfortunatally BASH generates a “terminate” message on standard out when
the command is killed due to timeout :-(
Full Example
try with/without a timeout argument of 15…
=======8< ——–
#!/bin/sh
#
# Countdown [abort_after]
#
COUNTDOWN_FROM=10 # count down from this value
COUNTDOWN_ABORT=${1:-4} # abort countdown after this long
countdown() {
i=$COUNTDOWN_FROM
echo “Countdown continuing at T minus $i”
while [ $i -gt 0 ]; do
sleep 1
i=`expr $i – 1`
[ $i -eq 3 ] && echo “Main engine start”
|| echo “–> $i”
done
echo “We have lift off!!!”
exit 0;
}
countdown & # the command that we wish to timeout
cmd_pid=$!
( sleep $COUNTDOWN_ABORT;
kill $cmd_pid
echo “countdown aborted”;
) &
kill_pid=$!
wait $cmd_pid
echo “Launch status: $?”
kill $kill_pid 2>/dev/null
#/bin/ps # uncomment to show if sleep is still running
=======8< ——–
—- Attempt 3 —-
Reports a `Terminate’ error to stdout if command timesout.
TIMEOUT=60 # timelimit for command
command_which_can_hang &
cmd_pid=$!
sleep $TIMEOUT | (
read nothing # wait on sleep to finish
kill $cmd_pid 2>/dev/null; # otherwise abort the command!
) &
sleep_pid=$!
wait $cmd_pid
kill -ALRM $sleep_pid 2>/dev/null # kill sleep if still running
The reason this works is that we are specifically killing the sleep (we hope).
This is because $! returns the pid of the first command in a command pipeline,
so we fake such a pipeline.
WARNING: Some shells may have $! set to the last command in pipeline! which
make this equivelent to “Attempt 2” though will also work as expected.
Using an ALRM signal to kill sleep is especially good as it means sleep exits
normally and cleanly, stopping a shell like “bash” from producing “Terminated”
messages all over the place.
In the “kill command” sub-shell, the “read” is needed to ensure it waits for
non-existant output from the sleep command. It will automatically exit when
the sleep exits without producing any output.
A “kill -0 $pid” could be used to check if the pid given is still running or
not, and in a full working version is probably a good idea.
I suppose we could also backgound the sleep, and have a background kill
process wait specifically on the background sleep process but this seems to
complex to me…. It also does NOT work as you wither have to put a a wait
for the sleep the sub-shell (or a looped pid poll) which did NOT launch the
sleep and thus does not know its pid, or the background sleep in the
sub-shell, in which case the main script did not launch it so can’t abort it.
Very difficult to sort that out. Better to use a command pipeline.
I supose you could use a `SIGCHLD trap’ but that is a completely different
method. If anyone have created such a trap in shell, please email me your
code, so I can include it here (with your name of course).
Full example..
TIMEOUT=20 # Limit name server lookup to 20 seconds
# …
do_nslookup() {
# Do a FAST nslookup – even if primary namserver is down
# Background the required command – note the process id
# any output to be printed to standard argument and calling function
nslookup 2>/dev/null < <-EOF | sed -n ‘/arpa/s/.*name = //p’ &
set type=PTR
$1
EOF
cmd_pid=$!
#
# Wait for it (sleep pipeline) – note sleep’s process id
sleep $TIMEOUT | (
read nothing # wait on sleep to finish
kill -0 $cmd_pid 2>/dev/null || exit; # cmd finished? – exit sub-shell
kill $cmd_pid 2>/dev/null;
echo >&2 “WARNING: nslookup failed to return in $TIMEOUT seconds!”
) &
sleep_pid=$!
#
# wait for nslookup to finish or be killed
wait $cmd_pid
kill -0 $sleep_pid 2>/dev/null || return; # sleep finished?
kill $sleep_pid 2>/dev/null
}
# …
Reverse_IP_Arg=”4.3.2.1.in-addr.arpa”
FQDN_hostname=`do_nslookup $Reverse_IP_Arg`
——————————————————————————-
Loop until parent process dies
Background tasks have an annoying habit of continuing to run AFTER you have
logged out. This following example program looks up the launching parent
process (typically your login shell) and only loops if the parent process is
still alive.
WARNING: The PS command varies from UNIX system to UNIX system so you will
have to tweek the arguments to the `ps’ command to make this script work on
your UNIX system
=======8< ——–
#!/bin/sh
#
# Loop until parent dies
#
sleep_time=300 # time between background checks
# Pick the appropriate ps options for your UNIX system
# Uncomment ONE of the following lines
#job_opt=xj; sep=””; ppid=1; # SunOS
job_opt=xl; sep=” “; ppid=4; # Solaris
#job_opt=xl; sep=” “; ppid=4; # IBM UNIX (aix)
#job_opt=xl; sep=” “; ppid=4; # SGI UNIX (irix)
# Discover the parents process ID
set – `ps $job_opt$sep$$ | tail -n+2` “1”
eval parent=$$ppid
# While parent is still alive
# The kill command “-0” option checks to see if process is alive!
# It does NOT actually kill the process (EG: test process)
while kill -0 $parent 2>/dev/null; do
# …
# Do the background job here
# …
sleep $sleep_time
done
# Parent process has died so we also better die.
exit 0
=======8< ——–
Also see the script ~anthony/bin/scripts/hostspace
as an example of a shell script for multiple hosts
——————————————————————————-
Convert ls permissions to octal format
… anyone? …
——————————————————————————-
Time in Seconds, NOW! (for later comparision)
EG: getting the number of seconds since epoch (midnight, 1/1/1970, GMT)
This is for the purposes of timing various command and comparing the start
and end times. The problem is as there is no standard way for geting the
system clock under normal plain shell scripts.
Under linux (Gnu-date) you can
date +%s
With perl you can
perl -e ‘print time(), “n”‘
This seems to work very well on all system I know of, and can be used for
sequencing events. Do not rely on it for time deltas (how long) as it does not
handle years (%j is day within the year)
date +’%j * 86400 + %H * 3600 + %M * 60 + %S’ | bc
Convert Time since epoch to Date
(for more see the perl/general.hints page)
perl -e ‘require “ctime.pl”; print &ctime(1000000000);
——————————————————————————-
Get the date of yesterday (without a lot of fuss)
Getting yesterdays day is usually best done in either C or perl.
#!/usr/bin/perl
require “ctime.pl”;
print &ctime($^T – 60*60*24);
By changing your timezone you can fake the getting of yesterdays date,
But ONLY if you live in Australia where it is already tomorrow!.
It is not perfect (2 hours short but close enough in most cases)
env TZ=GMT-12 date
——————————————————————————-
Four Digit Year in shell (for logfiles)
For Newer UNIX Machines, IEEE date format
date=`date +’%Y-%m-%d %R:%S’`
For Older UNIX machines (SunOS)…
While the newest UNIX’s allow a %Y for a four digit year older machines
like SunOS does NOT. This means the year will need to be extracted
from the date output (or auturnative format code).
year=`date | sed ‘s/.*([0-9][0-9][0-9][0-9]).*/1/’`
date=`date $year-%m-%d %H:%M:%S”
If you have the MH mail system install you can convert the date to rfc822 date
format, use the “dp” program. This has its own mh output format codes.
/usr/lib/mh/dp “`date`”
or /usr/lib/mh/dp -format ‘%(year{text})’ “`date`”
or /usr/lib/mh/dp
-format ‘%04(year{text})-%02(mon{text})-%02(mday{text})’
“`date`”
——————————————————————————-
Setting a timed alarm in shell
This can be done (as a background task) exactly how is an another matter
As an educated guess probably something like..
# Trap the USR1 signal
trap “do timeout commands” 16
( sleep $timeout; kill -16 $$; ) &
See also “Command timeout” above which was a later addition to this file.
——————————————————————————-
Am I a Non-Interactive Shell
bourne sh:
if [ -z “$PS1″ ]; then # script/remote execution (non-interactive)
csh:
if ( ! $?prompt ) then # script/remote execution (non-interactive)
In the bourne shell a better way is to test the shell options “$-”
for the interactive flag directly.
case $- in
*i*) ;; # do things for interactive shell
*) ;; # do things for non-interactive shell
esac
——————————————————————————-
Merge multiple blank lines into one line.
This is not easy, as we want to preserve a blank line, removing extras.
Two methods, print paragraphs, or delete extra blanks.
awk ‘{ printf “%s “, $0 } NF == 0 { print “n” }’ filename
cat -s # will do this (if available)
perl -ne ‘if (/S/) { print; $i=0 } else {print unless $i; $i=1; }’
perl -ne ‘print if /S/../^s*$/’
sed ‘/./,/^$/!d’ # NOTE: (t)csh users must backslash the !
sed ‘/[^ ]/,/^[ ]*$/!d’ # For blank lines with spaces and tabs
# Line joining in the vim editor! as a macro
:map QE :$s/$/rZ/
or :%s/ns*n(s*n)*/rr/
——————————————————————————-
LIST functions
(Also usable for PATH-like environment variables)
list=”list_variable” # a:b:c:d
element=”element_of_list” # e
sep=”list_seperator” # :
Append to list (even if empty!)
list=”${list:+$list$sep}$element”
Split up list
function split_path () {
( IFS=”$sep”
set — $list
for f in “$@”; do
echo -n “${f:-.} ”
done; echo
)
}
Count of elements
count=`IFS=”$sep”; set – $list; echo $#`
get I’th element (shell must understand shift argument)
element=`IFS=”$sep”; set – $list; shift $I; echo $1`
get first element
element=`IFS=”$sep”; set – $list; echo $1`
delete first element
list=`echo “$list” | sed “/$sep/!d; /:/s/^[^$sep]*$sep//;”`
delete specific element over the whole list
list=`echo “$list” | sed “s|${element}${sep}||g; s|${sep}${element}$||;”`
——————————————————————————-
Checking for a numerical value
case “$var” in
” | *[!0-9]*) echo “non-numeric” ;;
*) echo “numeric” ;;
esac # — Nick Holloway (alfie@dcs.warwick.ac.uk)
OR
echo “$arg” | egrep ‘^[0-9]+$’ >/dev/null || echo “not-numeric”
Get value from option (NOT zero)
expr “$var” : ‘([0-9]*)$’ || echo “not a non-zero-numeric value”
——————————————————————————-
random number generation in a shell
nawk:
set range = ????
set random = `nawk ‘BEGIN { srand();
printf “%d”, rand()*’$range’
}’ /dev/null`
date:
set range = ???? # file length — `wc -l < $file`
set date = `date +%j%H%M%S`
set random = `expr $date % $range`
(k/z/ba)sh
$RANDOM
Posible improvment…
!/bin/bash
MX=”0123456789″
NL=”5″ # size of the random number wanted
while [ ${n:=1} -le $NL ]
do
NUM=”$NUM${MX:$(($RANDOM%${#MX})):1}”
let n+=1
done
echo “$NUM”
Programed (csh) (could be via expr too)
set multiplier = 25173
set modulus = 65536
set increment = 13849
set seedfile = $HOME/.rnd # seed file to use
if ( ! -f $seedfile ) then
echo ’17’ > $seedfile
endif
@ number = ( `cat $seedfile` * $multiplier + $increment ) % $modulus
echo $number > $seedfile
@ number = $number % $range
echo $number
——————————————————————————-
increment a character in shell
char=`echo $char | tr ‘ -~’ ‘!-~’
or (sutable for hex chars)
str1=”ABCDEFGHIJKLMNOPQRSTUVWXYZ”
str2=”BCDEFGHIJKLMNOPQRSTUVWXYZA”
pos=`expr index $str1 $char`
char=`expr substr $str2 $pos 1`
echo $char
In perl the “increment alpha string” makes this easy
perl -e ‘$n=”‘”$char”‘”; print substr( ++$n, -1), “n”;’
——————————————————————————-
Protecting shell scripts from ^Z
This signal can’t normally be stoped in a shell, the trick is to change
key generating the signal (Don’t forget to return it to normal).
stty susp undef — if available
stty susp ‘^-‘ — maybe system dependant
——————————————————————————-
Curses in shell script
To use termcap entries in a shell script use the `tput’ command
EXAMPLES
/usr/5bin/tput bold # bold (extra half brigtht mode)
/usr/5bin/tput bink # bink mode (if available)
/usr/5bin/tput rev # reverse video
/usr/5bin/tput smul # underline mode
/usr/5bin/tput smso # standout (usually reverse)
/usr/5bin/tput rmso # end standout (return to normal)
/usr/5bin/tput clear # clear screen
/usr/5bin/tput cup 5 23 # cursor to move to row 5 column 23
See terminfo(5) for more info.
——————————————————————————-
Split pipe into two separate commands (executable tee)
awk ‘{ print | “‘cmd1′” ; print }’ | cmd 2
——————————————————————————-
Capitalize the first word.
The initial solutions in the news group cam out to more than 10 lines of code!
# Ken Manheimer (expr-tr-expr) NOTE: fails if word=”match”
Word=`expr “$word” : “(.).*” | tr a-z A-Z“expr “$word” : “.(.*)”`
# Paul Falstad (cut-tr-sed)
Word=`echo $word|tr a-z A-Z|cut -c1“echo $word|sed s/.//`
==> # Logan Shaw (cut-tr-cut)
Word=`echo “$word” | cut -c1 | tr [a-z] [A-Z]“echo “$word” | cut -c2-`
# Harald Eikrem (sed only)
Word=`echo $word | sed -e ‘
h; ‘y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/;
G; ‘s/(.).*n./1/; ‘ `
# Tom Christiansen (perl)
Word=`echo $word | perl -pe ‘s/.*/u$&/’`
# Jean-Michel Chenais
# (korn shell built-ins only)
typeset -u W;typeset -l w;W=${word%${word#?}};w=$W;Word=$W${word#${w}}
# perl of course makes this easy
perl -e ‘print “u'”$word”‘”‘
Multi-line Paragraph handline
Most text files including this one consists of paragraphs of multiple lines
seperated by a blank line.
Convert all paragraphs into a single line (blank line seperated)
sed ‘/^$/d; :loop y/n/ /; N; /n$/! b loop; s/ */ /g; s/^ //; s/ $//’
Also remove blank lines between paragraph lines
sed ‘/^$/d; :loop N; s/n$//; T loop; y/n/ /; s/ */ /g; s/^ //; s/ $//’
WARNING: better space handling is probably needed, especially for a last
paragraph that has no final blank line after it.
——————————————————————————-
Find a Process of a particular name — see csh alias pf
Usign grep
ps auxgww | grep “$NAME” | grep -v grep | cut -c1-15,36-99
Merging the two greps..
ps uxw | grep “[s]sh-agent” | awk ‘{print $2}’
The “[s]sh-agent” will not match the grep process itself!
EG: it will not match the “[s]sh-agent” string in the grep process
Using awk…
ps auxgww | awk “/$NAME/ && ! /(awk)/” | cut -c1-15,36-99
or for a exact process name
ps auxgww | awk ‘/(^| |(|/)$NAME( |)|$)/’ | cut -c1-15,36-99
or alturnativeally which matches under a lot of conditions…
EG: matches :01 NAME arg
:01 some/path/NAME
:01 NAME:
ps auxgww |
awk ‘/:[0-9][0-9] (|[^ ]*/)$NAME($| |:)/’ | cut -c1-15,36-99
Perl version (also matches on username)
ps auxgww |
perl -nle ‘print if $. == 1
|| /^s*!:1s/o
|| /:dd (|[ *|[^ ]*/)!:1($|[]: ])/o; ‘
As you can see things can get complex very quickly
Context Grep
Or how to display lines before/after search pattern.
GNU grep, has context built in to it, check out the -A, -B, and -C options.
otherwise
grep -v pattern file | diff -c3 – file | grep ‘^. ‘ | colrm 1 2
or
grep -n $1 $2 | awk -F: ‘{ print $1 }’ | while read linenum
do
awk ‘NR>target-5 && NR
echo “——-next match———”
done
Source: http://www.cit.gu.edu.au