A chronological documentation test project, nothing serious, really!

20 May 2007 Ten Cool Coreutils Commands

While a few commands, such as ‘cd’ are built directly into bash; many of the most important commands come from coreutils, a GNU package containing over a hundred commands.

Some are the well known commands such as ‘ls’, ‘mv’ and ‘cat’. While there are some more obscure ones that are not always discovered because there are also more modern programs that can do similar jobs.

These less well known commands often reflect the history of computing where once upon a time you would do everything possible in the Bourne or Bash shell, rather than waiting a long time for program such as emacs or vi to start, which at the time were considered to be memory intensive but now seem lightning fast (now you can even run them from a mobile phone).

Here we will look at ten useful commands that might come in handy one day.

1. cat

One of the key commands is ‘cat’, short for concatenate, which you can use to add the contents of one file to another, or just to print a file or set of files out to the screen.

cat 1.txt 2.txt > 3.txt # Adds 1 and 2 together as a new file 3.

cat 3.txt # Prints file 3 out to the screen.

However, tac is cat backwards, it lets you concatenate and print files in reverse. It could come in very handy for some uses, for example if you want to reverse a list or a log file.

tac 1.txt > 2.txt # 2 is a reversed copy of 1

2. tee

Often you will pipe the output of one program as the input of another:

ps -e | grep apache

This command will feed to grep the output of ps, thus showing what apache processes are currently running.

Or you may redirect the output of a command to a file:

ps -e | grep apache > apache-processes.txt

However, what if you want to see the results and write the file or files? This is where the tee command is useful:

ps -e | grep bash | tee bash-processes.txt # Print the results to the screen but also to a file.

You can enter more than one filename if you want multiple copies.

3. pr

Most printers these days will attempt to print out anything that you throw at them. However, you still may want to format a file in a certain way before sending it to the printer. For example, you may regularly discuss a log or some records at a meeting and want them to look the same every week.

The pr command gets a text file ready for the printer. For example you may want a set page or column format.

pr +10 -h”Apache Errors” -l25 error_log | lpr -# 5

Reading the pr command from left to right, the options are to start from page 10, then add a header to each page, make each page 25 lines long and lastly the filename. The result is then piped to lpr which will submit the file to the printer with a request for 5 copies.

4. stat

Using ‘ls -l’ will give a lot of information about a file, enough for me at least. If you are a glutten for punishment, you can use the ’stat’ command to get more information:

$ stat stat
File: `stat’
Size: 34684 Blocks: 72 IO Block: 4096 regular file
Device: 302h/770d Inode: 7586763 Links: 1
Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2007-01-06 00:35:15.000000000 +0000
Modify: 2006-09-18 09:50:24.000000000 +0100
Change: 2007-01-04 14:51:47.000000000 +0000

5. yes

‘yes’ is one of those strange legacy commands, the most comic featured here. ‘yes’ prints a string and a newline and then repeats until interrupted.

As dumb as it sounds, it does have a couple of uses. Firstly, if you want to turn an interactive command into a non-interactive one.

There are some command line programs which you run and then it asks something like “Are you sure?”. For example:

yes 5 | command

This will run command then send the string 5 and a newline (i.e. as if you had pressed return).

It almost goes without saying that you can also use yes to turn your processor up to 100% usage, for example if you are testing some fans or cooling system or if you otherwise want to punish your machine.

6. expand

So you wrote lots of nice Python files, indenting them with tabs. But shock horror, you found out that the current fashion is to use four spaces not tabs!

No problem, the expand command converts tabs to spaces, here we have chosen to use 4 spaces:

expand -4 >

To go from spaces to tabs, you can use ‘unexpand’ command.

7. split

’split’ takes a file and splits it into chunks for you. For example:

split -l 20 access_log part

This command will split the file access_log into chunks of 20 lines each, the name of each file will begin with part.

8. uniq

Sometimes you will have a file that is a long list of items and you want to remove all the repeated lines, or possibly you want to group the items into sets with a number of occurrences.

So to remove all repeated lines:

uniq file.txt

You can also use the output of another command:

cat file.txt | uniq

To include the occurrence count:

uniq -c file.txt

9. wc

wc allows you to count words, lines and bytes. The default is to show all three:

wc ulyss12.txt
32758 267235 1561677 ulyss12.txt
[lines] [words] [bytes]

Using -l just gives you lines:

wc -l ulyss12.txt
32758 ulyss12.txt

Using -w just gives you words:

wc -w ulyss12.txt
267235 ulyss12.txt

10. shred

‘rm’, the remove command, unlinks a file so the space can be reused. However, files deleted can sometimes be recovered with a bit of persistence and luck.

Our final command is shred. ’shred’ overwrites a file repeatedly, making it much harder to recover. This can be useful for personal financial information such as your credit card details.

shread -u mastercard.txt

You can even use shred on a device such as a partition. If you want to completely clear your home partition which happens to be stored on /dev/sda6. Then you would use:

shread /dev/sda6

Alert readers will note that I used -u in the first example but not the second. This because you want to unlink (i.e. rm) a file but not a device – you will want empty the /dev/sda6 to exist.

Now there are some complications to this. Firstly, a complex RAID setup might interpret this as hardware failure and replace the data. Likewise some corporate setups will sync the files with a server. Shred will not remove these copies.

Lastly, some fancy modern journaled filesystems such as reiserfs will have a backup in the journal, you will need to mount the partition in a non-journaled mode for shread to work completely.

There we go, ten commands that you may not have heard of, I hope you can find at least one that is useful in your computing activities.


Tags: , , ,

Posted by

09 May 2007 Script to unmount Busy Devices

First the script tries to umount the drive normally. If that fails, it tries to restart famd which is the most common problem. If that fails it tries to restart xinetd. If that fails it uses the command fuser -ki which asks you if you want to kill each process that’s using that folder. Be careful with this – you should know what each process is before you kill it. If that fails, probably because you didn’t kill all the processes, it asks you if you want to lazily unmount the volume, which removes the drive even though processes are still using it.


if [ `whoami` != "root" ]; then
      echo "You must run this as root"

# testing args
if [ $# -ne 1 ]; then
  echo "usage : $0 "


# first try unmounting it without doing anything special
testumount=`umount $dir 2>&1`
if [ `echo $testumount | grep "not mounted" | wc -l` -gt 0 ]; then
      echo "$1 isn't mounted, exiting"
if [ `echo $testumount | grep "not found" | wc -l` -gt 0 ]; then
  echo "$1 does not exists, exiting"

if [ `echo $testumount | grep busy | wc -l` -gt 0 ]; then
       echo "Having trouble, checking famd..."
     echo "unmounted $dir without any trouble..."

# check famd
if [ `ps -e |grep famd|wc -l` -gt 0 ]; then
      /etc/init.d/famd restart
      # try unmounting again
      if [ `umount $dir 2>&1 | wc -l` -gt 0 ]; then
            echo "I tried restarting famd, but that didn't work. checking xinetd"
            echo "Unmounted $dir by restarting famd"
      echo "famd isn't running so it couldn't be that.."

# check xinetd
if [ `ps -e | grep xinetd | wc -l` -gt 0 ]; then
      /etc/init.d/xinetd stop
      # try unmounting again
      if [ `umount $dir 2>&1 | wc -l` -lt 1 ]; then
            /etc/init.d/xinetd start
            echo "Unmounted $1 by restarting xinetd"
      /etc/init.d/xinetd start
      echo "I tried stopping xinetd, but that didn't work."

      echo "xinetd isn't running so it couldn't be that.."

echo "I'm going to list processes that are using the folder you are
trying to umount. Answer whether you'd like to kill them or not.
You should look up each process before you answer. use the
following command in a separate shell:"
echo "	ps -e | grep
echo "where 
 is the number of the process."
fuser -ki $dir
if [ `echo $testumount | grep busy | wc -l` -gt 0 ]; then
      echo "OK, this is the last resort. Do you want to umount the volume using "
      echo "the -l option? According to the umount man page -l means:"
      echo "	Lazy unmount. Detach the filesystem from the filesystem
      hierarchy now, and cleanup all references to the filesystem as
      soon as	it is not busy anymore.  (Requires kernel 2.4.11 or later.)"
      echo "Use -l option? (y\n)"
      read yn
      if [ $yn == "y" ]; then
            umount -l $dir
            echo "OK. Sorry I couldn't help"
      echo "You're all good :)"


Tags: , , , , ,

Posted by