April 24, 2012

Quick everyday scripting in bash

From my experience, mostly in grading courses and everyday tasks on my linux box, the most common requirements for scripting deal with iterating over a given set of entities and performing some task, or iterating over a given folder's contents and performing some task. Here are some sample cases and solutions to performing the same quickly. (I use bash so I am sure all the following work in bash).

Case 1 : 

You have a program X and you want to verify that it runs correctly (lets assume running correctly means the same as the program X exiting with success) over, say 100 iterations.


for i in `seq 100`
    ./X [ args for X]
    if [ $? -ne 0 ]; then
        echo "X failed in iteration $i"
        exit 1

The construct `...` runs the command inside and is replaced by the output of that command. the "basename" command gives you the last literal in a path.
The special variable $? stores the exit code of the previous program that was run in the shell script, this is typically what you return at the end of the main() function in a C program. A return code of 0 stands for success.
Note that the above snippet will work assuming that you are in the directory which contains the file (program) X.

Case 2 : 

A folder X has contents a, b, c, d, e, f, g , each of which is a folder. You want to create a file [parent_folder_name].txt in each of those folders.

for file in X/*
    touch $file/$(basename $file).txt

The construct $(...) runs the command inside and is replaced by the output of that command. the "basename" command gives you the last literal in a path. so

$ basename X/a
$ basename X/b

A slight modification to case 2, if X has both files and directories and you want to run the operation only for directories then you can modify the above snippet to check if the entry is a file/directory and act accordingly :

for file in X/*
    if [ -d $file ]; then
        touch $file/$(basename $file).txt

Note that the above two snippets (for Case 2) will work assuming that your current working directory is the parent of directory X.

Case 3 :

Execute a command via ssh on a set of remote machines, whose hostnames are in lexicographic order. For example the CS department here at purdue has the following set of machines : 
sslab01-sslab21 and suppose I want to run the command "who" on each one of them.

for i in `seq 21`
    if [ $i -lt 10 ]; then
        echo "Running 'who' on sslab0$i"
        ssh sslab0$i "who"
        echo "Running 'who' on sslab$i"
        ssh sslab$i "who"

The if conditions in the above snippet are required because, by default the "seq"  does not output numbers of equal width, that is , it does not pad zeros at the beginning. However if you pass the "-w" option to seq, you can simplify the above snippet to : 

for i in `seq -w 21`
    echo "Running 'who' on sslab$i"
    ssh sslab$i "who"

The "-w" option ensures that `seq` prints its output in fixed width, padding the number with zeroes when ever necessary.

I will edit the post and add more common scenarios as I come across new ones, but all the above snippets are short and once you get hold of things, you can actually type them in on the prompt whenever you want rather than storing them in a script file and running that file. 

Adding Scripts to the PATH

However if you do want to run them as a script (from a file), it is a better option to do the following : 

1. if  ! [ -d ~/bin ]; then mkdir ~/bin; fi
2. touch ~/bin/script_I_run_often
3. Add the snippet to the script
4. chmod +x ~/bin/script_I_run_often
5. add "export PATH=$PATH:~/bin" to ~/.bashrc

Now you can just run the script as : 

$ script_I_run_often
Other Special Variables

Some other special variables which are very useful in everyday scripting are : 

$# : the number of arguments passed to the script, you can check this variable to see if your script got the correct number of arguments or not 
$1, $2, $3 ... : are the arguments passed to the script
$$ : Pid of the current process

NOTE : None of the information mentioned above is "new" or "novel", the post is just meant to serve as a compilation of some quick bash tricks. I am sure many other such compilations exist on the internet.