Linux Tips: overclock your command line skills with bash

This is just a quick tutorial on how to enhance your command line skills for those that are relatively new to Linux or the Unix world, and in particular, the bash shell.

A long time ago, when I was a Unix/Linux newbie myself, I met an old school emacs/bash dude named Kevin. The guy was one of those freak'n smart dudes that studied and did research in artificial intelligence and cognitive sciences and designed all sorts of software. I was pretty new to the Unix/Linux world and didn't quite "see the light" just yet, but Kev showed me some pretty cool stuff with the bash shell that made me say "wow! I want to be able to do that too!". So, here I am several years later, i figure i'd share it with other new comers too. And this being OCN, I figure there would be some interest in getting all the "performance" one can out of the command line.

Before I got into Linux and Unix, I use to think, "man, why do these people still use the command line? that's so old school when it's so much easier to use a GUI.." Well, the reason is that in many cases, especially if you know what you're doing (that's the key), using the command line can be much faster than pointing and clicking. Think about it; if your fingers are right over a key, how long does it take for you to hit that key, or combination of keys? Now, if you had to move your hand over to your mouse, move the mouse cursor over to some button and click on it, there's a lot more involved there. It requires hand-eye coordination, not to mention if you were clicking on a menu, you'd have to click a couple more times to get to where you want to go. So, the key to getting really fast at working on Linux/Unix is to be able to do as much as possible without having to move your hand away from the keyboard. And if you believe in this methodology, then bash has some really cool features that will turbocharge your command line skills.

Tip #1: just because it's a command line doesn't mean you have to type everything!

The first tip, and in my opinion, the easiest to learn is command line completion, some times also called 'tab' completion. Basically, whenever you are typing a command or a path to a directory or file, bash will help you complete the word you're typing if you hit the 'tab' key. If there are multple possibilities, and bash can't figure out exactly what you were trying to type, you can hit 'tab' twice for it to show you all the possibilities it knows of.

Here's an example, say I want to 'cd' into the directory /etc/sysconfig/network-scripts, this is where network configurations are found in Redhat/Fedora systems. Well, that path is 30 characters long! Instead of
typing all 30 characters, let's see how tab completion can help us. Let's start with:

(i've typed 2 characters of the path, now hit 'tab')

(bash figured out that there's only one directory that starts with 'e', so it automatically completes the 't' and 'c' and '/' for me. but I'm not done yet...)

(i start typing 's' and 'y' for 'sysconfig', and then i hit tab)

(again, only one directory starts with 'sy', so bash typed the rest for me.)

(i start with 'n' for 'network-scripts' and hit tab)

(bash helped me complete the word 'network', but it stopped short of finishing it with 'network-scripts' which is where i want to go. let's hit 'tab' twice to see why it didn't complete all the way)

(here, bash is telling me there are two directories that start with the word 'network' and it doesn't know which one i really want. well, i'll give it another hint by typing '-' and hitting tab)

That's exactly what i wanted! So this is what i actually would have typed:

/e <tab> sy <tab> n <tab> - <tab>

That's 10 key strokes that replaced the original 30! For Unix/Linux novices that watch me work on servers, they always ask me at some point, "how do you type so fast?!?!?" ... well, I tell them, "I don't, I let the computer do most of the typing, that's what they are for!"

The bash shell has all sorts of "completion" capabilities, like completing user names, host names, etc. But by far, the 'tab' key is the easiest and most useful.


Tip #2: if you've typed that command once, or something like it, don't type it all over again!

This tip is really about a feature in bash called command recall. The simplest form of this that is often discovered and used by beginners is the "up" arrow key. Open a command shell and type a few commands like 'ls', 'free', 'df', etc. Now hit the "up" arrow key and you'll see that your previously typed command is recalled for you to re-use or modify, should you choose to do so. Consecutive strokes of the "up" arrow key will recall even earlier commands you've typed in reverse history order. Using the "down" arrow key does the opposite and brings you back to later commands you've typed and eventually back to your blank command prompt.

Now, remember what I said about not moving your fingers away from the keyboard? Well, if you go for the "up" or "down" arrow keys, which usually reside to the right of the main alphabet keys, you've just added some latency by having to move your right hand over to the arrow keys and then back to the alphabet keys to continue typing. What a shame to waste such hand movement just to hit an arrow key! Here's how to accomplish the same thing even quicker! : Ctrl-P and Ctrl-N.

Ctrl-P (P for Previous) will recall the previous command. It is basically like using the "up" arrow.

Ctrl-N (N for Next) will recall the next command. It is basically like using the "down" arrow.

Now you can accomplish the same command recall without moving your fingers away from the alphabet keys allowing you to immediately resume your typing with little latency!

Let's do a simple example. Let's say i want to see a list of processes owned by the user "BLinux". So I might use a command like:

Now, let's say i want to know how many processes were listed from the above command. I can accomplish this by doing a "line count" with the command 'wc -l'. But, I don't have to type that command all over again and add the 'wc -l'. I can just hit ctrl-P, followed by "| wc -l"

This tells me that there are 64 processes being run by user "BLinux". Had I re-typed the whole "ps -ef | grep ^BLinux" part, that would be another 20 characters, on top of the " | wc -l" (8 characters) part. Command recall with ctrl-P just saved me from typing 20 characters and my fingers never left the alphabet keys so I could immediately type the additional pipe ("|") and line count ("wc -l").

Well, ctrl-N and ctrl-P are great, but it is really only useful if you're trying to recall one or two commands back. What if you wanted to recall that command that you typed 300 commands ago? What if you don't even remember how far back you have to go? What if you need that long command you typed yesterday? Bash has another way to recall previously typed commands, so long as they are in your command history buffer. (the default buffer length is usually 1000 entries) The magic key combination is ctrl-R.

When you hit ctrl-R, you get the following prompt:

from this point on, you can start typing a pattern or the beginning of a command you previously entered. as you type the pattern, bash will do a reverse search of commands that match the pattern you enter. you can keep typing more of the pattern to narrow down the search, or you can hit ctrl-R again to search further back using the current search pattern. Once you find the command that you are looking for, you can hit 'enter' to execute it, or 'esc' to stop the search and edit the command.

Examples are always useful. Let's say that 3 days ago, I mounted a CIFS windows share and have since umounted it. Today, I want to mount that share again, but not only was it a rather long command, I don't exactly remember the server name, the domain name, or user account that i used. *BUT* I do remember that it is one of the few commands i've typed in recent history that had 'cifs' in it. So, i'll do a ctrl-R reverse search for 'cifs':
You can see as soon as i typed 'cifs', it recalled the command i entered several days ago. Now, I can just hit enter and it will immediately run that command. With just 5 key strokes (ctrl-R, 'c', 'i', 'f', 's'), I have recalled a somewhat cumbersome command that is 97 characters long! This Ctrl-R form is probably the most powerful of all the command recall techniques.


Tip #3: Moving around the command line doesn't mean repeated use of the arrow keys!

Let's say that I recall the 'mount' command in the last example of Tip #2. But last time I connected as user 'BLinux', I couldn't access all the files and so this time I want to connect to that CIFS/SMB share as 'Administrator'. Well, I would have to change the user=BLinux part to user=Administrator. So, if after searching for 'cifs' and I hit 'esc', instead of running the command i recalled, i can now edit it and make changes before running it again. When i hit 'esc', i get this:

In the above line, i have put brackets [ ] around the 'c' to point out the location of the cursor. So, if I want to change the user=BLinux to something else, I would have to hit the right arrow key several times, or hold it down and wait for the cursor to move all the way to the end of the command so i can change 'BLinux' to 'Administrator'. That's a lot of arrow key strokes and a pretty long latency... and we know latency is our enemy! Not to mention, we don't want to move our fingers away from the alphabet keys!

Instead, bash gives us very quick ways to move around the command line to edit commands. Here are some of the key combinations that I use the most for cursor movement:

Ctrl-A : move cursor to the beginning of the command line
Ctrl-E : move cursor to the end of the command line
Ctrl-F : move cursor one character forward
Ctrl-B : move cursor one character backward
Alt-F : move cursor one word forward
Alt-B : move cursor one word backward

There are even key combinations to help you delete things faster:

Ctrl-d : delete the character under the cursor
Alt-d : delete the word starting from the cursor
Ctrl-k : delete everything from the cursor to the end of the line (kill the line forward)
Ctrl-u : delete everything from the cursor to the beginning of the line

So, with these in mind, we can use Ctrl-E to go to the end of line, Alt-B to move the cursor one word back, which puts our cursor on top of the 'B' in 'BLinux', and finally we can delete 'BLinux' with Alt-d. Then we can enter 'Administrator' and hit enter.

Using [] to represent our cursor, this is what it would look like:

a. we use ctrl-r to search for 'cifs' and hit 'esc' so that we can edit the command:

b. next, we use ctrl-e to get our cursor to the end of the line:

c. next, we use alt-b to move the cursor back one word to the beginning of 'BLinux':

d. next, we delete 'BLinux' with alt-d:

e. finally, we can enter 'Administrator' and hit 'enter' to run the command:

Notice how we can manipulate the command line without ever having our fingers leave the main alphabet keys which allows us to continue entering commands efficiently without any hand movement delays.

Tip #4: work in multiple directories with stacking

Ever have to make configuration changes in one directory, then have to go back to your home directory, and maybe even have to work in a 3rd directory? If so, do you ever get tired of 'cd' back and forth? Even if you use 'tab' completion or command recall to make your 'cd <path>' commands relatively quick, it can get annoying.

To make life easier in this situation, bash has a feature called 'directory stacking'. It's very simple, and there are only three commands you really need to know to use this. But, before I get into that, since not everyone will be familiar with the concept of a 'stacks', which is also known as a LIFO (Last In, First Out), let me start by explaining the 'stack' concept.

You can think of a 'stack' like a stack of dishes. When you put a new dish onto the stack, that dish becomes the top of the stack. When you want to remove a dish from the stack, you have to remove the top dish first (hence: last in, first out) before you can get to the dishes underneath it. If you wanted to get at a dish that is somewhere in the middle of the stack, you have to remove all the dishes on top of that dish first before you can get to it.

Now, instead of 'dishes', imagine that you are stacking up directories. Here are the commands you need to know to use directory stacking:

dirs : list your directory stack
pushd : push (add) a new directory onto the stack
popd : pop (remove) a directory from the stack

Here's an example. Let's say I want to be able to work in my home directory, the /etc directory, and the temp directory /var/tmp. We start out in my home directory, /home/BLinux, and i'll also list my stack with 'dirs':

Now, let me add /etc to my stack with 'pushd':

You'll see that we've moved into /etc, it is now the top of the stack (left most entry in the listing), and the command automatically lists my stack as "/etc ~" which is /etc and ~ (my home directory). Next, let me also add /var/tmp to the stack too:

Now, i'm in /var/tmp, which is the top of the stack, with '/etc' underneath it, and '~' at the very bottom. Now that i have all 3 directories in my stack, I can move around to the different directories with the 'pushd' command. Lets say I want to swap the top directory with the one underneath it, i simply use 'pushd' with no arguments:

Now, what if I want to jump two directories down my stack to my home directory (~)? Easy, just tell 'pushd' that you want to go to jump 2 directories down:

Next, let's say that I don't need my home directory (~) in my stack anymore. I can remove it from the stack with the 'popd' command:

You can see that the home directory entry (~) has been removed, and I've ended up on the next directory in the stack, which is /etc.

In this way, you can stack up all the directories you want to work in, and be able to move between them with relatively simple commands instead of having to type long path names.

Tip #5: just because you have one command prompt doesn't mean you can only run one program at a time.

This is about a feature called job control. For many who are just learning how to get comfortable on the command line in Unix/Linux, it is often assumed that you can only run one program at a time and you have to wait for the current running program to complete and return your prompt before you can execute the next command. However, the truth is that you can run as many commands as you want from a single command prompt. Here's how:

background jobs: if you append '&' to the end of a command, that command will run in the background and your command prompt will return to you immediately for you to issue the next command even if the 1st command is still running.

For example, let's say you want to delete a directory with a million files. Normally, this operation might take several minutes, but you don't want to sit around waiting for it to complete. So, you run the program in the background like this:

As you can see, the command prompt returns immediately. And when i run 'jobs' it shows that the 'rm' command is still running (in the background). So, now you can run programs in the background and not be held up by them. This is really useful for programs that might take a long time to execute like compressing a large file. It's also good to know about the 'jobs' command, as we'll use it some more later on.

Now, what if we didn't initially know that a certain command was going to take a long time and didn't run it with '&' appended? You can still put a program into the background even if you didn't initially do so. First, you have to send a STOP signal to the current program and get back to the prompt. This is done by hitting Ctrl-Z. When you hit Ctrl-Z, this stops the current program (it is frozen, no longer running) and returns you back to the prompt. Next, since we don't want to freeze the program we were just running, we just want it to continue running in the background, you use the 'bg' command. The 'bg' command will unfreeze the program and continue running it in the background.

Let's look at our compression example. Let's say we started compressing a large file without knowing that it was a really large file. We'll use Ctrl-Z to stop the program can get back to our command prompt:

As you can see, the bzip2 program is stopped. Let's make it continue in the background with 'bg':

You can see that the bzip2 is now running in the background. But wait, didn't we say we could run "as many commands" as we want from one command prompt? Yes we can:

As you can see, we now have two programs running at the same time, and I have my prompt back to run others. We now know how to run several programs in the background, but what if we want to also be able to interact with more than one program at a time? When you send a program to the background, you can't really interact with it anymore. The way around this is the 'fg' command, or foreground command.

Here's an example; let's say you're downloading some files via FTP using the standard ftp client, meanwhile, you also want to take notes of what you're doing in a personal log file called personal.log. You're going to have to interact with the ftp program, but maybe while it is downloading you want to write entries in your personal log.

We start this example by opening our personal.log in vi. After making our first log entry, we freeze vi to start our FTP session.

After we login to the FTP site and start downloading a file, say an ISO image of Fedora 11, we can return to our command prompt:

In the above, we started a download of Fedora 11, but we didn't want to sit around and wait, so we froze the FTP program with Ctrl-Z. But we don't want to stop the download, so we must let the program continue running in the background. We accomplish this with 'bg %2'. The %2 here designates job #2, which is the ftp program, where as job #1 is our vi session editing personal.log.

Now, let's say while the ftp download is happening, I want to write a few more entries in my personal log, which is already opened in the vi session i started at the beginning. We know the vi session is job #1 (the 'jobs' command tells us so), so we can bring that job back into the foreground with 'fg %1':

Let's say we want to be back in our FTP program now, we can do that by foregrounding the FTP job:

So, in this way, we can run many programs, freeze them (Ctrl-Z), run them in the background (bg), or recall them into the foreground again (fg) so long as we can reference their job numbers, which is displayed when we run 'jobs'.

It is indeed possible to be very productive on a single command prompt, while download files in the background and doing other things like compressing files and writing a journal log.

TIP #6: install some pre-programmed completion customizations from "bash-completion"

I should have added this before but working in the server world, I'm not always allowed to install whatever i want on my client's machines so I forgot about this. The 'bash-completion' package provides a bunch of pre-programmed completion rules so that bash automatically recognizes what should follow certain commands. It covers a LOT of stuff and I've found it to be very useful where I am able to use it. It can be found here:

http://www.caliban.org/bash/

Well, that's it for now. Once you master these skills, i can promise you that you'll be able to work faster on the command line than you ever will moving a mouse cursor around and clicking buttons and menus. Just remember, the key is to learn how to keep your fingers on top of the alphabet keys as much as possible without having to move your hand around. Once you've mastered doing things quickly on the command line, you'll actually start getting annoyed when you *have to* use the mouse for certain things.
Edited by BLinux - 4/23/11 at 3:34am

Thanks guys for the thumbs up! There was actually more I wanted to write about, but ran out of steam. I may follow up and add more later. If you guys have any suggestions/feedback, i'm open to hearing them too!

Thanks guys! I've added a 4th tip about directory stacking.

Yup, the history buffer is written to a file in your home directory named .bash_history when you exit the bash shell. When you fire up the shell, it reads in that file so your command history is retained. This is a very useful feature, but also a potential security problem, so beware. For example, if you type a command that includes a password in the argument list, that password gets recorded in your history file. So, someone who gets access to that file can potentially find your passwords revealed. If you want to make sure that such commands never get written to your history file, you can symbolically link your history file to /dev/null like:

# ln -s /dev/null ~/.bash_history

Of course, if you do this, your history buffer is not retained between bash shell invocations.