BA 372 - Linux Lab

This lab contains exercises that familiarize you with the Linux command-line environment. It is loosely based on two labs developed by Hanne Munkholm of the IT University of Copenhagen. Following the IT University licensing of this material, the materials in this lab are copyright (c) 2007 by Dr. René Reitsma. This material may be distributed only subject to the terms and conditions set forth in the Open Publication License, v1.0 or later (the latest version is presently available at http://www.opencontent.org/openpub/).

Linux & ONID

All OSU students have access to a Linux command-line interpreter through their ONID account and ssh (Secure SHell), a terminal protocol which--through encryption--protects sensitive information such as user id's and passwords when exchanged between machines.

If you want to use ssh to access your ONID/Linux account from home, download and install an ssh client from the Web (Windows users might look at http://www.openssh.com/windows.html) or install it from the ORST cd.

In the COB labs, start your ssh client session from All Programs --> Remote Connection --> SSH Secure Shell --> Secure Shell Client --> Quick Connect.

Connect to the ONID ssh server at shell.onid.oregonstate.edu

Use your ONID login and password to login.

You are logged in when you see the command-line prompt: shell:~>

The command line was the primary user interface to computers, prior to the window-based point-and-click graphical desktops came along. You interfaced with the computer by typing commands on a text terminal, that way telling the computer which programs to run, and how to behave. Most Linux distributions nowadays offer a full line of pick-click-and-drag user interfaces, yet most Linux users remain very fond of their command-line and use it a lot.
Why do we still use the command line today? Because it is much more efﬁcient for certain things than a point-and-click interface. Also, the Unix/Linux command lines are very powerful and allow complex actions to be carried out with just a few commands. However, command lines do take a bit more effort to get started; you have to learn some commands and you have to master the way in which these commands can be connected into chains that do good things. Nevertheless, when you have learned a certain amount of basic commands, and you are getting the hang of it, you will get a control and understanding of your computer that you usually don’t get in the point-and-click world. It’s your choice. There will always be more to learn...

The program that is displaying the command prompt and that interprets the commands you type, is called a shell. Many different shells exist for UNIX/Linux, and they have some differences. Your default ONID/Linux shell is the so-called cshell (csh). Another, well-know Linux command shell is Bash or the ”Bourne again shell” (named after the older Bourne shell).

Notice that the shell's prompt is set to shell:~>. You type in commands after the prompt, hit enter/return and the shell runs the commands.

The first command that we often use when logging into a computer for the very first time is a command that tells us what operating system we're running:

uname -a

The result: Linux shell 2.6.19.2-osu-2 #1 SMP Tue Jan 16 15:38:34 PST 2007 i686 GNU/Linux shows that:

We are running Linux with kernel release: 2.6.19.2-osu-2.

The kernel version: #1 SMP Tue Jan 16 15:38:34 PST 2007.

That we are running on a (32-bit) Pentium (i686).

That we are running the version of uname developed at the Free Software Foundation (GNU/Linux).

Notice how in the above example uname is the command and -a is a so-called command option. In this case, -a is the option which will give us all available information from the uname command. Most Linux commands have one or more options.

To see which other options are available for the uname command and to get information on the uname command in general, use the man (for manual) command:

man uname.

Notice that the manual page gets displayed in a 'funny' display mode (you can scroll up and down, but how do you get out? Use q to get out.

Pretty much all Linux command are supported by a so-called man(ual) page.

Let's look at some very basic shell commands:
- ls
- pwd
- At this point we must say a few things about the file system:
- Let's explore the file system just a bit:
Some more commands:

You can list the contents of text files using the more command; e.g.

more /etc/passwd

more /etc/services

Notice how you can go backward and forward in the file with control-f (forward) and control-b (backward). Get out with q.

See how many and which CPU's we're running:

more /proc/cpuinfo

Ah, not bad; four 2.9GHz Xeons.

All Linux command shells provide a few handy services to make typing commands faster (Try these out!!):

The up/down arrows take you back/forward in the command history.

When typing a command, you can enter the command with the Enter/Return key even if your cursor is not positioned at the end of the command.

You can rerun any previous command by recalling it from the shell's memory. It goes like this:

!more

runs the last command that started with more.

runs the last command that started with cd.

runs the previous command.

If you ran a (complex) command a while ago but you don't remember it and you don't want to go back through the entire command stack one command at the time to find it, select it from the command history and run it; e.g.,

history

shows you a numbered list of previous commands. To rerun one of them use !<number>; e.g.,

reruns command 12.

To see only the last n commands you entered use history <n>; e.g., history 5 to see the last 5 commands..

Typing (long) filenames can get tedious and takes time. Hence, the Linux shells offer so-called file name completion. Start typing the name of a file--e.g., as part of a command--and after typing the first few characters of the filename, ask the shell to complete it for you by hitting the <tab> key; e.g.,

more ~/.pub ...followed by a <tab>

Notice how the shell resolves the whole filename for you.

Some basic file management:

First return to your home directory:

Let's create a directory called foo:

mkdir foo

touch foo/mytest

creates a(n empty) file named mytest in the directory foo. Check if it's there with

ls foo

cp foo/mytest test_in_home

copies foo/mytest into test_in_home.

Notice how test_in_home sits in your home directory (that is, if you are running the cp command from your home directory).

Also notice that we don't want to use spaces in file and directory names (use underscores (_) instead). Technically you can use spaces, but it's a REALLY BAD IDEA!! DON'T DO IT!!

cp foo/mytest .

copies foo/mytest to your current directory ("." stands for 'current' or 'this' directory).

rm mytest

removes mytest from the current directory.

Check if the file was removed with

Notice how the rm command did not ask you to confirm the remove; it just went ahead and removed the file.

If you want rm to ask for confirmation first, use the -i (interrogate) option:

rm -i foo/mytest

Answer with y to remove, with n to keep it.

Make sure that you have a file called mytest in your home directory. If you don't have it anymore, create it again like so:

touch mytest

mv mytest foo/newtest

renames mytest to foo/newtest.

Check the result with ls foo.

Remove the foo directory:

rmdir foo

Notice how the shell complains that the foo directory is not empty and hence, cannot be removed.

You now have two options:

Remove the files inside foo first, then remove foo (cumbersome, but it will work).

Try removing foo with the rm command:

rm foo

Again, the shell complains about foo not being empty.

OK, then use the -r option with the rm command to recursively remove all files in foo; i.e., to descend into the subdirectories of foo and into all of their subdirectories, etc., so as to remove the entire contents from foo, and then remove foo itself:

rm -r foo

Check the result (ls)

Of course, since we did not invoke rm with the -i option, it did not prompt us for confirmation.

To do this, simply combine the -r and -i options into -ri:

mkdir foo

touch foo/mytest

ls foo

rm -ri foo

File access privileges:

cd to your home directory and make a file mytest there:

touch mytest

Whereas the ls command shows a listing of files in the current directory, ls -l shows you an extended list of files as well as some of their attributes:

ls -l

You should see something like this:
drwx------ 2 your_login students 96 Sep 12 2002 mail
-rw-r--r-- 1 your_login students 0 Feb 23 16:24 mytest
drwxr-xr-x 2 your_login students 96 Feb 14 16:27 public_html

For each file you see (from left to right):

The file access permissions: we'll get back to these in just a second.

The number of nodes in the file (this is a technical detail that you can ignore for now).

The file's owner (your_login).

The work group to which the owner belongs (students).

The number of bytes in the file (notice how mytest is empty; try a more on it).

The date and time the file was last modified.

The name of the file.

Of this information, the file access privileges on the left (e.g., drwxr-xr-x) require some explanation:

First, the 'd' on the left side of two of the mail and public_html entries indicate that those are in fact directories (remember, in Linux, everything is a file, so a directory is just a special type of file).

Second, access privileges in Linux come in three types:

read (r) privileges, write (w) privileges and execute (x) privileges.

These privileges are distributed over three types of users:

The file's owner.

The members of the file owner's work group; aka 'the group.'

All other logins; aka 'the world.'

For each of these user types, each of the three access privileges is specified. Hence, for the file mytest:

The owner has read and write privileges but not the execute privilege (rw- on the left).

The owner's group members have read privilege but no write or execute privilege (r-- in the middle).

All the rest of the logins have read privilege but no write or execute privileges (r-- on the right).

For the directory public_html (a directory):

The owner has read, write and execute privileges (rwx on the left).

The owner's group members have read and execute privileges (r-x in the middle).

All the rest of the logins have read and execute privileges (r-x on the right).

Note that in order to cd to a directory, it must have the execute privilege switched on.

Obviously, all these privileges can be granted and revoked through a command (chmod), but we'll skip that for now.

Some more advanced concepts;

To see how handy a command line can be, let's send ourselves an email:

mail your_email_address

subject: type-in-a-subject

type a short email; just a couple random keys will do.

control-d

cc: control-d again.

Now go check your email (give it a minute or so).

Not convinced? let's email ourselves the listing of all files in our home directory:

ls -l | mail your-email-address

go check your email.

This last command reveals some of the considerable power of UNIX and Linux command shells and the operating system as a whole. The '|' symbol is a so-called pipe. What it effectively does is hand over the results of the ls -l command to the mail command.

Let's apply this principle to a few commands. First the commands themselves:

ps -ef

shows a list of all processes running on our machine (their names, owners, when they got started, etc.) .

wc /etc/services

counts the number of lines, words and characters in the file /etc/services.

Now let's use ps and wc in combination with a pipe to see how many processes we're running:

ps -ef | wc

takes the results of the ps command and pipes these into wc. Once again, everything in Linux is a file, even the output of a command.

We can, of course, continue this chain: let's find out how many of these processes are ours.

ps -ef | grep your_login | wc

Explanation: the results of ps -ef are piped into the grep command that is used to find any line with the string your_login in it. The results from that grep are then handed over to wc.

The grep command, can of course be run on the output of any other command; e.g.

which files in my home directory have html in their name:

ls | grep html

The latter, however, can also be accomplished just using the ls command:

ls *html*

The asterisks are wild cards that match any character string.

Sort all processes running on this machine by owner:

ps -ef | sort

How many people (logins) are currently logged onto this machine?

who | wc

How many processors are we running again?

more /proc/cpuinfo | grep processor | wc

Yup, four!

Notice how much you can have done with just a few, cleverly constructed commands.

Not only can we pipe the output from one command into the input of another, we can also redirect inputs and outputs:

For instance, let's assume we want to store the output of ls -l in a file called foo:

ls -l > foo

Now let's mail ourselves the contents of foo:

mail your_login < foo

Give it a minute and go check your email.

Alternatively; you can separate the two commands with a semicolon and type them as one command:

ls -l > foo; mail your_login < foo

Let's control (start & stop) our processes:

Linux gives you full command-line control of all your processes.

Start a second ssh session (you should have two ssh sessions open on shell.onid.oregonstate.edu)

In one of the shells, start a more process:

more /etc/services

Notice that more displays about 9% of the services file on its first page.

Leave more running in this window.

In the other shell, inquire about the processes that we are running:

ps -ef | grep your_login

You'll see entries like the following (your_login instead of reitsmar):

root        9033 2091 0 09:38 ?        00:00:00 sshd: your_login [priv]
your_login 9039 9033 0 09:38 ?        00:00:00 sshd: your_login@pts/79
your_login 9040 9039 0 09:38 pts/79   00:00:00 -csh
your_login 31419 9040 0 17:39 pts/79   00:00:00 more /etc/services
your_login 31496 9040 0 17:42 pts/79   00:00:00 ps -ef
your_login 31497 9040 0 17:42 pts/79   00:00:00 grep your_login

Notice how your_login is (was) running the following processes:

csh (the cshell itself).

ps -ef: the ps -ef command you just ran.

grep: the grep command you just ran.

more: the more command you're currently running in the other terminal window.

Notice how each process has two numbers associated with it (listed immediately to the right of the process owner):

The number on the left is the so-called process id or pid of the process.

The number on the right is the pid of the process's parent process (notice how the grep and ps -ef processes have csh as their parent).

Now, from this shell (the shell not running the more), let's kill the more process running in the other shell:

kill pid_of_the_more_process (kill 31419 in the above example)

Notice what happened to the more process in the other terminal window.

Of course, this sort of process control can all be done with a graphics interface as well, but especially when logging on to servers, one often does not have or need a GUI and a quick series of typed commands is often significantly faster than picking and clicking our way through life.

Often command-line users do not want to have to go through the hassle of starting a second shell in order to issue some commands while a command in the first shell is still running. Nicely enough, in Linux we can temporarily stop a process, run some other processes and resume the previously stopped process:

Close one of the open shells:

control-d

In the remaining shell, start a more process:

more /etc/services

Now stop (not cancel!!) this more process:

control-z

Notice how the more process has 'disappeared' and we got our shell prompt back.

Let's see what the status of the more process is:

ps -ef | grep more

Notice how the more process still exists; we stopped it; but it's still there.

We can also see the actual status of the more process:

jobs

Notice how our more job is Suspended.

So at this point, we can enter new commands, start new shells, etc. while all along our more process is kept in suspended animation.

Now let's bring it back to life:

Notice how more is once again our current process.

There is no limit to how many processes we can start, stop and revive.

A slightly different situation occurs if we start a program that takes some time to run, yet we want to do some other work while the program is running. Of course, we could leave the running program sit in the one shell and start a new shell and issue commands there. But often we don't want to bother starting a new shell. Instead, we just want the original program to run and complete in the background (note: background running ≠ stopping) while we get our command line prompt back so we can continue working on something else.

Type the following command and note how it takes a while to complete (it will try 30 so-called hops before it terminates. If you get tired of waiting, quit it with control-c):

traceroute www.gelderlander.nl

So, rather than wait for the traceroute to complete, why not let the process run in the background, dump its results to a file foo, while we get our prompt back in the mean time:

traceroute www.gelderlander.nl > foo &

Now see the status of your jobs:

jobs

And notice that the traceroute is Running (not Suspended) in the background while you got your command prompt back.

Since Linux users are so fond of their command shells, a great many very powerful commands are available. Here's just a tiny sample:

Get the COB's web page and write its contents to the file foo:

wget -O foo www.bus.oregonstate.edu

Check the results:

more foo

Get the HTTP headers sent back from the server and print them to the screen. Dump the contents of the web page itself into oblivion (/dev/null):

wget -S -O /dev/null www.bus.oregonstate.edu

Note: /dev/null is known as the 'bit bucket' or 'null device.' Anything written to it 'disappears;' it's the digital garbage can that never fills up.

Find all files called passwd anywhere in the /etc directory or any of its subdirectories:

find /etc -name passwd -print

Display the first 10 lines of the file /etc/services:

head -n 10 /etc/services

Display the last 10 lines of the file /etc/services:

tail -n -10 /etc/services

Command-line file editing with sed:

sed is a complex and powerful command. You can edit a ﬁle 'on the ﬂy' from the command line, e.g., replacing one word or expression/pattern with another. Here's a simple example of using sed:

Let's make a ﬁle, linux1.txt, containing some text.

echo "A cow in the stable in the summer" > linux1.txt

echo "in the middle of June when all the" >> linux1.txt

echo "other cows are out in the field." >> linux1.txt

Notice how we use the combination of the echo command and file redirection (> and >>) to generate contents of the linux1.txt file (>> appends to a file).

Check the result with:

more linux1.txt

Use sed to replace every occurrence of the word cow in the file linux1.txt with computer, and save the result in linux2.txt.

cat linux1.txt | sed ’s/cow/computer/’ > linux2.txt

Notice how this uses a combination of the cat command (a kind of more), a pipe (|), a sed text replacement (s for substitute) and a file redirect (>).

Check the results with

more linux2.txt

This is the UNIX/Linux command shell in action!

Let's see all logins, sorted alphabetically, that are running processes on this machine. I only want to see the logins, not the rest of the ps -ef information. I also want each login to appear only once, regardless of how many processes it's running:

ps -ef | cut -d" " -f1 | sort -u

Explanation:

ps -ef: see above.

cut: command to select 'columns' from a text file:

-d" ": use the space (" ") as the delimiter of columns.

-f1: grab the first column.

sort -u: sort uniquely (remove duplicates).

Notice again how this little bit of command line does a significant amount of relatively complex work by chaining simple commands such as ps, cut and sort with pipes.

For the not-faint-of-heart:

Ever wanted to see the 'mysterious' binary code of an executable? Well, here's an impressive Linux command that does just that (courtesy of Yousef Alhashey). It uses some of the facilities we have described... plus some not described:

od -b --width=5 filename | perl -ne '@a=(0,1,10,11,100,101,110,111); @s = split(/ /, $_); print "$s[0] "; foreach(1..$#s) { @c = split(//, $s[$_]); foreach $c (@c) { $s .= sprintf("%03d", $a[$c]) } print substr($s,-8) . " "; $s = "" } print "\n"'

To run this, replace filename in the above command with the name of a binary file; e.g., /bin/ls and see what happens.

Notice that the above code consists of only two commands connected with a pipe:

The od command which allows you to dump files in various formats; e.g., octal, hex, etc.

A perl script that processes the output of the od command. It takes the octal bytes coming from od -b and converts them into binary.

If, at this time, you suspect that the Linux command shells are actually very powerful devices, you'd be right. In fact, the command shells support an entire programming language (for/while loops, if-then-else, etc.). Linux system administrators often use these facilities to write programs on the shell level that execute a variety of more or less complex OS-level operations. This sort of programming is known as UNIX/Linux shell scripting or shell programming.

To get just a feel for this shell scripting business, let's look at just one example of a shell script:

Grab the script from the course page:

wget classes.bus.oregonstate.edu/ba372/labs/shellscript.sh

Check to see what it looks like:

more shellscript.sh

Explanation of the script contents:

#!/bin/sh
Start a bash shell script (note that bash has a somewhat different language than csh. csh is not a great shell for scripting. Hence, many programmers use bash instead).

# My first shell script.
# Tells how much space my home directory occupies.
Comments

echo "Calculating disk space"
echo ""
Write some text to the screen

cd
cd to my home directory

USAGE=`du -sh ./ 2>/dev/null | cut -f1`
Set a variable USAGE to the amount of space used in the current directory (du is a command that reports on disk usage; cut is used to select the first column returned by du).

Notice the `back tics` around the du | cut command. These indicate that the results of the part of the command beteween the ticks must be computed before the right hand side of the assignment is assigned to the left hand side (if you leave them off, the du | cut command itself rather than its results gets assigned to USAGE).

echo "My homedir occupies $USAGE".
echo "" Do some more writing to the screen. Notice how the USAGE variable that got a value assigned in the previous line is now referenced using the $ sign.

Now make the script into an executable file (notice that although shell scripts are text/ASCII files, they can be made executable):

chmod +x shellscript.sh

Check what the chmod command did with ls -l

Now run the script:

./shellscript.sh

You may have wondered why you had to use wget to download the script file. Why not just cut and paste the text from this Web page into a text file? Well, text editing in UNIX/Linux, especially through a command shell, requires some more learning:

ssh command shells can be started and accessed from any UNIX/Linux machine in the world. However, they are keyboard- rather than mouse-based tools. Hence, they do not provide a graphics-based, pick-drag-and-click text/file editor. Although such editors are available when you are at the graphics-based console of a UNIX/Linux machine or when you are a machine running a so-called X-server, most Linux administrators do not rely on graphics-based editing. Instead they use keyboard-based editors. A variety of these editors are available but because they are all keyboard- rather than mouse based, they are all a little tricky to learn. On the up side, just as with Linux command shells, once you master one of them, you are generally a lot better (i.e., faster) off using one of them than using a pick-and-click-based graphical editor. In addition, with a keyboard-based editor you can work with any Unix/Linux machine anywhere in the world.

The three most used Linux text editors are vi, emacs, and to a (much) lesser extent pico.

Users of especially vi and emacs staunchly defend their choice and pretend loathing the other alternative (it's a cultural thing).

But really, vi or emacs are both fine. If you're serious about wanting to learn UNIX or Linux, pick one of them and live with it.