Non-interactive, background computing

How to multitask from the command-line interface

Though the command-line interface seems very primitive, keep in mind that it is only the interface to a very modern operating system, one that has all of the niceties that you've come to expect in contemporary computing.

One of those features is background processing – e.g. how you can listen to a song on Spotify while texting a friend, and while waiting for the new version of Angry Birds to download. The command-line interface kind of obscures the multitasking of the modern Unix-like system, because oftentimes, we're doing commands one at a time.

In this tutorial, we'll go over some of the basic ways to multitask from the command-line, something which is necessary when running time-intensive programs.

Quick tip: Much of this tutorial deals with some details of the Unix operating system and some of what we have to be aware with on Stanford FarmShare.

The main takeaway is this: If you want to run a long-running script on `` so that it keeps going, even for a full week after you've logged off, run your command like so:

     nohup krenew -t -- your_command_here

Now, if your program is well-behaved, you should be able to start it off and then logout for a peaceful night's sleep. However, if you want to know such details like, how to find and _kill_ off a program that has run amok, you should probably keep reading.

Send a job to the background

The sleep command will suspend program execution for a given number of seconds, minutes, hours or days. To "sleep" for 2 seconds, run:

sleep 2

Executing this from the command-line will cause the interface to seemingly freeze. We can either wait 2 seconds or hit Ctrl-C to kill the sleep process.

When executing a one-line command, appending an ampersand will tell the command-line interpreter to send the process to the background. The following command will "sleep" for 1000 seconds, but by using the ampersand, the sleep process will be sent to the background, and we'll continue to be able to execute commands as before:

sleep 1000 &

Executing the command should return something like this:

[1] 28893

That number – in my example, 28893 – is the process ID. After the process ID is printed to screen, we should be returned to the prompt. So…did anything happen? Did we actually go to sleep? Since sleep has no effects except to pause execution, and since it's now been sent to the background, it's as if running sleep 1000 & had no effect.

In fact, even though there is no noticeable effect, a sleep process has been spawned. Run the following commands to see the jobs that are running in the background:

jobs -l
[1]+ 28893 Running                 sleep 1000 &

Let's spawn off a few more sleep processes, then run jobs -l again. Here's what the sequence looks like:

dun@corn33:~$ sleep 10000 &
[2] 29163
dun@corn33:~$ sleep 4000 &
[3] 29166
dun@corn33:~$ sleep 300 &
[4] 29167
dun@corn33:~$ jobs -l
[1]  28893 Running                 sleep 1000 &
[2]  29163 Running                 sleep 10000 &
[3]- 29166 Running                 sleep 4000 &
[4]+ 29167 Running                 sleep 300 &

Kill a background job

Those sleep processes don't seem to have any negative effect. Still, if you're a bit OCD, you may not want them to just be "running". This is where the kill command comes in.

The jobs -l command lists your current running jobs and their process IDs. The kill command will "kill" jobs based on process IDs.

kill 29167

Running jobs -l again will show that the job in question has been "terminated":

dun@corn33:~/comm213/tmp$ jobs -l
[1]  28893 Running                 sleep 1000 &
[2]  29163 Running                 sleep 10000 &
[3]- 29166 Running                 sleep 4000 &
[4]+ 29167 Terminated              sleep 300

Running a script from the background

For this next step, create and change into a throwaway directory (i.e., somewhere in /tmp)

Let's create a process that, unlike sleep, actually has an effect. The following loop will create 100 empty files in the current directory:

for n in `seq 1 100`; do
  touch "$n.stuff"

Run it once, then do a rm -f *.stuff (you are in an otherwise empty directory, right? Because if you aren't…) to clean up the directory.

Now create a new script named and use the same loop as above, but with a call to sleep:

for n in `seq 1 100`; do
  touch "$n.stuff"
  sleep 3

Then execute the script – in the background – from the command-line:

bash &

Every three seconds, if you run the ls command, you should see a new file in the current directory. Run jobs -l to confirm that bash & is indeed running in the background.

What happens upon logout

Now, instead of killing the script, log out. And look at what machine you are on. For example, if your prompt says your_sunet@corn33, you'll want to ssh directly into that machine again (otherwise, will take you to a random machine):


After you log in, change back into whatever directory you ran in. List the files and take note of how many have been created. Then run jobs -l to see if bash & is still running.

The jobs -l should return an empty queue. And if you run ls again, no new files should have been created. What we might infer is that logging out will terminate our jobs that ran in the background.

However, this may or may not be the case. Run this command to list processes that involve "vanwinkle":

ps aux | grep ''

You might see the bash process still running. However, it's likely ineffective, hence, why no new .stuff files are being created.

Using nohup and krenew

There are two things that might have happened to our background process:

  1. Logging out might have sent it a "hangup" signal, causing the process to stop.
  2. Logging out invalidates your authentication credentials on; so bash is still running, but it loses the ability to access your file system (i.e write files to disk)

Either way, this is the way to keep a task running even after you've logged off. I've split the single command into several lines using the backslash operator for easier reading:

nohup \
  krenew -t -- \
  bash &
nohup krenew -t -- bash &

Running the command above will produce this notification:

 nohup: ignoring input and appending output to 'nohup.out'

This simply means that whatever your program is outputting to screen, if anything, will be in the file named nohup.out (which you can cat.

You should now be able to log out of your current session, log back in, change into your working (junk) directory, and see that the .stuff files are being created.

Finding and killing a program

Shorthand version:

If you are doing this on one of the computers, you need to be on the same machine as the one on which you started the process (i.e. a machine with a specific number, like corn33).

To list all of the current processes/jobs, run:

ps aux 

This will show you all the jobs, so just grep for something that matches the script's name that you just ran. Or, grep for your SUnet ID.

The second column will contain the process ID of the job. If, for example, the process ID is 5599, this is how you kill it:

kill 5599