Get acquainted with the Unix shell on Stanford’s FarmShare and set up accounts on Github and Twitter so that we have a (remote) place to work and a usable data source.
Github gives us a convenient place to store, backup, and publish our work. Sign up for the free plan. Apply for the Github Education Pack.
This is where you will submit homework assignments for the class. If you have already been given the Github Education Pack, make the compciv repo private.
If your Github username is hello, the URL for your repo will be github.com/hello/compciv. Also, if you can figure it out, give my Github account (dannguyen) read access to your compciv repo. (if you can’t figure it out, we’ll go over it in class).
Set up password-less access to Github from corn.stanford.edu via a SSH key-pair.
Register at twitter.com; you can use an existing personal Twitter account if you have one. Then follow at least 50 other users.
From the corn.stanford.edu command-line, send me an email to dun@stanford.edu with the following:
~/.ssh/compciv_id_rsa.pubIf you've never done this before, this will probably be the most seemingly arbitrary and confusing set of steps that you'll have to take for this course. But they are necessary for setting up your the working environment for the rest of this course, and also serves as an introduction to remote/cloud computing.
But there's nothing here meant to trick you. Just follow each step one at a time and please contact me or post on Piazza when you run into problems.
Creating a Twitter and Github account are straightforward enough.
However, working from corn.stanford.edu and setting up the SSH keys may be new to you, so follow these steps.
I've listed every command here: if you're completely new, read the commands and re-type them yourself (don't copy and paste). Just the physical act of typing helps you get a feel of how things work, even if you don't have any clue as to what the commands are doing. That's OK, this is the arbitrary and confusing part of the course.
You have a less direct path to logging in. Follow these instructions by Stanford's IT department on Downloading and Configuring SecureCRT 7. At the end of the instructions, you should be able to log into corn.stanford.edu
You should see a prompt (the same as for the OS X users) that looks like this:
dun@corn01:~$
In your Finder, open the folder at Applications > Disk Utilities and double-click the Terminal app. Then type (using your own SUnet ID):
ssh YOUR_SUNET_ID@corn.stanford.edu
You'll receive a response like this:
The authenticity of host 'corn01.stanford.edu (171.67.216.66)' can't be established.
RSA key fingerprint is 0b:e7:b4:95:03:c1:1e:07:df:04:ca:a2:3d:8e:e3:37.
Are you sure you want to continue connecting (yes/no)?
Just type in yes (for yes) and hit Enter. You will then be asked for your password. Type it in (it'll seem like nothing's happening, but that's because the Terminal is obscuring your password) and hit Enter.
At this point, you should see a prompt that looks like this:
dun@corn01:~$
Note: We may not need to actually do this SSH key thing (it's cumbersome on corn.stanford.edu). Read the guide on /recipes/devops/git-and-github-setup for more details, particularly the section about the "access token"
At this point, the instructions are similar to what you'll find on Github…but not exactly. So keep following along here.
At the prompt, type the following commands in and hit Enter after each one:
mkdir -p ~/.ssh
cd ~/.ssh
Note: If it didn't seem like anything happened, i.e. a confirmation or error message, then you're good. This is a consequence of the Unix Philosophy tenet, "Rule of Silence":
Rule of Silence: When a program has nothing surprising to say, it should say nothing.
Type the following, substitute your SUnet ID (i.e your email), and hit Enter:
ssh-keygen -t rsa -C "your_sunet_id@stanford.edu"
You'll see this output:
Generating public/private rsa key pair.
And then you will be given a prompt that looks like this…don't hit Enter just yet:
Enter file in which to save the key (/afs/.ir/users/d/u/dun/.ssh/id_rsa):
At the prompt here, type in:
compciv_id_rsa
Then hit Enter. This compciv_id_rsa string is just the filename for the key.
You'll then be asked for a "passphrase" (i.e. a password). This is optional but I recommend specifying a passphrase.
You'll then see this (and a few other lines of code you can ignore)
Your identification has been saved in compciv_id_rsa.
Your public key has been saved in compciv_id_rsa.pub.
The file with the .pub extension – that is your public key. You can generally share this with the world, and in fact, you are going to share it with Github in the next step.
Let's print it out to screen:
cat compciv_id_rsa.pub
The output will look something like this:
ssh-rsa AAAAB3NzaC1yc2EAAAAXbZuyQIh/g9+LjVb39hjRm+HNgcurEUiRbPdcdMTB2YhReiPTDAQABAAABAQDSmc8DCOkHaloYGOounfjEHv0jk2+XEBfR99Ynj9nq21F7H8qipCjB9h5aGJTeMjxOqPEoIy3wUw43q76CZcJ0eLyn14YS7b+pMV0mFIMGfumYtuiybpg3E0RHlQtd6BVsGaF dun@stanford.edu
Highlight all of that line (and just that line, down to your email address) and copy it to your clipboard (via Ctrl-C, or what have you).
By the way, that other file – compciv_id_rsa? That's your private key. You never just want to give that out to anyone else. You can cat it just to see what's in it though. As you can see, it's much longer than your public key. Do not copy the private key to your clipboard
Assuming you have copied your public key to your clipboard, you can follow the rest of the Github instructions at Step 3: Add your SSH key to your [Github] account
When you get to the step of typing in ssh -T git@github.com, you will probably receive an error like this:
Permission denied (publickey).
Read on to the next step.
So try this command:
ssh -T git@github.com
I'm assuming you get this error:
Permission denied (publickey).
The Github help page, Error: Agent admitted failure to sign, has the information.
Execute this command:
eval "$(ssh-agent -s)"
You should see this:
Agent pid 45765
Then run this command and enter your passphrase if you set it up:
ssh-add ~/.ssh/compciv_id_rsa
Now you should be good to go:
ssh -T git@github.com
# Response
Hi dannguyen! You've successfully authenticated, but GitHub does not provide shell access.
At this point, you're good to go. If you already know how to set up Github repos, you can try doing so from the command line, but we'll do it in class on Wednesday.
As stated in the deliverables, one of the things you need to send me, via email, is that public key, which we'll do in the next step.
As before, I list out the steps needed to finish this part of the homework. Re-type them manually to get a feel for how things work. If you want a little more insight, read the Software Carpentry guide to the Shell
If you aren't still logged in, log back into corn.stanford.edu.
The Stanford corn server allows you to send mail, via the command-line, from your Stanford email address. However, it only allows you to send emails to other @stanford addresses.
Type in the following at the prompt:
mail your_sunet_id@stanford.edu
You should then be prompted for a Subject, and when you hit Enter, your prompt will move to a blank line. At this point, you can type whatever you want.
At the end of it, hit Control-D on your keyboard to exit out.
You will then be prompted for a Cc: (hit enter to leave it blank). Then, the mail program sends the email (again, Unix will not tell you when things go as expected.
Your screen will probably look like this:
corn01:~$ mail your_sunet@stanford.edu
Subject: heysadf
This is an email blah
blah blaha
BLAH
Cc:
@corn01:~$ # back to the prompt
Most command-line programs have options (also known as flags) that can be set. There's no consistent rhyme or reason to them across different programs (i.e. -s for the mail program is not the same as it is for a program like ls)
@corn01:~$ mail your_sunet@stanford.edu -s "This is my subject"
Hello again
I now know how to use options
Cc:
@corn01:~$ # back to the prompt
Now let's send an email the more Unixy way (i.e faster, with less typing).
The Software Carpentry's guide to Unix pipes is relevant here. But jumping straight into the action:
The echo command simply takes a string of text and prints it to screen. The usage looks like this (note that what's printed is in single quotation marks):
echo 'hello world'
# outputs:
hello world
Using a pipe, denoted by the | character, we can send the text from echo to any other program that can be connected to a pipe (which is most Unix programs). So let's send it to mail:
echo 'Pipes are pretty fun' | mail dun@stanford.edu -s 'PIPES WHOOOooo'
You should see that message in your inbox straightaway. If you try it without the -s flag, you send it again, except without a subject. Oftentimes, the use of a pipe to connect programs will let you run them with the minimal amount of options and fuss.
Once you're done testing mail, now you can complete the assignment which involves sending me (my SUnet is dun):
~/.ssh/compciv_id_rsa.pubIt doesn't matter what order they're sent in, just that they are separated by a newline or a space.
Let's follow the same process as before, but to make things a little conceptually easier, we'll create a temporary text file that we'll pipe into the mail command.
First, change directory to your home directory:
cd ~
Using echo and the > operator, we can send text into a new file:
echo 'Hey dan this is my homework' >> stuff.txt
Now add your Twitter and Github URLs in the same fashion, but use the >> (double brackets) so that the text is appended to stuff.txt (as opposed to overwriting stuff.txt):
echo 'https://github.com/dannguyen' >> stuff.txt
echo 'https://twitter.com/dancow' >> stuff.txt
Go ahead and cat stuff.txt to see that your text is in there. The cat program is short for concatenate, and so every argument (in this case, filename) passed into cat will be added together. Try this:
cat stuff.txt ~/.ssh/compciv_id_rsa.pub
That's pretty much your homework! Try using pipe to mail me in one line (note: the | and > and >> seem to do the same thing, but they don't. The pipe character facilitates the passing of text between programs. Whereas the right-angle-brackets sends text into files)
After you're done, go ahead and delete the stuff.txt (not that there's anything private there):
rm stuff.txt
This concludes the "mail-in" part of the homework.