The CfN cluster is a linux-based system (CentOS 6.9). You connect to it using ssh, a tool for remote secure connections between computers. The initial connection and your interaction with the cluster are done using a standard linux shell environment (aka 'command line', 'command line interface (CLI)' or 'terminal'). This is a text window where you type commands and get text back as a response.

The default cluster linux shell is bash. Other shells you may see mentioned elsewhere are sh, csh, tcsh, and ksh. These are all variants of each other and can have confusing differences.

Modify your search path in your ~/.bash_profile file, and not ~/.bashrc. If you don't know what this means, see the tutorials below.

If you've never used a linux or unix shell before, you'll want to learn something about it. There are a lot of resources online, or your colleague might have a favorite book. Here are some places to start:

Here's one for the very beginners that takes you through a tutorial: Linux Tutorial

Here's a good-looking introduction to the basics of the shell and writing scripts. It's also available as a more in-depth eBook online: The Linux Command Line. You can get it in print too.

A shorter introduction, more technical: http://www.tldp.org/LDP/Bash-Beginners-Guide/html/chap_01.html

The screen and tmux tools are very useful. They let you create multiple terminal instances from one screen, and most importantly, let you 'detach' from a terminal instance/session and reconnect to it later, even if you've logged out, or lost your network connection or have closed your terminal window. This is very useful for running a long terminal program without having to keep your terminal and computer running.

Screen - Very Simple Usage Instructions

tmux - A Gentle Introduction to tmux

Here's a simple example of a Bash script to loop through patients and run a qsub job on the cluster.

There are a number of ways to use Linux/Bash commands on Windows machines. This can be useful for running rsync for file copying, for example. The options include:

This is a new subsystem in Windows 10 for running Bash shell. It's not a full Linux VM.

https://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/

Tried and true, a little awkward though. Known to work for ssh and rsync, although I'm not sure about X-Windowing/X11/GUI support.

https://www.howtogeek.com/175008/the-non-beginners-guide-to-syncing-data-with-rsync/

Git on windows includes its own shell app. This has been touted as being good and simple for ssh. rsync might work as well.

http://www.hurryupandwait.io/blog/need-an-ssh-client-on-windows-dont-use-putty-or-cygwinuse-git

Run your terminal program to get a command line and login to the cluster.

These have a terminal application built-in. In the Desktop search tool, search for terminal and it should pop up.

Sadly, it's not so easy for Windows. You need to install your own terminal application.

MobaXterm is a newer, good-looking ssh client terminal that includes an integrated X-Server. I haven't tried it, but some users like it.

A popular free terminal app is PuTTy. You can download it here.

PuTTy is kind of awkward to work with. Some quick notes:

• From the main window ('Session' on the left category menu)
  1. In Host Name (or IP address), enter chead.uphs.upenn.edu
  2. In Port, enter 22
  3. For Connection Type, choose SSH
  4. In Saved Sessions, enter chead
  5. Click the Save button to the right

• To setup X11 windowing (X-forwarding) for GUI applications
  1. In the Category menu on the left, click SSH, then X11
  2. Click Enable X11 forwarding so the checkbox has a check in it
  3. For Remote X11 authentication protocol, choose MIT-Magic-Cookie-1
  4. In the Category menu on the left, click Session at the top
  5. You're now back in the main window
  6. Click Save button to the right again
  7. You also need an X-Server app if you don't already. See below.

• When you start a session in Putty (by clicking the “Open” button):
  1. at the login as: prompt, enter just your username. Do NOT enter “ssh my-username@chead”

Note that PuTTy makes a connection before it starts up a terminal window (i.e. where you type you commands), so the ssh command described below happens through the setup and when you choose Load or double-click a session name, and not from the terminal itself.

From your terminal, run the 'ssh' command to connect to the cluster, like so (unless you're using PuTTy on Windows, in which case see above):

ssh -Y your-username@chead   (if this doesn't work, try the full address: chead.uphs.upenn.edu)

The '-Y' option is needed to run GUI applications on the cluster, see below.

If you see an error like one of these:

Error: Can't open display: 
Warning: No display specified.
Warning: No window system found
cannot connect to X server

it means the X-forwarding isn't working. Make sure you used the -Y option to ssh, or otherwise setup your terminal software properly (e.g. PuTTy has particular settings).

If you get an error like:

ssh: Could not resolve hostname chead: nodename nor servname provided, or not known

it most like means you are not connected to the UPHS Network, and need to use VPN.

If you want to run GUI applications (aka graphical, or windowed applications) on the cluster (for example the Matlab graphical desktop environment), you need to run an X11 windowing session. That is, you need to have a X Server running on your computer. Note, the server/client terminology is reversed for X11 from traditional usage - your local computer on your desktop is the X11 Server, and the remote computer that's running the GUI application is the X11 Client.

If running a GUI application over X11 is slow, you'll probably be much better off using the x2go remote desktop tool. See x2go Remote Desktop. Connecting over VPN from off-campus is often much better using x2go, but sometimes on-campus connections can be slow and x2go can help.

When you ssh to chead, be sure to use the -Y option as mentioned above.

Once you follow instructions below to get setup, a quick test of your X11 functionality is to login to chead and run the command xclock . You should get a graphical clock window open up on your computer.

There nothing to do, Linux should be setup out-of-the-box to accept X11 connections.

You need to install XQuartz, the open-source X Server for Mac (developed orginally by Apple).

The issue mentioned below for 2.7.9 persists, however the fix is simpler. On your local machine (i.e. your laptop or desktop, not the cluster), run this command in Terminal:

  defaults write org.macosforge.xquartz.X11 enable_iglx -bool true

June 2016

Xquartz 2.7.9 may break some programs like ITK-SNAP when running over ssh. Here's a fix (thanks to Phil Cook!):

To fix, I had to edit the startup config on my own machine (not on chead):

sudo vi /usr/X11R6/bin/startx   <--- do this on your machine, not on chead

changing Line 59 to

defaultserverargs="+iglx"

Then from the command line,

defaults write org.macosforge.xquartz.X11 enable_iglx -bool true

After this and a restart of XQuartz, it works as normal.

If this still doesn't work, you might have luck downgrading to version 2.7.8

You'll need an X Server app like MobaXterm , or the Xming.

Note that mobaXterm is multi-tool application, including a terminal and sftp client, but its X11 Server is available to other applications on your system (like the x2go client), in our experience.

You'll need a terminal application, that is, a command-line text application that can connect to remote servers. We recommend the mobaXterm app (see above) that includes terminal and sftp (and other) tools along with the X11 Server functionality.

If you're using PuTTy as your terminal app, make sure to follow the directions under Windows terminal to setup X-forwarding.

For SecureCRT, see this page: http://www.seas.upenn.edu/cets/answers/x11-forwarding.html

To change your password, run the passwd command on the command line.

• You must use a strong password, that does not include dictionary words, repeating characters, and that does include some mix of capital letters, symbols and/or numbers.

This is a general warning about using the command rm and in particular rm -fr.

When you delete a file in Linux using rm, you can NOT recover it. We maintain tape backups of all cluster data on a 2-3 month cycle. These are intended for worst-case scenario disaster recovery, and thus generally backups are not available for mistaken deletions by individual users, although you can check with the admins.

This command removes whatever files and directories you list after it without asking for confirmation, and recursively, meaning it goes into every sub-directory and deletes that too. One user recently ran rm -fr /* instead of rm -fr ./*. The missing '.' meant that instead of trying to delete everything in her current working directly, it tried to delete everything on the whole front end. Most files were protected so she couldn't delete them, but when the command hits /data, it will delete everything in there that is owned by her, or for which she has permissions to delete because of group membership. This could be very bad, obviously. Luckily in the recent case the user stopped the command before it got to the /data directory.

This warning also applies to the rm and rm -r commands, which by default will remove files without asking confirmation.

You may also have an issue, for example, if you are in a different directory than you think, and run something like rm *.nii. It will remove all files in the current directory without prompting for confirmation, and you'll have lost them for good.

Be careful. It's often good to replace your rm command with ls first to make sure you're getting what you intend. After you run ls <my list of files or glob command>, use the up-arrow key in your terminal to recall the previous command and edit it to replace ls with your rm command, so that you're sure not to introduce a typo in the list of files or glob command.

You'll probably need to edit text files. If you're new to linux/bash, here are some common text editors. To use them, type their names on the command line:

• gedit - a popular simple graphical text editor
• emacs - the classic unix/linux text editor. On the cluster it comes up as a GUI editor so is easier to use than the classic text-only version. It's all powerful but kinda dense to learn.
• vi - simpler than emacs, but even more dense

Your data and home directories have quotas setup that limit how much disk space you can use in them. To check your quota run:

df -h /data/<mydir>

where <mydir> is something like jet/mgstauff or jag/myusername.

Generally you'll need to move files between your local/desktop computer and data directories on the cluster.

See this page for details.