This guide provides reference information for users to get a quick start in using SC’s shared Linux systems, Ruby, Pearcey and Bragg in particular. Please follow the procedures and links to get started.
Connecting to SC systems via a SSH client
A Secure Shell (SSH) (external link) client allows users to securely log onto Linux systems using encrypted sessions based on the SSH protocol. Note that in most cases, SC's Linux systems can only be accessed by applications that support the SSH protocol.
You can connect to our Linux systems from Windows and any Unix-style machines using your NEXUS credentials. Once an SSH connection is established, and you have logged in successfully, you will be given a shell for running commands.
More information about SSH can be found on the SSH guide.
You will need to know the host names of our systems before you can connect to them. The host name of each system is mentioned in the System Guides.
ssh on Linux and Mac desktops
By default, almost all Linux and Mac desktop machines will have a SSH client available. The command for this client is
ssh. To connect to our Linux systems, typically you will need to do the following:
- Open a terminal
- Run this command:
ident is your Nexus ident and ruby
.hpc.csiro.au is the hostname of our Ruby multiprocessor.
On Mac OSX, the terminal software is located in Applications -> Utilities. On Linux, it is usually located in either Applications / Main Menu -> Accessories or Applications / Main Menu -> System.
Using PuTTY on Windows desktops
First of all, please follow the steps below to manually configure a SSH connection profile in PuTTY. Note that the below example tells you how to connect to Ruby from your Windows desktop.
- Select and run Putty from Start->Programs->Putty menu.
- Enter ruby.hpc.csiro,.au in the "Host Name" field. Leave the number 22 in the "Port field."
- Type ruby in the "Saved Sessions" field and click "Save" button.
- Click on ‘Session’ in ‘Category‘ window again and click the ‘Save’ button. Your Putty is now configured with a profile to connect to Ruby.
To log onto Ruby, simply double click on the saved session, or select a saved session and click "Open".
Accessing a Linux GUI desktop via VNC
There are a number of ways to access applications that have a GUI (Graphical User Interface) frontend on a remote Linux machine. We recommend running GUI via VNC, which offers a remote display system that allows users to run software with graphics capability.
As part of our continuous service improvement, we have developed a desktop application, named , which allows you to manage your remote VNC sessions via a GUI. This application is currently only available to Windows users (we plan to support Linux and Mac in the near future). You can download the latest version from http://ruby.hpc.csiro.au/user/software/SC_Launcher/.
For Linux / Mac users, our VNC helper script allows you to manage your VNC sessions on your systems. After logging onto a system via SSH, you can start a new VNC session by running:
Once the new session is running, you will be given the host address and port number. You will need the details for connecting to your session.
To connect to your VNC session from your own computer, follow these steps:
- download the VNC viewer from http://ruby.hpc.csiro.au/user/software/RealVNC/
- run the viewer
- enter the hostname and port number into the VNC Server field, such as: ruby.hpc.csiro.au:2
- click "Connect"
Check out the VNC helper script section for more details.
The SC Launcher uses the VNC helper script to manage your remote VNC sessions.
Finding out what software is available on SC systems
Information on installed packages in SC systems can be found in http://intra.hpsc.csiro.au/user/pkginfoweb/. For available commercial software, you can check the SC software map (external link). You can check the usage status of our commercial software licenses via our Traffic Lights system.
Setting up your shell environment to run software
Available software in the SC systems requires specific shell environment settings. Our systems provides the Environment Modules system, which allows you to customise your environments in order to use particular software packages. You can learn more about shell environments and the Environment Modules system by reading the Shell Environments guide.
To see what software is available, please run this command:
To run a software program, you will need to load the corresponding module first:
(note: replace <software> with the software, and optionally, the software version.)
To check what modules are already loaded:
module avail will give you a list of available software as well as the available versions for each software (and tell you which version is the default, if set). When you are loading a module, you may also specify the version, e.g.
module load foo/1.0. If you don't specify a version, and there is no default version set, then the latest version will be loaded.
Running batch jobs
When running software or compiled codes on SC shared systems, you need to submit jobs to a batch system . Submitted jobs require resources such as memory and time specified for the run. Use the sbatch command with the memory amount and length of time resources required. e.g.
Check the Running jobs on a Linux system section for detailed use of the batch commands.
Interactive use of the clusters
While batch jobs are needed to effectively utilize the resources available on a HPC cluster, interactive usability is also important to make it as easy as possible for people to manage jobs and visualize data. To enable interactive cluster use and visualization, a choice of options are available to suit a range of needs. See the Interactive access and visualization page for details.
Understanding file systems and data management
Information on the file system structure of the SC systems is located on the "File systems" section in the System Guides. It is important to read the Data Store section to get an understanding of SC’s data store policies and how large amounts of data are managed. In particular, for some directories like
$TMPDIR where data can be purged regularly, users need to save important data elsewhere.
File transfer and remote access to files
You can use the
sftp commands to transfer files between Linux systems. Password-less connection can be done by setting up SSH keys.
To transfer files from and to your Windows desktop machines, we recommend WinSCP, which is a free and open source scp and sftp clients for Windows. You can download WinSCP from the WinSCP homepage (external link).
To access and transfer files from and to your Mac desktop with a gui interface, we recommend Filezilla for mac. You can download it from CNET Download (external link). You can also access your files using Finder via samba. To do this, go to Finder menu, click Go->Connect to server (or command key+K) and in the server field, type smb://bragghome.csiro.au/home/your_ident or smb://ruby.hpc.csiro.au/your_ident and click Connect button.
Developing HPC applications
If you are building code and have significant computational requirements, please refer to the Application Development, as well as the Developing Parallel Applications and the Performance Analysis and Optimisation.
Learning more about Linux
If you want to learn more Linux (e.g. learn how to work with files, learn more about shell scripts, review what commands are available, etc), an introductory guide is available from the University of Edinburgh's Unixhelp System (external link).