From 10b8a56c2ea20135eaf92d28662111e5c6ebeedb Mon Sep 17 00:00:00 2001 From: Github CI <> Date: Mon, 18 Sep 2023 14:39:02 +0000 Subject: [PATCH] Deploy to GitHub Pages: c52a5ec8ddd03395e3b035826e8c08300ff08eef --- .../python-packages/index.html | 2 +- Walkthroughs/Logging_In/index.html | 6 ++++++ search/search_index.json | 2 +- sitemap.xml.gz | Bin 810 -> 810 bytes 4 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Installed_Software_Lists/python-packages/index.html b/Installed_Software_Lists/python-packages/index.html index 32ea7c323..ae836ce28 100644 --- a/Installed_Software_Lists/python-packages/index.html +++ b/Installed_Software_Lists/python-packages/index.html @@ -1352,7 +1352,7 @@

Python PackagesWindows Command PromptIt shows you your username, which Myriad login node you are on, and where you are (~ is a short way to reference your home directory). You can now look at what software is available and write jobscripts.

+

Helpful resources:

+ +

After you have successfully logged in for the first time, visit the remote access for further resources on accessing UCL services from outside the UCL firewall and creating an SSH key pair to help with logging in.

Desktop@UCL Anywhereยง

You can log in to this from a web browser or download a Citrix client. Once logged in, Desktop@UCL is inside the UCL network so we can log straight into a cluster with no need diff --git a/search/search_index.json b/search/search_index.json index 7d27a1910..fff77bcea 100644 --- a/search/search_index.json +++ b/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Research Computing Services","text":"

This documentation is maintained by the Research Computing team for the purpose of sharing information about our services, including user guides, service updates and account request and renewal support.

"},{"location":"#documentation","title":"Documentation","text":"

General User Information:

Clusters:

"},{"location":"#email-support","title":"Email Support","text":"

For support for any of our services, contact us at: rc-support@ucl.ac.uk

We will endeavour to answer queries on any aspect of computing related to your research, whatever your skill level or requirements.

"},{"location":"#how-to","title":"How to...","text":""},{"location":"#training","title":"Training","text":"

Due to the on-going COVID-19 pandemic, our ordinary training courses have been suspended indefinitely.

We have an online Moodle course \"Introduction to the Unix Shell\" and infrequently provide a training course aimed at getting users up and running on one of our main clusters. \"Introduction to the Unix Shell (Moodle) (UCL users), GitHub-Pages (non-UCL users), additional resources can be found on UCL mediacentral.

We have an online Moodle course \"Introduction to High Performance Computing at UCL\" aimed at getting users comfortable with using HPC at UCL. \"Introduction to High Performance Computing at UCL\" (Moodle) (UCL users), UCL mediacentral.

Please see our Research IT Training page for details.

"},{"location":"Account_Services/","title":"Account Services","text":""},{"location":"Account_Services/#cluster-access-application","title":"Cluster access application","text":"

There is a single online form for applying to the majority of our clusters. Use of these services is subject to a common set of terms and conditions. All granted applications give you a Myriad account while access to other systems depends on what types of work you tell us you will be doing.

"},{"location":"Account_Services/#account-sponsors","title":"Account sponsors","text":"

If you are a student or postdoctoral researcher, your application must be approved by a permanent member of staff (normally your supervisor or PI). This will be automatically determined when you begin your application, and when you submit it, an email will be sent to this person asking them to approve the application before the account can be created.

Please note that the form requires your sponsor's UCL username and not their UPI.

Permanent members of staff do not need a sponsor and their accounts will be automatically approved.

"},{"location":"Account_Services/#apply-for-an-account","title":"Apply for an account","text":"

The online application form can be found here:

You will need a UCL user name and password. These are the same credentials used to access core services like Portico and HR systems.

"},{"location":"Account_Services/#application-process","title":"Application process","text":"

The application process has these steps:

  1. Enter your UCL username and password to access the application form.

  2. Complete the application form, reading the instructions carefully. Tip: hover your mouse over text boxes for more information.

  3. When you successfully submit the form, your sponsor will be sent an email message asking them to approve your application. If you do not require a sponsor, your application will be automatically approved.

  4. Your sponsor should click on the link in the email, log in and approve the account.

  5. You will then receive an email when your account is approved, if you have a sponsor. (This does not mean your account has been created yet).

  6. You should receive an email once we have created your account. Please note that there may be a delay of up to one working day between an application being approved and your account being created.

If your sponsor does not receive the email to approve your account, send them the link to your application directly (it will look like dashboard.rc.ucl.ac.uk/computing/requests/xxxx) and there will be a button at the top to approve it.

Please note: most delays in the application process are caused by sponsors missing or not receiving the approval email, so check with your sponsor first if your application is stuck waiting for approval. Research Computing can only create your account after it is approved.

"},{"location":"Account_Services/#accounts-for-visitors","title":"Accounts for visitors","text":"

UCL visitors are welcome to apply for accounts on Research Computing Services. Please note that:

  1. Applicants must have a central UCL account. UCL Visitor accounts can be arranged by your Departmental Administrator, so you should speak to them first.
  2. Account applications should specify the UCL grant under which the work is being carried out, if possible, as well as an associated UCL Group or Researcher.
  3. Account applications may not be submitted on behalf of another, except to cover accessibility requirements, as the account application process includes agreeing to relevant legal terms and conditions.
"},{"location":"Account_Services/#accounts-for-honorary-staff","title":"Accounts for Honorary staff","text":"

UCL Staff Members may nominate Honorary members (named individuals) to be provided with access to Research Computing services where this is beneficial to UCL's research interests.

Nomination should be made via the CRAG, explaining the benefit arising to UCL.

Proposals will be reviewed by the CRAG at their monthly meeting and on a case by case basis.

All accounts thus provided are subject to all other 'standard' T&C's relating to their use of Research Computing Services.

"},{"location":"Account_Services/#charges-for-use-of-research-computing-services","title":"Charges for use of Research Computing services","text":"

Research Computing services are free at point of use by default. There are no direct charges for your usage under standard resource allocation policy as defined by the CRAG.

Several methods are available to researchers who wish to gain access to additional resources, or obtain 'priority' use, including chargeable options. Details are available at Additional Resource_Requests and Purchasing in Myriad.

"},{"location":"Additional_Resource_Requests/","title":"Additional Resource Requests","text":"

We recognise that researchers may sometimes require a higher throughput of work than it is possible to achieve with free \u2018fair share\u2019 usage of Myriad and Kathleen. There a couple of ways of obtaining additional Myriad resources beyond this fair share:

"},{"location":"Additional_Resource_Requests/#make-a-special-request-to-the-crag-for-free-access-to-additional-resources","title":"Make a special request to the CRAG for free access to additional resources","text":"

Users who wish to request additional resources or reserve resources beyond those provided can complete the additional resource request form in collaboration with your supervisor or the project's principal investigator. This includes requests for increased storage quotas.

The completed form should be sent to the Research Computing Platforms team at rc-support@ucl.ac.uk, for technical review. If successful, your case will be presented to the CRAG for consideration at the next meeting of the Group. The CRAG meets monthly, usually on the second Tuesday of the month, and users will be informed of the Group\u2019s decision as soon as possible after their next meeting.

Note that an application to the CRAG for additional resources is only likely to be approved if the impact on other users is not deemed to be significant, or of long duration.

"},{"location":"Additional_Resource_Requests/#examples-of-requests","title":"Examples of requests","text":""},{"location":"Additional_Resource_Requests/#request-hosting-of-shared-datasets","title":"Request hosting of shared datasets","text":"

We have provision for hosting shared datasets for users on Myriad. These can be datasets that are freely accessible by all users, or ones limited to groups.

Hosted datasets:

They are likely to be managed by a role account - access to the role account will be by ssh key.

To apply for a hosted dataset, please send this form to rc-support@ucl.ac.uk.

"},{"location":"Additional_Resource_Requests/#purchase-dedicated-compute-nodes-or-priority-cycles-on-myriad","title":"Purchase dedicated compute nodes or priority cycles on Myriad","text":"

Researchers may purchase additional resources to be used as part of the Myriad High Performance Computing cluster if the free service does not meet their needs. See Paid-for resources: Purchasing in Myriad for details.

"},{"location":"Additional_Resource_Requests/#further-information","title":"Further information","text":"

For further advice or information on future hardware options, please contact rc-support@ucl.ac.uk.

"},{"location":"Contact_Us/","title":"Contact and Support","text":"

Users should direct any queries relating to their use of Research Computing services to the Research Computing Support Team at rc-support@ucl.ac.uk (see below). The team will respond to your question as quickly as possible, giving priority to requests that are deemed urgent on the basis of the information provided.

Availability: 9:30am - 4:30pm, Monday - Friday, except on Bank Holidays and College Closures.

We aim to provide to you with a useful response within 24 hours.

Please do not email individuals unless you are explicitly asked to do so; always use the rc-support email address provided as this is the best way for your request to be processed.

"},{"location":"Contact_Us/#drop-in-sessions","title":"Drop-In Sessions","text":"

Research IT Services holds drop-in sessions roughly every two weeks which at least one member of the Research Computing team usually attends. More details and dates for these sessions are available on the the RITS pages.

If you have a particularly complex problem, it may be useful to email the support address, rc-support@ucl.ac.uk, beforehand so that the person attending can prepare.

"},{"location":"Contact_Us/#location","title":"Location","text":"

The Research Computing Team are located at:

38-50 Bidborough Street Floor 3 London WC1H 9BT

We are keen to collaborate and welcome visitors to our offices to talk about all things research computing. However, we do not operate a walk-up service desk: if you are frustrated by slow response to a support ticket, we are sorry but please do send reminders as there is probably a good reason why your request is not being processed.

"},{"location":"Example_Jobscripts/","title":"Example Jobscripts","text":"

On this page we describe some basic example scripts to submit jobs to our clusters.

After creating your script, submit it to the scheduler with:

qsub my_script.sh

"},{"location":"Example_Jobscripts/#service-differences","title":"Service Differences","text":"

These scripts are applicable to all our clusters, but node sizes (core count, memory, and temporary storage sizes) differ between machines, so please check those details on the cluster-specific pages. Some clusters are diskless and have no temporary space that can be requested.

"},{"location":"Example_Jobscripts/#working-directories-and-output","title":"Working Directories and Output","text":"

The parallel filesystems we use to provide the home and scratch filesystems perform best when reading or writing single large files, and worst when operating on many different small files. To avoid causing problems, many of the scripts below are written to create all their files in the temporary $TMPDIR storage, and compress and copy them to the scratch area at the end of the job.

This can be a problem if your job is not finishing and you need to see the output, or if your job is crashing or failing to produce what you expected. Feel free to modify the scripts to read from or write to Scratch directly, however, your performance will generally not be as good as writing to $TMPDIR, and you may impact the general performance of the machine if you do this with many jobs simultaneously. This is particularly the case with single-core jobs, because that core is guaranteed to be writing out data.

Please be aware that some clusters are diskless (eg Kathleen) and have no $TMPDIR available for use - in those you must remove the request for tmpfs in your script. Check the cluster-specific pages.

Note that there is also the option of using the Local2Scratch process (see below), which takes place after the job has finished, in the clean-up step. This gives you the option of always getting the contents of $TMPDIR back, at the cost of possibly getting incomplete files and not having any control over where the files go.

"},{"location":"Example_Jobscripts/#note-about-projects","title":"Note about Projects","text":"

Projects are a system used in the scheduler and the accounting system to track budgets and access controls.

Most users of UCL's internal clusters will not need to specify a project and will default to the AllUsers project. Users of the Thomas and Michael services should refer to the specific pages for those machines, and the information they were given when they registered.

To specify a project ID in a job script, use the -P object as below:

#$\u00a0-P\u00a0<your_project_id>\n
"},{"location":"Example_Jobscripts/#resources","title":"Resources","text":"

The lines starting with #$ -l are where you are requesting resources like wallclock time (how long your job is allowed to run), memory, and possibly tmpfs (local hard disk space on the node, if it is not diskless). More details on how to request resources can be found here.

If you have no notion of how much you should request for any of these, have a look at How do I estimate what resources to request in my jobscript?

Useful resources:

"},{"location":"Example_Jobscripts/#serial-job-script-example","title":"Serial Job Script Example","text":"

The most basic type of job a user can submit is a serial job. These jobs run on a single processor (core) with a single thread.

Shown below is a simple job script that runs /bin/date (which prints the current date) on the compute node, and puts the output into a file.

#!/bin/bash -l\n\n# Batch script to run a serial job under SGE.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set the name of the job.\n#$ -N Serial_Job\n\n# Set the working directory to somewhere in your scratch space.  \n#  This is a necessary step as compute nodes cannot write to $HOME.\n# Replace \"<your_UCL_id>\" with your UCL user ID.\n#$ -wd /home/<your_UCL_id>/Scratch/workspace\n\n# Your work should be done in $TMPDIR \ncd $TMPDIR\n\n# Run the application and put the output into a file called date.txt\n/bin/date > date.txt\n\n# Preferably, tar-up (archive) all output files onto the shared scratch area\ntar -zcvf $HOME/Scratch/workspace/files_from_job_$JOB_ID.tar.gz $TMPDIR\n\n# Make sure you have given enough time for the copy to complete!\n
"},{"location":"Example_Jobscripts/#multi-threaded-job-example","title":"Multi-threaded Job Example","text":"

For programs that can use multiple threads, you can request multiple processor cores using the -pe smp <number> option. One common method for using multiple threads in a program is OpenMP, and the $OMP_NUM_THREADS environment variable is set automatically in a job of this type to tell OpenMP how many threads it should use. Most methods for running multi-threaded applications should correctly detect how many cores have been allocated, though (via a mechanism called cgroups).

Note that this job script works directly in scratch instead of in the temporary $TMPDIR storage.

#!/bin/bash -l\n\n# Batch script to run an OpenMP threaded job under SGE.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM for each core/thread \n# (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set the name of the job.\n#$ -N Multi-threaded_Job\n\n# Request 16 cores.\n#$ -pe smp 16\n\n# Set the working directory to somewhere in your scratch space.\n# Replace \"<your_UCL_id>\" with your UCL user ID\n#$ -wd /home/<your_UCL_id>/Scratch/output\n\n# 8. Run the application.\n$HOME/my_program/example\n
"},{"location":"Example_Jobscripts/#mpi-job-script-example","title":"MPI Job Script Example","text":"

The default MPI implementation on our clusters is the Intel MPI stack. MPI programs don\u2019t use a shared memory model so they can be run across multiple nodes. This script differs considerably from the serial and OpenMP jobs in that MPI programs need to be invoked by a program called gerun. This is our wrapper for mpirun and takes care of passing the number of processors and a file called a machine file.

Important: If you wish to pass a file or stream of data to the standard input (stdin) of an MPI program, there are specific command-line options you need to use to control which MPI tasks are able to receive it. (-s for Intel MPI, --stdin for OpenMPI.) Please consult the help output of the mpirun command for further information. The gerun launcher does not automatically handle this.

If you use OpenMPI, you need to make sure the Intel MPI modules are removed and the OpenMPI modules are loaded, either in your jobscript or in your shell start-up files (e.g. ~/.bashrc).

#!/bin/bash -l\n\n# Batch script to run an MPI parallel job under SGE with Intel MPI.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM per process (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space per node \n# (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set the name of the job.\n#$ -N MadScience_1_16\n\n# Select the MPI parallel environment and 16 processes.\n#$ -pe mpi 16\n\n# Set the working directory to somewhere in your scratch space.\n# Replace \"<your_UCL_id>\" with your UCL user ID :\n#$ -wd /home/<your_UCL_id>/Scratch/output\n\n# Run our MPI job.  GERun is a wrapper that launches MPI jobs on our clusters.\ngerun $HOME/src/science/simulate\n
"},{"location":"Example_Jobscripts/#array-job-script-example","title":"Array Job Script Example","text":"

If you want to submit a large number of similar serial jobs then it may be easier to submit them as an array job. Array jobs are similar to serial jobs except we use the -t option to get Sun Grid Engine to run 10,000 copies of this job numbered 1 to 10,000. Each job in this array will have the same job ID but a different task ID. The task ID is stored in the $SGE_TASK_ID environment variable in each task. All the usual SGE output files have the task ID appended. MPI jobs and parallel shared memory jobs can also be submitted as arrays.

#!/bin/bash -l\n\n# Batch script to run a serial array job under SGE.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set up the job array.  In this instance we have requested 10000 tasks\n# numbered 1 to 10000.\n#$ -t 1-10000\n\n# Set the name of the job.\n#$ -N MyArrayJob\n\n# Set the working directory to somewhere in your scratch space. \n# Replace \"<your_UCL_id>\" with your UCL user ID :)\n#$ -wd /home/<your_UCL_id>/Scratch/output\n\n# Run the application.\n\necho \"$JOB_NAME $SGE_TASK_ID\"\n
"},{"location":"Example_Jobscripts/#array-job-script-example-using-parameter-file","title":"Array Job Script Example Using Parameter File","text":"

Often a user will want to submit a large number of similar jobs but their input parameters don't match easily on to an index from 1 to n. In these cases it's possible to use a parameter file. To use this script a user needs to construct a file with a line for each element in the job array, with parameters separated by spaces.

For example: 

0001 1.5 3 aardvark\n0002 1.1 13 guppy\n0003 1.23 5 elephant\n0004 1.112 23 panda\n0005 ...\n

Assuming that this file is stored in ~/Scratch/input/params.txt (you can call this file anything you want) then the user can use awk/sed to get the appropriate variables out of the file. The script below does this and stores them in $index, $variable1, $variable2 and $variable3. So for example in task 4, $index = 0004, $variable1 = 1.112, $variable2 = 23 and $variable3 = panda.

Since the parameter file can be generated automatically from a user's datasets, this approach allows the simple automation, submission and management of thousands or tens of thousands of tasks.

#!/bin/bash -l\n\n# Batch script to run an array job.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set up the job array.  In this instance we have requested 1000 tasks\n# numbered 1 to 1000.\n#$ -t 1-1000\n\n# Set the name of the job.\n#$ -N array-params\n\n# Set the working directory to somewhere in your scratch space.\n# Replace \"<your_UCL_id>\" with your UCL user ID :)\n#$ -wd /home/<your_UCL_id>/Scratch/output\n\n# Parse parameter file to get variables.\nnumber=$SGE_TASK_ID\nparamfile=/home/<your_UCL_id>/Scratch/input/params.txt\n\nindex=\"`sed -n ${number}p $paramfile | awk '{print $1}'`\"\nvariable1=\"`sed -n ${number}p $paramfile | awk '{print $2}'`\"\nvariable2=\"`sed -n ${number}p $paramfile | awk '{print $3}'`\"\nvariable3=\"`sed -n ${number}p $paramfile | awk '{print $4}'`\"\n\n# Run the program (replace echo with your binary and options).\n\necho \"$index\" \"$variable1\" \"$variable2\" \"$variable3\"\n
"},{"location":"Example_Jobscripts/#example-array-job-using-local2scratch","title":"Example Array Job Using Local2Scratch","text":"

Users can automate the transfer of data from $TMPDIR to their scratch space by adding the text #Local2Scratch to their script on a line alone as a special comment. During the clean-up phase of the job, a tool checks whether the script contains that text, and if so, files are transferred from $TMPDIR to a directory in scratch with the structure <job id>/<job id>.<task id>.<queue>/.

The example below does this for a job array, but this works for any job type.

#!/bin/bash -l\n\n# Batch script to run an array job under SGE and \n#  transfer the output to Scratch from local.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set up the job array.  In this instance we have requested 10000 tasks\n# numbered 1 to 10000.\n#$ -t 1-10000\n\n# Set the name of the job.\n#$ -N local2scratcharray\n\n# Set the working directory to somewhere in your scratch space.\n# Replace \"<your_UCL_id>\" with your UCL user ID :)\n#$ -wd /home/<your_UCL_id>/Scratch/output\n\n# Automate transfer of output to Scratch from $TMPDIR.\n#Local2Scratch\n\n# Run the application in TMPDIR.\ncd $TMPDIR\nhostname > hostname.txt\n
"},{"location":"Example_Jobscripts/#array-job-script-with-a-stride","title":"Array Job Script with a Stride","text":"

If each task for your array job is very small, you will get better use of the cluster if you can combine a number of these so each has a couple of hours' worth of work to do. There is a startup cost associated with the amount of time it takes to set up a new job. If your job's runtime is very small, this cost is proportionately high, and you incur it with every array task.

Using a stride will allow you to leave your input files numbered as before, and each array task will run N inputs.

For example, a stride of 10 will give you these task IDs: 1, 11, 21...

Your script can then have a loop that runs task IDs from $SGE_TASK_ID to $SGE_TASK_ID + 9, so each task is doing ten times as many runs as it was before.

#!/bin/bash -l\n\n# Batch script to run an array job with strided task IDs under SGE.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set up the job array.  In this instance we have requested task IDs\n# numbered 1 to 10000 with a stride of 10.\n#$ -t 1-10000:10\n\n# Set the name of the job.\n#$ -N arraystride\n\n# Set the working directory to somewhere in your scratch space.\n# Replace \"<your_UCL_id>\" with your UCL user ID :)\n#$ -wd /home/<your_UCL_id>/Scratch/output\n\n# Automate transfer of output to Scratch from $TMPDIR.\n#Local2Scratch\n\n# Do your work in $TMPDIR \ncd $TMPDIR\n\n# 10. Loop through the IDs covered by this stride and run the application if \n# the input file exists. (This is because the last stride may not have that\n# many inputs available). Or you can leave out the check and get an error.\nfor (( i=$SGE_TASK_ID; i<$SGE_TASK_ID+10; i++ ))\ndo\nif [ -f \"input.$i\" ]\nthen\necho \"$JOB_NAME\" \"$SGE_TASK_ID\" \"input.$i\"\nfi\ndone\n
"},{"location":"Example_Jobscripts/#hybrid-mpi-plus-openmp-jobscript-example","title":"Hybrid MPI plus OpenMP jobscript example","text":"

This is a type of job where you have a small number of MPI processes, and each one of those launches a number of threads. One common form of this is to have only one MPI process per node which handles communication between nodes, and the work on each node is done in a shared memory style by threads.

When requesting resources for this type of job, what you are asking the scheduler for is the physical number of cores and amount of memory per core that you need. Whether you end up running MPI processes or threads on that core is up to your code. (The -pe mpi xx request is telling it you want an MPI parallel environment and xx number of cores, not that you want xx MPI processes - this can be confusing).

"},{"location":"Example_Jobscripts/#setting-number-of-threads","title":"Setting number of threads","text":"

You can either set $OMP_NUM_THREADS for the number of OpenMP threads yourself, or allow it to be worked out automatically by setting it to OMP_NUM_THREADS=$(ppn). That is a helper script on our clusters which will set $OMP_NUM_THREADS to $NSLOTS/$NHOSTS, so you will use threads within a node and MPI between nodes and don't need to know in advance what size of node you are running on. Gerun will then run $NSLOTS/$OMP_NUM_THREADS processes, round-robin allocated (if supported by the MPI).

#!/bin/bash -l\n\n# Batch script to run a hybrid parallel job under SGE.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM per core (must be an integer)\n#$ -l mem=1G\n\n# Request 15 gigabytes of TMPDIR space per node (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set the name of the job.\n#$ -N MadIntelHybrid\n\n# Select the MPI parallel environment and 80 cores.\n#$ -pe mpi 80\n\n# Set the working directory to somewhere in your scratch space. \n# This directory must exist.\n#$ -wd /home/<your_UCL_id/scratch/output/\n\n# Automatically set threads using ppn. On a cluster with 40 cores\n# per node this will be 80/2 = 40 threads.\nexport OMP_NUM_THREADS=$(ppn)\n\n# Run our MPI job with the default modules. Gerun is a wrapper script for mpirun. \ngerun $HOME/src/madscience/madhybrid\n

If you want to specify a specific number of OMP threads yourself, you would alter the relevant lines above to this:

# Request 80 cores and run 4 MPI processes per 40-core node, each spawning 10 threads\n#$ -pe mpi 80\nexport OMP_NUM_THREADS=10\ngerun your_binary\n
"},{"location":"Example_Jobscripts/#gpu-job-script-example","title":"GPU Job Script Example","text":"

To use NVIDIA GPUs with the CUDA libraries, you need to load the CUDA runtime libraries module or else set up the environment yourself. The script below shows what you'll need to unload and load the appropriate modules.

You also need to use the -l gpu=<number> option to request the GPUs from the scheduler.

#!/bin/bash -l\n\n# Batch script to run a GPU job under SGE.\n\n# Request a number of GPU cards, in this case 2 (the maximum)\n#$ -l gpu=2\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space (default is 10 GB)\n#$ -l tmpfs=15G\n\n# Set the name of the job.\n#$ -N GPUJob\n\n# Set the working directory to somewhere in your scratch space.\n# Replace \"<your_UCL_id>\" with your UCL user ID :)\n#$ -wd /home/<your_UCL_id>/Scratch/output\n\n# Change into temporary directory to run work\ncd $TMPDIR\n\n# load the cuda module (in case you are running a CUDA program)\nmodule unload compilers mpi\nmodule load compilers/gnu/4.9.2\nmodule load cuda/7.5.18/gnu-4.9.2\n\n# Run the application - the line below is just a random example.\nmygpucode\n\n# 10. Preferably, tar-up (archive) all output files onto the shared scratch area\ntar zcvf $HOME/Scratch/files_from_job_$JOB_ID.tar.gz $TMPDIR\n\n# Make sure you have given enough time for the copy to complete!\n
"},{"location":"Example_Jobscripts/#job-using-mpi-and-gpus","title":"Job using MPI and GPUs","text":"

It is possible to run MPI programs that use GPUs but our clusters currently only support this within a single node. The script below shows how to run a program using 2 gpus and 12 cpus.

#!/bin/bash -l\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 12 cores, 2 GPUs, 1 gigabyte of RAM per CPU, 15 gigabyte of TMPDIR space\n#$ -l mem=1G\n#$ -l gpu=2\n#$ -pe mpi 12\n#$ -l tmpfs=15G\n\n# Set the name of the job.\n#$ -N GPUMPIrun\n\n# Set the working directory to somewhere in your scratch space.\n#$ -wd /home/<your user id>/Scratch/output/\n\n# Run our MPI job. You can choose OpenMPI or IntelMPI for GCC.\nmodule unload compilers mpi\nmodule load compilers/gnu/4.9.2\nmodule load mpi/openmpi/1.10.1/gnu-4.9.2\nmodule load cuda/7.5.18/gnu-4.9.2\n\ngerun myGPUapp\n
"},{"location":"Experienced_Users/","title":"Quick Start Guide for Experienced HPC Users","text":""},{"location":"Experienced_Users/#what-services-are-available","title":"What Services are available?","text":""},{"location":"Experienced_Users/#ucl","title":"UCL","text":""},{"location":"Experienced_Users/#external","title":"External","text":""},{"location":"Experienced_Users/#how-do-i-get-access","title":"How do I get access?","text":"

UCL services: Fill in the sign-up form

Tier 2 services: Contact your point of contact.

"},{"location":"Experienced_Users/#how-do-i-connect","title":"How do I connect?","text":"

All connections are via SSH, and you use your UCL credentials to log in (external users should use the mmmXXXX account with the SSH key they have provided to their point of contact). UCL services can only be connected to by users inside the UCL network which may mean using the institutional VPN or \"bouncing\" off another UCL machine when accessing them from outside the UCL network. The Tier 2 services (Thomas and Michael) are externally accessible.

"},{"location":"Experienced_Users/#login-hosts","title":"Login hosts","text":"Service General alias Direct login node addresses Grace grace.rc.ucl.ac.uk login01.ext.grace.ucl.ac.uk login02.ext.grace.ucl.ac.uk Kathleen kathleen.rc.ucl.ac.uk login01.kathleen.rc.ucl.ac.uk login02.kathleen.rc.ucl.ac.uk Myriad myriad.rc.ucl.ac.uk login12.myriad.rc.ucl.ac.uk login13.myriad.rc.ucl.ac.uk Thomas thomas.rc.ucl.ac.uk login03.thomas.rc.ucl.ac.uk login04.thomas.rc.ucl.ac.uk Michael michael.rc.ucl.ac.uk login10.michael.rc.ucl.ac.uk login11.michael.rc.ucl.ac.uk

Generally you should connect to the general alias as this is load-balanced across the available login nodes, however if you use screen or tmux you will want to use the direct hostname so that you can reconnect to your session.

"},{"location":"Experienced_Users/#software-stack","title":"Software stack","text":"

All UCL services use the same software stack based upon RHEL 7.x with a standardised set of packages, exposed to the user through environment modules (the module command). By default this has a set of useful tools loaded, as well as the Intel compilers and MPI but users are free to change their own environment.

"},{"location":"Experienced_Users/#batch-system","title":"Batch System","text":"

UCL services use Grid Engine to manage jobs. This install is somewhat customised and so scripts for non-UCL services may not work.

We recommend that when launching MPI jobs you use our gerun parallel launcher instead of mpirun as it inherits settings from the job and launches the appropriate number of processes with the MPI implementation you have chosen. It abstracts away a lot of the complexity between different version of MPI.

# using gerun\ngerun myMPIprogram\n\n# using mpirun when a machinefile is needed (eg Intel MPI)\nmpirun -np $NSLOTS -machinefile $PE_HOSTFILE myMPIprogram\n

$NSLOTS is an environment variable containing the value you gave to -pe mpi so you do not need to re-specify it.

"},{"location":"Experienced_Users/#troubleshooting-gerun","title":"Troubleshooting gerun","text":"

If you need to see what gerun is doing because something is not working as expected, look at the error file for your job, default name $JOBNAME.e$JOB_ID. It will contain debug information from gerun about where it ran and the exact mpirun command it used.

You may need to use mpirun directly with different options if your program has sufficiently complex process placement requirements, or is using something like GlobalArrays and requires a different process layout than it is being given.

"},{"location":"Experienced_Users/#script-sections","title":"Script sections","text":""},{"location":"Experienced_Users/#shebang","title":"Shebang","text":"

It's important that you add the -l option to bash in the shebang so that login scripts are parsed and the environment modules environment is set up. #! must be the first two characters in the file, no previous white space.

#!/bin/bash -l\n
"},{"location":"Experienced_Users/#resources-you-can-request","title":"Resources you can request","text":""},{"location":"Experienced_Users/#number-of-cores","title":"Number of cores","text":"

For MPI:

#$ -pe mpi <number of cores>\n

For threads:

#$ -pe smp <number of cores>\n

For single core jobs you don't need to request a number of cores. For hybrid codes use the MPI example and set OMP_NUM_THREADS to control the number of threads per node. gerun will launch the right number of processes in the right place if you use it.

"},{"location":"Experienced_Users/#amount-of-ram-per-core","title":"Amount of RAM per core","text":"
#$ -l mem=<integer amount of RAM in G or M>\n

e.g. #$ -l mem=4G requests 4 gigabytes of RAM per core.

"},{"location":"Experienced_Users/#run-time","title":"Run time","text":"
#$ -l h_rt=<hours:minutes:seconds>\n

e.g. #$ -l h_rt=48:00:00 requests 48 hours.

"},{"location":"Experienced_Users/#working-directory","title":"Working directory","text":"

Either a specific working directory:

#$ -wd /path/to/working/directory\n

or the directory the script was submitted from:

#$ -cwd\n
"},{"location":"Experienced_Users/#gpus-myriad-only","title":"GPUs (Myriad only)","text":"
#$ -l gpu=<number of GPUs>\n
"},{"location":"Experienced_Users/#enable-hyperthreads-kathleen-only","title":"Enable Hyperthreads (Kathleen only)","text":"
#$ -l threads=1\n

The -l threads= request is not a true/false setting, instead you are telling the scheduler you want one slot to block one virtual cpu instead of the normal situation where it blocks two. If you have a script with a threads request and want to override it on the command line or set it back to normal, the usual case is -l threads=2. (Setting threads to 0 does not disable hyperthreading!)

With Hyperthreads enabled you need to request twice as many cores and then control threads vs MPI ranks with OMP_NUM_THREADS. E.g. 

#$ -pe mpi 160\n#$ -l threads=1\nexport OMP_NUM_THREADS=2\n

Would use 80 cores, with two threads (on Hyperthreads) per core. If you use gerun to launch your MPI process, it will take care of the division for you, but if you're using mpirun or mpiexec directly, you'll have to take care to use the correct number of MPI ranks per node yourself.

Note that memory requests are now per virtual core with hyperthreading enabled. If you asked for #$ -l mem=4Gon a node with 80 virtual cores and 192G RAM then you are requiring 320G RAM in total which will not fit on that node and so you would be given a sparse process layout across more nodes to meet this requirement.

"},{"location":"Experienced_Users/#temporary-local-disk-every-machine-except-kathleen","title":"Temporary local disk (every machine EXCEPT Kathleen)","text":"
#$ -l tmpdir=<size in G>\n

e.g. #$ -l tmpdir=10G requests 10 gigabytes of temporary local disk.

"},{"location":"Experienced_Users/#the-rest-of-the-script","title":"The rest of the script","text":"

You need to load any module dependencies, set up any custom environment variables or paths you need and then run the rest of your workflow.

"},{"location":"Experienced_Users/#submitting-your-jobscript","title":"Submitting your jobscript","text":"

Job scripts can be submitted with qsub, jobs can be monitored with qstat and deleted with qdel.

"},{"location":"Experienced_Users/#interactive-jobs","title":"Interactive jobs","text":"

If you need to run an interactive job, possibly with X forwarding, you can do so using qrsh. Please see our page on interactive jobs for more details.

"},{"location":"Interactive_Jobs/","title":"Interactive Job Sessions","text":"

For an interactive session, you reserve some compute nodes via the scheduler and then are logged in live, just like on the login nodes. These can be used for live visualisation, software debugging, or to work up a script to run your program without having to submit each attempt separately to the queue and wait for it to complete.

"},{"location":"Interactive_Jobs/#requesting-access","title":"Requesting Access","text":"

You will be granted an interactive shell after running a command that checks with the scheduler whether the resources you wish to use in your tests/analysis are available. Interactive sessions are requested using the qrsh command. It typically takes the form:

qrsh\u00a0-pe\u00a0mpi\u00a08\u00a0-l\u00a0mem=512M,h_rt=2:00:00\u00a0-now\u00a0no\n

In this example you are asking to run eight parallel processes within an MPI environment, 512MB RAM per process, for a period of two hours.

All job types we support on the system are supported via an interactive session (see our examples section). Likewise, all qsub options are supported like regular job submission with the difference that with qrsh they must be given at the command line, and not with any job script (or via -@).

In addition the -now option is useful when a cluster is busy. By default qrsh and qlogin jobs will run on the next scheduling cycle or give up. The -now no option tells it to keep waiting until it gets scheduled. Pressing Ctrl+C (i.e. the control key and the C key at the same time) will safely cancel the request if it doesn't seem to be able to get you a session.

More resources can be found here:

"},{"location":"Interactive_Jobs/#interactive-x-sessions","title":"Interactive X sessions","text":"

You can get an interactive X session from the head node of the job back to the login node. The way to do this is to run the qrsh command in the following generic fashion:

qrsh\u00a0<options> <command>\u00a0<arguments\u00a0to\u00a0<command>>\n

Where <command> is either a command to launch an X terminal like Xterm or Mrxvt or a GUI application like XMGrace or GaussView.

To make effective use of the X forwarding you will need to have logged in to the login node with ssh -X or some equivalent method. Here is an example of how you can get a X terminal session with the qrsh command:

qrsh\u00a0-l\u00a0mem=512M,h_rt=0:30:0\u00a0\\\n\u00a0\u00a0\u00a0\"/shared/ucl/apps/mrxvt/0.5.4/bin/mrxvt\u00a0-title\u00a0'User\u00a0Test\u00a0Node'\"\n
"},{"location":"Interactive_Jobs/#working-on-the-nodes","title":"Working on the nodes","text":"

If you want to run a command on one of your allocated nodes which is not the headnode, you can use a standard ssh command: 

ssh\u00a0<hostname> <command>\u00a0[args]\n

Where <hostname> can be obtained by inspecting the file $TMPDIR/machines.

"},{"location":"Interactive_Jobs/#gpu-test-nodes","title":"GPU test nodes","text":"

You can also run GPU jobs interactively simply by adding the -l gpu=1 or -l gpu=2 options to the qrsh command as normal.

For more information, please contact us on rc-support@ucl.ac.uk

"},{"location":"Job_Results/","title":"Where do my results go?","text":"

After submitting your job, you can use the command qstat to view the status of all the jobs you have submitted. Once you can no longer see your job on the list, this means your job has completed. To view details on jobs that have completed, you can run jobhist. There are various ways of monitoring the output of your job.

"},{"location":"Job_Results/#output-and-error-files","title":"Output and error files","text":"

When writing your job script you can either tell it to start in the directory you submit it from (-cwd), or from a particular directory (-wd <dir>), or from your home directory (the default). When your job runs, it will create files in this directory for the job's output and errors:

File Name Contents myscript.sh Your job script. myscript.sh.o12345 Output from the job. (stdout) myscript.sh.e12345 Errors, warnings, and other messages from the job that aren't mixed into the output. (stderr) myscript.sh.po12345 Output from the setup script run before a job. (\"prolog\") myscript.sh.pe12345 Output from the clean-up script run after a job. (\"epilog\")

Normally there should be nothing in the .po and .pe files, and that's fine. If you change the name of the job in the queue, using the -N option, your output and error files will use that as the filename stem instead.

Most programs will also produce separate output files, in a way that is particular to that program. Often these will be in the same directory, but that depends on the program and how you ran it.

"},{"location":"Job_Results/#grid-engine-commands","title":"Grid Engine commands","text":"

The following commands can be used to submit and monitor a job.

"},{"location":"Job_Results/#qsub","title":"Qsub","text":"

This command submits your job to the batch queue. You can also use options on the command-line to override options you have put in your job script.

Command Action qsub myscript.sh Submit the script as-is qsub -N NewName myscript.sh Submit the script but change the job's name qsub -l h_rt=24:0:0 myscript.sh Submit the script but change the maximum run-time qsub -hold_jid 12345 myscript.sh Submit the script but make it wait for job 12345 to finish qsub -ac allow=XYZ myscript.sh Submit the script but only let it run on node classes X, Y, and Z"},{"location":"Job_Results/#qstat","title":"Qstat","text":"

This command shows the status of your jobs. When you run qstat with no options, all of your jobs currently running will be displayed. By adding in the option -f -j <job-ID> you will get more detail on the specified job.

"},{"location":"Job_Results/#qdel","title":"Qdel","text":"

This command deletes your job from the queue. When deleting a job will need to run qdel <job-ID>, however qdel '*' can be used to delete all jobs. To delete a batch of jobs, creating a file with the list of job IDs that you would like to delete and placing it in the following commands will delete the following jobs: cat <filename> | xargs qdel

"},{"location":"Job_Results/#qsub-emailing","title":"Qsub emailing","text":"

We also have a mailing system that can be implemented to send emails with reminders of the status of your job through qsub. In your jobscript, or when you use qsub to submit your job, you can use the option -m. You can specify when you want an email sent to you by using the below options after qsub -m:

b Mail is sent at the beginning of the job. e Mail is sent at the end of the job. a Mail is sent when the job is aborted or rescheduled. s Mail is sent when the job is suspended. n No mail is sent. (The default.)

You can use more than one of these options by putting them together after the -m option; for example, adding the following to your job script would mean you get an email when the job begins and when it ends:

#$ -m be\n

Further resources can be found here:

"},{"location":"New_Users/","title":"Guide for New Users","text":""},{"location":"Planned_Outages/","title":"Planned Outages","text":"

The second Tuesday of every month is a maintenance day, when the following clusters should be considered at risk from 8:00AM: Myriad, Kathleen, Thomas, Young, Michael, Aristotle and the Data Science Platform. We won\u2019t necessarily perform maintenance every month, and notice by email will not always be given about maintenance day work that only puts services at risk.

Full details of outages are emailed to the cluster-specific user lists.

Generally speaking, an outage will last from the morning of the first date listed until mid-morning of the end date listed. The nodes may need to be emptied of jobs in advance ('drained'), so jobs may remain in the queue for longer before an outage begins.

If there is a notable delay in bringing the system back we will contact you after approximately midday - please don't email us at 9am on the listed end days!

After an outage, the first day or two back should be considered 'at risk'; that is, things are more likely to go wrong without warning and we might need to make adjustments.

Date Service Status Reason"},{"location":"Planned_Outages/#previous-outages","title":"Previous Outages","text":"Date Service Status Reason 29 Aug -> 1 Sept 2023 Kathleen Completed Our vendors need us to do some filesystem testing on Kathleen to prepare for the replacement of its Scratch filesystem with a new filesystem later this year. We will drain jobs for 8am on Tues 29th. Depending on how long it takes to gather this information, we may not be running jobs for the rest of the week. We will update you on Thursday around lunchtime. (Jobs were restarted on Mon 4 Sept 12:00) 11 July 2023 Myriad Completed Full shutdown for replacement of both Lustre controllers in the filesystem. There will be no login access, no jobs running. Jobs will be drained for 8am. Myriad will be returned to service during the day on Weds 12 July. 10 -> 15 May 2023 Myriad Completed Power work taking place in the datacentre that Myriad is in which may result in some cabinets losing power. We will be draining jobs for 8am on Weds 10 May and re-enabling them again once the work is complete, likely to be during Tues 16 May. 21 -> 23 Apr 2023 Young, Michael, Kathleen Completed Electrical works in the datacentre require us to switch off all three clusters. Jobs will be drained for 12:00pm on Friday 21 April. The clusters will be powered down shortly after 12:00pm. You won't be able to log in or access data from then or throughout the weekend. We will bring the clusters back up during Monday 24 April. This may take a few hours to most of the day. 31 Jan 2023 -> 1st Feb 2023 Myriad Completed Full shutdown of the system to perform firmware updates to the storage system, including all controllers and drives. Jobs will be drained for 08:00 on 31st. It is intended to make the system available again in the evening of the 1st. 23 Dec 2022 -> 3-4 Jan 2023 Young, Michael, Kathleen Complete Full shutdown during UCL Christmas closing to prevent risk of filesystem damage from instability or power issues. Jobs will be drained for 9am on 23rd, shutting down at 1pm. Will be brought back up throughout 3 Jan with intent to have jobs running at some point on 4 Jan. 13 Dec 2022 -> 14 Dec 2022 Myriad Postponed Upgrade of firmware on all Lustre storage disks. Jobs will be drained for 8am on Tues 13 Dec and we expect to re-enable jobs on Weds 14 Dec. Discovered that we would need to update the storage controller firmware as well, and possibly the OSSs (object store servers) and Lustre itself. We don't want to do this right before the Christmas closure so are postponing the update until later. 1 Nov 2022 -> 2 Nov 2022 Myriad Complete Upgrade of firmware on all Lustre storage disks. No reading or writing of files will be possible during the upgrade interval. Jobs will be drained for 8am on Tues 1 Nov and we expect to re-enable jobs on Weds 2 Nov. 27 Sep 2022 -> 28 Sep 2022 Myriad Completed Revised dates Moving switches. Jobs will be drained for 8am on 27 Sept and access to the login nodes disabled. Access should be restored during the day on Thurs 29. Myriad should be considered at risk for the rest of the week. 12 Apr 2022 -> 19 Apr 2022 Kathleen, Young, Michael Completed Datacentre work. Clusters will be drained of jobs and access to the login nodes disabled, for 4:00pm Tuesday 12th April. Clusters will be brought back into service during the day on Tuesday 19th April. They should be considered AT RISK that day. 1 Apr 2022 -> 4 Apr 2022 Kathleen, Young, Michael Completed Datacentre work. Clusters will be will be drained of jobs and access to the login nodes disabled, for 4pm Friday 1st April. Clusters will be brought back into service on Monday the 4th April. They should be considered AT RISK that day. 7 Jan 2022 -> 10 Jan 2022 Kathleen, Young, Michael Completed Datacentre power outage. Jobs will be drained for 2pm on Fri 7. Login nodes will be powered down. Clusters will be brought back online during Mon 10, should be considered at risk that day. 19 Nov 2021 Myriad Completed One of our suppliers will be onsite to replace a component. Queues are being drained. 9 Nov 2021 Myriad Completed Continued work to improve the file system. Myriad will be unavailable for the whole day. Queues are being drained for 8am. 12 Oct 2021 Myriad Completed Filesystem load tests before expansion. No jobs will be running and access to login nodes will be blocked during the test, expected to finish by end of day. 29 Sept 2021 -> 30 Sept 2021, 5 Oct 2021 Young Completed Extended: expected back in service midday Tues 5 Oct. Queues drained so ZFS can be upgraded to patch a bug causing metadata server issues. No jobs will be running. 14 Sept 2021 Myriad Planned Queues drained for 8am so that the central software installs can be redistributed over more servers. This is to help mitigate the effects of current filesystem issues. 03 Aug 2021 Young, Michael Completed Shortly after 8am: ~10 mins Gold outage for security updates. New job submission and Gold commands will not work during this. 12 July 2021 Myriad Completed Drain of d97a (Economics) nodes for 8am to update network config. May require less than the full day. Jobs will remain in queue until drain is over or run on other nodes if they can. 15 Jun 2021 -> 18 Jun 2021 Myriad Completed Datacentre network outage. No jobs will run on the 15th, and Myriad will be at risk for the rest of the week. 14 Jun 2021 Kathleen, Young and Data Science Platform Completed Datacentre network outage 8.30-9am. No login access and no jobs will run. 08 Jun 2021 Thomas Completed Storage will be at risk during an essential controller reboot. 22 May 2021 -> 24 May 2021 Michael Completed Datacentre network outage. Queues to be drained. Full outage, no access. 22 May 2021 -> 24 May 2021 Young Completed Gold and job submission expected to be unavailable while database uncontactable due to datacentre network outage. 22 May 2021 -> 24 May 2021 Myriad and Kathleen Completed Gold (i.e. high priority) job submission expected to be unavailable while database uncontactable due to datacentre network outage. 29 Mar 2021 -> 30 Mar 2021 Myriad Completed A number of GPU nodes will be reserved for the NVidia AI Bootcamp on the 29th and 30th March. Some users may experience longer than usual waiting times for GPUs, from Monday the 22nd until after the event, especially for long jobs. We apologise for any inconvenience this may cause. 30 Mar 2021 Kathleen Completed Essential Lustre filesystem maintenance. Queue draining starts 26th March, SSH access to login nodes disabled on the 30th. Full outage, no access. 23 Feb 2021 8-9:30am Young Completed Gold and job submission expected to be unavailable for two 10-min periods while network switches upgraded. 09 Feb 2021 8-1pm Young Completed Gold and job submission outage while we migrate our database server. 08 Jan 2021 -> 12 Jan 2021 Kathleen Completed Electrical work in datacentre. Logins delayed until Tues morning while tests completed. 08 Jan 2021 -> 12 Jan 2021 MMM Michael Completed Electrical work in datacentre. Logins delayed until Tues morning while tests completed. 08 Jan 2021 -> 12 Jan 2021 MMM Young Completed Electrical work in datacentre. Logins delayed until Tues morning while tests completed. 02 Jul 2020 -> 09 Jul 2020 Michael Rollback Lustre software upgrade to fix bug. Full outage, no access. Upgrade was unusable, downgrading to previous. 01 May 2020 -> 11 May 2020 Myriad Completed Storage upgrade. 20 Mar 2020 -> 30 Mar 2020 Myriad Stopped, Postponed Issues found with new storage during outage. Myriad returned to service 24th. Sync data; switch to new storage. (Important: new metadata servers). Begins midday. Full outage, no access. 16 Mar 2020 -> (was 23) 26 Mar 2020 Michael Completed Firmware upgrades to bridge old and new networks. Extended to get jobs working on new nodes. 2 Mar 2020 -> 9 Mar 2020 Michael Completed Installation of phase 2 hardware, network bridging 10 Feb 2020 -> 17 Feb 2020 Myriad Cancelled Storage upgrade to 3PB"},{"location":"Planned_Outages/#retirements","title":"Retirements","text":"

These clusters or services are being retired. Notable dates are below.

"},{"location":"Planned_Outages/#grace","title":"Grace","text":""},{"location":"Planned_Outages/#mmm-hub-thomas","title":"MMM Hub Thomas","text":"

This service has been retired as part of the MMM Hub. Some remains are still running temporarily for UCL users only.

"},{"location":"Planned_Outages/#rstudio-rstudiorcuclacuk","title":"RStudio (rstudio.rc.ucl.ac.uk)","text":""},{"location":"Remote_Access/","title":"Remote Access to Research Computing Resources","text":"

UCL's Research Computing services are accessible from inside the UCL firewall. If you wish to connect from outside, you need to either connect through a VPN or use SSH to log in to a machine accessible from outside and use that to \"jump\" through into the UCL network.

"},{"location":"Remote_Access/#connecting-to-the-jump-boxes","title":"Connecting to the jump boxes","text":"

You can connect to the jump boxes by connecting with your SSH client to:

ssh-gateway.ucl.ac.uk\n

Once connected you can then log on to the UCL RC service you are using as normal.

You can configure your ssh client to automatically connect via these jump boxes so that you make the connection in one step.

"},{"location":"Remote_Access/#single-step-logins-using-tunnelling","title":"Single-step logins using tunnelling","text":""},{"location":"Remote_Access/#linux-unix-mac-os-x","title":"Linux / Unix / Mac OS X","text":""},{"location":"Remote_Access/#on-the-command-line","title":"On the command line","text":"
# Log in to Kathleen, jumping via jump box\n# Replace ccxxxxx with your own username.\nssh -o ProxyJump=ccxxxxx@ssh-gateway.ucl.ac.uk ccxxxxx@kathleen.rc.ucl.ac.uk\n

or

# Copy 'my_file', from the machine you are logged in to, into your Scratch on Kathleen\n# Replace ccxxxxx with your own username.\nscp -o ProxyJump=ccxxxxx@ssh-gateway.ucl.ac.uk my_file ccxxxxx@kathleen.rc.ucl.ac.uk:~/Scratch/\n

This tunnels through the jump box service in order to get you to your destination - you'll be asked for your password twice, once for each machine. You can use this to log in or to copy files.

You may also need to do this if you are trying to reach one cluster from another and there is a firewall in the way.

Useful resources can be found here:

"},{"location":"Remote_Access/#using-a-config-file","title":"Using a config file","text":"

You can create a config which does this without you needing to type it every time.

Inside your ~/.ssh directory on your local machine, add the below to your config file (or create a file called config if you don't already have one).

Generically, it should be of this form where <name> can be anything you want to call this entry. You can use these as short-hand names when you run ssh.

Host <name>\n   User <remote_user_id>\n   HostName <remote_hostname>\n   proxyCommand ssh -W <remote_hostname>:22 <remote_user_id>@ssh-gateway.ucl.ac.uk\n

This proxyCommand option causes the commands you type in your client to be forwarded on over a secure channel to the specified remote host.

On newer versions of OpenSSH, you can use ProxyJump <remote_user_id>@ssh-gateway.ucl.ac.uk instead of this proxyCommand line.

Here are some examples - you can have as many of these as you need in your config file.

Host myriad\n   User ccxxxxx\n   HostName myriad.rc.ucl.ac.uk\n   proxyCommand ssh -W myriad.rc.ucl.ac.uk:22 ccxxxxx@ssh-gateway.ucl.ac.uk\n\nHost kathleen01\n   User ccxxxxx\n   HostName login01.kathleen.rc.ucl.ac.uk\n   proxyCommand ssh -W login01.kathleen.rc.ucl.ac.uk:22 ccxxxxx@ssh-gateway.ucl.ac.uk\n\nHost aristotle\n   User ccxxxxx\n   HostName aristotle.rc.ucl.ac.uk\n   proxyCommand ssh -W aristotle.rc.ucl.ac.uk:22 ccxxxxx@ssh-gateway.ucl.ac.uk\n

You can now just type ssh myriad or scp file1 aristotle:~ and you will go through the jump box. You'll be asked for login details twice since you're logging in to two machines, the jump box and your endpoint.

"},{"location":"Remote_Access/#file-storage-on-the-gateway-servers","title":"File storage on the Gateway servers","text":"

The individual servers in the pool for the Gateway service have extremely limited file storage space, intentionally, and should not be used for storing files - if you need to transfer files you should use the two-step process above. This storage should only be used for SSH configuration files.

This storage is not mirrored across the jump boxes which means if you write a file to your home directory, you will not be able to read it if you are allocated to another jump box next time you log in.

"},{"location":"Remote_Access/#key-management","title":"Key management","text":"

Warning

If you use SSH keys you absolutely MUST NOT STORE UNENCRYPTED PRIVATE KEYS ON THIS OR ANY OTHER MULTI-USER COMPUTER. We will be running regular scans of the filesystem to identify and then block unencrypted key pairs across our services.

There are currently two servers in the pool, internally named ejp-gateway01 and ejp-gateway02.

Because the /home filesystem is not shared across the jump boxes, you need to sync SSH configuration files like ~/.ssh/authorized_keys across all the available jump boxes so that the change takes effect whichever jump box you are allocated to.

You can see which machine you are logged into by the bash prompt.

So for example, if on ejp-gateway02 then do:

[ccaaxxx@ad.ucl.ac.uk@ejp-gateway02 ~]$ scp -r ~/.ssh ejp-gateway01:\n\nPassword:\nknown_hosts 100% 196 87.1KB/s 00:00\nauthorized_keys 100% 0 0.0KB/s 00:00\n[ccaaxxx@ad.ucl.ac.uk@ejp-gateway02 ~]$\n

and similarly if on ejp-gateway01 do scp -r ~/.ssh ejp-gateway02:

"},{"location":"Status_page/","title":"Status of machines","text":"

This page outlines that status of each of the machines managed by the Research Computing team at UCL. We endeavour to keep this page as up-to-date as possible but there might be some delay. Also there are spontaneous errors that we have to deal with (i.e. high load on login nodes) but feel free to report them to rc-support@ucl.ac.uk. Finally, details of our planned outages can be found here.

"},{"location":"Status_page/#myriad","title":"Myriad","text":""},{"location":"Status_page/#kathleen","title":"Kathleen","text":""},{"location":"Status_page/#young","title":"Young","text":""},{"location":"Status_page/#michael","title":"Michael","text":""},{"location":"Status_page/#thomas","title":"Thomas","text":""},{"location":"Terms_and_Conditions/","title":"Terms and Conditions","text":"
  1. All use of Research Computing Platforms is subject to the UCL Computing Regulations.
  2. All users will be required to renew their account once per year. Users will receive a reminder one month prior to suspension of their Myriad account sent to their Live@UCL e-mail address. Funding information will need to be provided upon application, and publication information upon renewal.
  3. Users are forbidden from performing production runs on the login nodes.
  4. The Research Computing Platform Services Team reserve the right to suspend or ban without prior warning any use of the system which impairs its operation.
  5. With the exception of in cases where there is imminent harm or risk to the service, the Research Computing Platform Services Team will not access your files without permission.1
  6. Official service notifications are sent to the myriad-users (or the equivalent for other services) mailing list. Users are automatically subscribed to this list using their Live@UCL e-mail address and should read notices sent there.
  7. The Research Computing Platform Services Team reserve the right to suspend users' accounts, without notice, in the event of a user being the subject of any UCL disciplinary procedure, or where a user is found to be in breach of UCL\u2019s Computing Regulations or best practice guidelines regarding password management, as provided by Information Services Division.
  8. Users are required to acknowledge their use of Myriad and associated research computing services in any publications describing research that has been conducted, in any part, on Myriad. The following words should be used: \"The authors acknowledge the use of the UCL Myriad High Performance Computing Facility (Myriad@UCL), and associated support services, in the completion of this work\".
  9. All support requests should be sent by e-mail to rc-support@ucl.ac.uk.

  1. When you submit a job script, a copy is made, and that copy is public. This may appear as an exception to this term. Please do not include passwords or API keys in your job scripts.\u00a0\u21a9

"},{"location":"UCL_Service_For_Me/","title":"Which service(s) at UCL are right for me?","text":"

Depending on the type of jobs you would like to run each cluster can have different requirements that your jobs must meet. When you submit your user account application form you will be given access to the cluster depending on resources selected.

Currently we categorise intended workloads into ones that use:

Each cluster has its own specifications for the types of jobs that they run which is all dependable on list above. The cluster machines we have available are:

Myriad is designed to be most suitable for serial work, including large numbers of serial jobs, and multi-threaded jobs (using e.g. OpenMP). It also includes a small number of GPUs for development or testing work. It went into service in July 2018. See Myriad.

Kathleen is intended for multi-node jobs (using e.g. MPI) and went into service in Feb 2020 as a replacement for Grace. We recommend using Kathleen if you intend to use more than 36 cores per job. See Kathleen.

Thomas is the UK's Tier 2 Materials and Molecular Modelling Hub. It is accessible by members of partner institutions and relevant consortia, and is for materials and molecular modelling work only. It has separate access procedures from UCL's central clusters. Access is managed by a Point of Contact from the relevant institution or consortia, not by Research Computing. See Thomas.

Michael is an extension to the UCL-hosted Hub for Materials and Molecular Modelling, an EPSRC-funded Tier 2 system providing large scale computation to UK researchers; and delivers computational capability for the Faraday Institution, a national institute for electrochemical energy storage science and technology. Access is managed by a Point of Contact from the Faraday Institution, not by Research Computing. See Michael.

"},{"location":"howto/","title":"How do I?","text":"

I have an account, now:

"},{"location":"howto/#how-do-i-log-in","title":"How do I log in?","text":"

Logging in is most straightforward if you are inside the UCL firewall. If you are logging in from home or other external networks then you first have to get on to the UCL network.

"},{"location":"howto/#linux-unix-mac-os-x","title":"Linux / Unix / Mac OS X","text":"

Use the terminal and type the below command to secure shell (ssh) into the machine you wish to access. Replace <your_UCL_user_id> with your central UCL username, and <system_name> with the name of the machine you want to log in to, eg. myriad, kathleen, aristotle. 

ssh <your_UCL_user_id>@<system_name>.rc.ucl.ac.uk\n
"},{"location":"howto/#windows","title":"Windows","text":"

On Windows you need something that will give you a suitable terminal and ssh - usually PuTTY, or on Windows 10 you can use OpenSSH from a command prompt and type the same ssh command as the Linux instructions.

"},{"location":"howto/#using-putty","title":"Using PuTTY","text":"

PuTTY is a common SSH client on Windows and is available on Desktop@UCL. You can find it under: Start > P > PuTTY 0.76 (64-bit) > PuTTY 0.76 or type \"putty\" in the toolbar's search box.

You will need to create an entry for the host you are connecting to with the settings below. If you want to save your settings, give them an easily-identifiable name in the \"Saved Sessions\" box and press \"Save\". Then you can select it and \"Load\" next time you use PuTTY.

You will then be asked to enter your username and password. Only enter your username, not @<system_name>.rc.ucl.ac.uk. The password field will remain entirely blank when you type in to it - it does not show placeholders to indicate you have typed something.

The first time you log in to a new server, you'll get a popup telling you that the server's host key is not cached in the registry - this is normal and is because you have never connected to this server before. If you want to, you can check the host fingerprint against our current key fingerprints.

"},{"location":"howto/#logging-in-from-outside-the-ucl-firewall","title":"Logging in from outside the UCL firewall","text":"

You will need to either use the UCL Virtual Private Network or ssh in to UCL's Gateway system ssh-gateway.ucl.ac.uk first. From there you can then ssh in to our systems. 

ssh <your_UCL_user_id>@ssh-gateway.ucl.ac.uk\nssh <your_UCL_user_id>@<system_name>.rc.ucl.ac.uk\n

Advanced: If you find you need to go via the Gateway often, you can set up this jump automatically, see Single-step logins using tunnelling

"},{"location":"howto/#china-connect","title":"China Connect","text":"

If you are on China Connect, you will still need to ssh in to a UCL gateway first as described above.

"},{"location":"howto/#login-problems","title":"Login problems","text":"

If you experience difficulties with your login, please make sure that you are typing your UCL user ID and your password correctly. If you have recently updated your password, it takes some hours to propagate to all UCL systems.

If you still cannot get access but can access other UCL services like the SSH Gateway, please contact us on rc-support@ucl.ac.uk. Your account may have expired, or you may have gone over quota.

If you cannot access anything, please see UCL MyAccount - you may need to request a password reset from the Service Desk.

If you get a host key error message, you will need to delete old host keys - continue reading!

"},{"location":"howto/#remote-host-identification-has-changed","title":"Remote host identification has changed","text":"

When you log in via SSH, it keeps a record of the host key for the server you logged in to in your .ssh/known_hosts file in your home directory, on the machine you are logging in from. This helps make sure you are connecting directly to the server you think you are, but can cause warnings to show up if the host key on that machine has genuinely changed (usually because of an update or reinstall).

Check the host key warning against our current key fingerprints:

The error message looks like this if you are using OpenSSH in a terminal:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\n@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @\n@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\nIT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!\nSomeone could be eavesdropping on you right now (man-in-the-middle attack)!\nIt is also possible that a host key has just been changed.\nThe fingerprint for the ECDSA key sent by the remote host is\nSHA256:7FTryal3mIhWr9CqM3EPPeXsfezNk8Mm8HPCCAGXiIA.\nPlease contact your system administrator.\nAdd correct host key in /Users/uccaxxx/.ssh/known_hosts to get rid of this message.\nOffending ECDSA key in /Users/uccaxxx/.ssh/known_hosts:11\nECDSA host key for myriad.rc.ucl.ac.uk has changed and you have requested strict checking.\nHost key verification failed.\nKilled by signal 1.\n

This tells you that the old key is in line 11 of your known_hosts file. Sometimes it will give you a direct command you can run to remove that specific key:

ssh-keygen -R myriad.rc.ucl.ac.uk\n

or you can manually delete line 11 yourself in a text editor.

If you are logging in via the Gateway, you will need to remove the old key there too. On the Gateway, nano and vim are available text editors. If you are not already familiar with vim, use nano - it has the command shortcuts shown at the bottom, where ^O means press Ctrl and then the letter o.

# to open the file for editing in nano\nnano ~/.ssh/known_hosts\n

Once you have removed the old host key you will be able to ssh in again. The first time you log in to an unknown server you will get a message like this:

The authenticity of host 'myriad.rc.ucl.ac.uk can't be established.\nECDSA key fingerprint is SHA256:7FTryal3mIhWr9CqM3EPPeXsfezNk8Mm8HPCCAGXiIA.\nAre you sure you want to continue connecting (yes/no)?\n

Typing yes will allow you to continue logging in.

PuTTY will display a warning and you can choose to continue or not.

WinSCP will say Server's host key does not match the one that WinSCP has in cache. and you will have the option to update the key.

"},{"location":"howto/#mac-os-x-connection-failures","title":"Mac OS X connection failures","text":"

If you are on Mac OS X and getting many ssh connection failures and broken pipe messages when trying to log in, try adding an ssh timeout to your ssh command:

ssh -o ConnectTimeout=10 <your_UCL_user_id>@myriad.rc.ucl.ac.uk\n

This has particularly been a problem with Mac OS X Big Sur when using the VPN.

"},{"location":"howto/#how-do-i-transfer-data-onto-the-system","title":"How do I transfer data onto the system?","text":"

You can transfer data to and from our systems using any program capable of using the Secure Copy (SCP) protocol. This uses the same SSH system as you use to log in to a command line session, but then transfers data over it. This means that if you can use SSH to connect to a system, you can usually use SCP to transfer files to it.

"},{"location":"howto/#copying-files-using-linux-or-mac-os-x","title":"Copying files using Linux or Mac OS X","text":"

You can use the command-line utilities scp, sftp or rsync to copy your data about. You can also use a graphical client (Transmit, CyberDuck, FileZilla).

"},{"location":"howto/#scp","title":"scp","text":"

This will copy a data file from somewhere on your local machine to a specified location on the remote machine (Myriad etc).

scp <local_data_file> <remote_user_id>@<remote_hostname>:<remote_path>\n
# Example: copy myfile from your local current directory into Scratch on Myriad\nscp myfile ccxxxxx@myriad.rc.ucl.ac.uk:~/Scratch/\n

This will do the reverse, copying from the remote machine to your local machine. (This is still run from your local machine).

scp <remote_user_id>@<remote_hostname>:<remote_path><remote_data_file> <local_path>\n
# Example: copy myfile from Myriad into the Backups directory in your local current directory\nscp ccxxxxx@myriad.rc.ucl.ac.uk:~/Scratch/myfile Backups/\n
"},{"location":"howto/#sftp","title":"sftp","text":"

You can use sftp to log in to the remote machine, navigate through directories and use put and get to copy files from and to your local machine. lcd and lls are local equivalents of cd and ls so you can navigate through your local directories as you go. 

sftp <remote_user_id>@<remote_hostname>\ncd <remote_path>\nget <remote_file>\nlcd <local_path>\nput <local_file>\n
# Example: download a copy of file1 into your local current directory,\n# change local directory and upload a copy of file2\nsftp ccxxxxx@myriad.rc.ucl.ac.uk\ncd Scratch/files\nget file1\nlcd ../files_to_upload\nput file2\n
"},{"location":"howto/#rsync","title":"rsync","text":"

rsync is used to remotely synchronise directories, so can be used to only copy files which have changed. Have a look at man rsync as there are many options.

"},{"location":"howto/#copying-files-using-windows-and-winscp","title":"Copying files using Windows and WinSCP","text":"

WinSCP is a graphical client that you can use for scp or sftp.

  1. The login/create new session screen will open if this is the first time you are using WinSCP.
  2. You can choose SFTP or SCP as the file protocol. If you have an unstable connection with one, you may wish to try the other. SCP is probably generally better.
  3. Fill in the hostname of the machine you wish to connect to, your username and password.
  4. Click Save and give your settings a useful name.
  5. You'll then be shown your list of Stored sessions, which will have the one you just created.
  6. Select the session and click Login.
"},{"location":"howto/#mobaxterm","title":"MobaXterm","text":"

If using MobaXterm, you may need to set a password for the left side file manager panel separately as well as for the main panel, to allow you to drag and drop files and have them transferred to the cluster.

"},{"location":"howto/#transferring-files-from-outside-the-ucl-firewall","title":"Transferring files from outside the UCL firewall","text":"

As when logging in, when you are outside the UCL firewall you will need a method to connect inside it before you copy files.

You can use the UCL Virtual Private Network and scp direct to our systems or you can do some form of SSH tunnelling.

"},{"location":"howto/#single-step-logins-using-tunnelling","title":"Single-step logins using tunnelling","text":""},{"location":"howto/#linux-unix-mac-os-x_1","title":"Linux / Unix / Mac OS X","text":""},{"location":"howto/#on-the-command-line","title":"On the command line","text":"
# Log in to Myriad, jumping via the Gateway (replace ccxxxxx with your own username)\nssh -o ProxyJump=ccxxxxx@ssh-gateway.ucl.ac.uk ccxxxxx@myriad.rc.ucl.ac.uk\n

or

# Copy 'my_file' from the machine you are logged in to into your Scratch on Grace\n# Replace ccxxxxx with your own username.\nscp -o ProxyJump=ccxxxxx@ssh-gateway.ucl.ac.uk my_file ccxxxxx@myriad.rc.ucl.ac.uk:~/Scratch/\n

This tunnels through the Gateway in order to get you to your destination - you'll be asked for your password twice, once for each machine. You can use this to log in or to copy files.

You may also need to do this if you are trying to reach one cluster from another and there is a firewall in the way.

"},{"location":"howto/#using-a-config-file","title":"Using a config file","text":"

You can create a config file which does this without you needing to type it every time.

Inside your ~/.ssh directory on your local machine, add the below to your config file (or create a file called config if you don't already have one).

Generally, it should be of this form where <name> can be anything you want to call this entry.

Host <name>\n   User <remote_user_id>\n   HostName <remote_hostname>\n   proxyCommand ssh -W <remote_hostname>:22 <remote_user_id>@ssh-gateway.ucl.ac.uk\n

This causes the commands you type in your client to be forwarded on over a secure channel to the specified remote host.

Here are some examples - you can have as many of these as you need in your config file.

Host myriad\n   User ccxxxxx\n   HostName myriad.rc.ucl.ac.uk\n   proxyCommand ssh -W myriad.rc.ucl.ac.uk:22 ccxxxxx@ssh-gateway.ucl.ac.uk\n\nHost myriad12\n   User ccxxxxx\n   HostName login12.myriad.rc.ucl.ac.uk\n   proxyCommand ssh -W login12.myriad.rc.ucl.ac.uk:22 ccxxxxx@ssh-gateway.ucl.ac.uk\n\nHost aristotle\n   User ccxxxxx\n   HostName aristotle.rc.ucl.ac.uk\n   proxyCommand ssh -W aristotle.rc.ucl.ac.uk:22 ccxxxxx@ssh-gateway.ucl.ac.uk\n

You can now just type ssh myriad or scp file1 aristotle:~ and you will go through the Gateway. You'll be asked for login details twice since you're logging in to two machines, a Gateway server and your endpoint.

"},{"location":"howto/#windows-winscp","title":"Windows - WinSCP","text":"

WinSCP can also set up SSH tunnels.

  1. Create a new session as before, and tick the Advanced options box in the bottom left corner.
  2. Select Connection > Tunnel from the left pane.
  3. Tick the Connect through SSH tunnel box and enter the hostname of the gateway you are tunnelling through, for example ssh-gateway.ucl.ac.uk
  4. Fill in your username and password for that host. (Central UCL ones for the Gateway).
  5. Select Session from the left pane and fill in the hostname you want to end up on after the tunnel.
  6. Fill in your username and password for that host and set the file protocol to SCP.
  7. Save your settings with a useful name.
"},{"location":"howto/#creating-a-tunnel-that-other-applications-can-use","title":"Creating a tunnel that other applications can use","text":"

Some applications do not read your SSH config file and also cannot set up tunnels themselves, but can use one that you have created separately. FileZilla in particular is something you may want to do this with to transfer your files directly to the clusters from outside UCL using a graphical client.

"},{"location":"howto/#ssh-tunnel-creation-using-a-terminal","title":"SSH tunnel creation using a terminal","text":"

You can do this in Linux, Mac OS X and the Windows Command Prompt on Windows 10 and later.

Set up a tunnel between a port on your local computer (this is using 3333 as it is unlikely to be in use, but you can pick different ones) to Myriad's port 22 (which is the standard port for ssh), going via a UCL gateway.

# replace ccxxxxx with your UCL username\nssh -L 3333:myriad.rc.ucl.ac.uk:22 ccxxxxx@ssh-gateway.ucl.ac.uk \n

You may also want to use the -N option to tell it not to execute any remote commands and -f to put this command into the background if you want to continue to type other commands into the same terminal.

The tunnel now exists, and localhost:3333 on your computer connects to Myriad.

You can do this with ports other than 22 if you are not wanting to ssh in but to instead connect with a local browser to something running on Myriad. Here the port remains as 3333, something could be launched on that port on Myriad and your browser could be pointed at localhost:3333 to connect to it.

# replace ccxxxxx with your UCL username\nssh -L 3333:myriad.rc.ucl.ac.uk:3333 ccxxxxx@ssh-gateway.ucl.ac.uk\n

Do not leave things like this running for long periods on the login nodes.

"},{"location":"howto/#ssh-tunnel-creation-using-putty","title":"SSH tunnel creation using PuTTY","text":"

On Windows you can also set up a tunnel using PuTTY.

"},{"location":"howto/#connect-to-your-tunnel-with-an-application-like-filezilla","title":"Connect to your tunnel with an application (like FileZilla)","text":"

You can then tell your application to connect to localhost:3333 instead of Myriad. If it has separate boxes for hostname and port, put localhost as the hostname and 3333 as the port.

"},{"location":"howto/#managing-your-quota","title":"Managing your quota","text":"

After using lquota to see your total usage, you may wish to find what is using all your space.

du is a command that gives you information about your disk usage. Useful options are:

du -ch <dir>\ndu -h --max-depth=1\n

The first will give you a summary of the sizes of directory tree and subtrees inside the directory you specify, using human-readable sizes with a total at the bottom. The second will show you the totals for all top-level directories relative to where you are, plus the grand total. These can help you track down the locations of large amounts of data if you need to reduce your disk usage.

"},{"location":"howto/#how-do-i-connect-to-ucl-group-folders-n-drive-or-other-smb-mounts","title":"How do I connect to UCL group folders, N drive or other smb mounts?","text":"

You may have data stored in a UCL group folder (S drive) that you normally mount using smb. You can use smbclient to copy the files across onto Myriad (you do want them to be copied onto Myriad before you run any jobs using them, otherwise the compute node will be sitting there waiting for the copy to complete before it can do anything useful).

If the address you are trying to mount looks like smb://ad.ucl.ac.uk/groupfolders then you would do this:

smbclient //ad.ucl.ac.uk/groupfolders \n

This will give you a prompt where you can access that storage in an ftp-like way, where you can use get commands to copy files from there on to Myriad, or put commands to copy data into there from Myriad.

You can look at man smbclient on Myriad for the manual.

If you get an error like this:

\\ad.ucl.ac.ukgroupfolders: Not enough '\\' characters in service\n

then you need to change the format from \\\\ad.ucl.ac.uk\\groupfolders to //ad.ucl.ac.uk/groupfolders instead.

"},{"location":"howto/#ucl-n-drive","title":"UCL N drive","text":"

The N drive (Windows filestore, Filestore@UCL) can also be accessed using smb. To find out what smb address to use, look at Adding your home N drive to a Linux machine. Then use smbclient as above to connect to it.

"},{"location":"howto/#how-do-i-connect-out-to-an-ftp-server","title":"How do I connect out to an FTP server?","text":"

You cannot connect in to Myriad using FTP (we only allow SFTP access) but you can connect out to FTP servers run by other people.

Load the GNU inetutils module which provides ftp, telnet and tftp clients.

module load inetutils/1.9.4\n\n# connect to your desired server\nftp servername.ac.uk\n

You can then use put and get commands to put data on the remote FTP server or download it from there to Myriad.

"},{"location":"howto/#how-do-i-submit-a-job-to-the-scheduler","title":"How do I submit a job to the scheduler?","text":"

To submit a job to the scheduler you need to write a jobscript that contains the resources the job is asking for and the actual commands you want to run. This jobscript is then submitted using the qsub command.

qsub myjobscript\n

It will be put in to the queue and will begin running on the compute nodes at some point later when it has been allocated resources.

"},{"location":"howto/#passing-in-qsub-options-on-the-command-line","title":"Passing in qsub options on the command line","text":"

The #$ lines in your jobscript are options to qsub. It will take each line which has #$ as the first two characters and use the contents beyond that as an option.

You can also pass options directly to the qsub command and this will override the settings in your script. This can be useful if you are scripting your job submissions in more complicated ways.

For example, if you want to change the name of the job for this one instance of the job you can submit your script with:

qsub -N NewName myscript.sh\n

Or if you want to increase the wall-clock time to 24 hours:

qsub -l h_rt=24:0:0 myscript.sh\n

You can submit jobs with dependencies by using the -hold_jid option. For example, the command below submits a job that won't run until job 12345 has finished:

qsub -hold_jid 12345 myscript.sh\n

You may specify node type with the -ac allow= flags as below: 

qsub -ac allow=XYZ myscript.sh\n

That would restrict the job to running on nodes of type X, Y or Z (the older Legion nodes).

Note that for debugging purposes, it helps us if you have these options inside your jobscript rather than passed in on the command line whenever possible. We (and you) can see the exact jobscript that was submitted for every job that ran but not what command line options you submitted it with.

"},{"location":"howto/#checking-your-previous-jobscripts","title":"Checking your previous jobscripts","text":"

If you want to check what you submitted for a specific job ID, you can do it with the scriptfor utility.

scriptfor 12345\n

As mentioned above, this will not show any command line options you passed in.

"},{"location":"howto/#how-do-i-monitor-a-job","title":"How do I monitor a job?","text":""},{"location":"howto/#qstat","title":"qstat","text":"

The qstat command shows the status of your jobs. By default, if you run it with no options, it shows only your jobs (and no-one else\u2019s). This makes it easier to keep track of your jobs.

The output will look something like this:

job-ID  prior   name       user         state submit/start at     queue                          slots ja-task-ID \n-----------------------------------------------------------------------------------------------------------------\n123454 2.00685 DI_m3      ccxxxxx      Eqw   10/13/2017 15:29:11                                    12 \n123456 2.00685 DI_m3      ccxxxxx      r     10/13/2017 15:29:11 Yorick@node-x02e-006               24 \n123457 2.00398 DI_m2      ucappka      qw    10/12/2017 14:42:12                                    1 \n

This shows you the job ID, the numeric priority the scheduler has assigned to the job, the name you have given the job, your username, the state the job is in, the date and time it was submitted at (or started at, if it has begun), the head node of the job, the number of 'slots' it is taking up, and if it is an array job the last column shows the task ID.

The queue name (Yorick here) is generally not useful. The head node name (node-x02e-006) is useful - the node-x part tells you this is an X-type node.

If you want to get more information on a particular job, note its job ID and then use the -f and -j flags to get full output about that job. Most of this information is not very useful.

qstat -f -j 12345\n
"},{"location":"howto/#job-states","title":"Job states","text":"

Many jobs cycling between Rq and Rr generally means there is a dodgy compute node which is failing pre-job checks, but is free so everything tries to run there. In this case, let us know and we will investigate.

If a job stays in t or dr state for a long time, the node it was on is likely to be unresponsive - again let us know and we'll investigate.

A job in Eqw will remain in that state until you delete it - you should first have a look at what the error was with qexplain.

"},{"location":"howto/#qexplain","title":"qexplain","text":"

This is a utility to show you the non-truncated error reported by your job. qstat -j will show you a truncated version near the bottom of the output.

qexplain 123454\n
"},{"location":"howto/#qdel","title":"qdel","text":"

You use qdel to delete a job from the queue.

qdel 123454\n

You can delete all your jobs at once:

qdel '*'\n
"},{"location":"howto/#more-scheduler-commands","title":"More scheduler commands","text":"

Have a look at man qstat and note the commands shown in the SEE ALSO section of the manual page. Exit the manual page and then look at the man pages for those. (You will not be able to run all commands).

"},{"location":"howto/#nodesforjob","title":"nodesforjob","text":"

This is a utility that shows you the current percentage load, memory used and swap used on the nodes your job is running on. If your job is sharing the node with other people's jobs, it will show you the total resources in use, not just those used by your job. This is a snapshot of the current time and resource usage may change over the course of your job. Bear in mind that memory use in particular can increase over time as your job runs.

If a cluster has hyperthreading enabled and you aren't using it, full load will show as 50% and not 100% - this is normal and not a problem.

For a parallel job, very low (or zero) usage of any of the nodes suggests your job is either not capable of running over multiple nodes, or not partitioning its work effectively - you may be asking for more cores than it can use, or asking for a number of cores that doesn't fit well into the node sizes, leaving many idle.

[uccacxx@login02 ~]$ nodesforjob 1234\nNodes for job 1234:\n  Primary:\n    node-r99a-238:  103.1 % load, 12.9 % memory used, 0.1% swap used\n  Secondaries:\n    node-r99a-206:  1.7 % load, 1.6 % memory used, 0.1% swap used\n    node-r99a-238:  103.1 % load, 12.9 % memory used, 0.1% swap used\n    node-r99a-292:  103.1 % load, 12.9 % memory used, 0.1% swap used\n    node-r99a-651:  1.6 % load, 3.2 % memory used, 0.1% swap used\n

The above example shows a multi-node job, so all the usage belongs to this job itself. It is running on four nodes, and node-r99a-238 is the head node (the one that launched the job) and shows up in both Primary and Secondaries. The load is very unbalanced - it is using two nodes flat out, and two are mostly doing nothing. Memory use is low. Swap use is essentially zero.

"},{"location":"howto/#jobhist","title":"jobhist","text":"

Once a job ends, it no longer shows up in qstat. To see information about your finished jobs - when they started, when they ended, what node they ran on - type jobhist.

[uccacxx@login02 ~]$ jobhist\n        FSTIME        |       FETIME        |   HOSTNAME    |  OWNER  | JOB NUMBER | TASK NUMBER | EXIT STATUS |   JOB NAME    \n----------------------+---------------------+---------------+---------+------------+-------------+-------------+---------------\n  2020-06-17 16:31:12 | 2020-06-17 16:34:19 | node-h00a-010 | uccacxx |    3854822 |           0 |           0 | m_job   \n  2020-06-17 16:56:50 | 2020-06-17 16:56:52 | node-d00a-023 | uccacxx |    3854836 |           0 |           1 | k_job  \n  2020-06-17 17:21:12 | 2020-06-17 17:21:46 | node-d00a-012 | uccacxx |    3854859 |           0 |           0 | k_job\n

FSTIME - when the job started running on the node FETIME - when the job ended HOSTNAME - the head node of the job (if it ran on multiple nodes, it only lists the first) TASK NUMBER - if it was an array job, it will have a different number here for each task

This shows jobs that finished in the last 24 hours by default. You can search for longer as well:

jobhist --hours=200\n

If a job ended and didn't create the files you expect, check the start and end times to see whether it ran out of wallclock time.

If a job only ran for seconds and didn't produce the expected output, there was probably something wrong in your script - check the .o and .e files in the directory you submitted the job from for errors.

"},{"location":"howto/#how-do-i-run-interactive-jobs","title":"How do I run interactive jobs?","text":"

Sometimes you need to run interactive programs, sometimes with a GUI. This can be achieved through qrsh. We have a detailed guide to running interactive jobs.

"},{"location":"howto/#how-do-i-estimate-what-resources-to-request-in-my-jobscript","title":"How do I estimate what resources to request in my jobscript?","text":"

It can be difficult to know where to start when estimating the resources your job will need. One way you can find out what resources your jobs need is to submit one job which requests far more than you think necessary, and gather data on what it actually uses. If you aren't sure what 'far more' entails, request the maximum wallclock time and job size that will fit on one node, and reduce this after you have some idea.

Run your program as:

 /usr/bin/time --verbose myprogram myargs\n

where myprogram myargs is however you normally run your program, with whatever options you pass to it.

When your job finishes, you will get output about the resources it used and how long it took - the relevant one for memory is maxrss (maximum resident set size) which roughly tells you the largest amount of memory it used.

If your job is not completing successfully or you need to know how the memory usage changes throughout the job, there is a tool called Ruse that can measure this for you.

Run your program as:

module load ruse/2.0\n\n# sample the current memory usage every 120s and output each step to stdout\nruse --stdout --time=120 -s myprogram myargs\n

where myprogram myargs is however you normally run your program, with whatever options you pass to it.

Remember that memory requests in your jobscript are always per core, so check the total you are requesting is sensible - if you increase it too much you may end up with a job that cannot be submitted.

You can also look at nodesforjob while a job is running to see a snapshot of the memory, swap and load on the nodes your job is running on.

"},{"location":"howto/#how-can-i-see-what-types-of-node-a-cluster-has","title":"How can I see what types of node a cluster has?","text":"

As well as looking at the cluster-specific page in this documentation for more details (for example Myriad), you can run nodetypes, which will give you basic information about the nodes that exist in that cluster.

[uccacxx@login12 ~]$ nodetypes\n    3 type * nodes: 36 cores, 188.4G RAM\n    7 type B nodes: 36 cores,   1.5T RAM\n   66 type D nodes: 36 cores, 188.4G RAM\n    9 type E nodes: 36 cores, 188.4G RAM\n    1 type F nodes: 36 cores, 188.4G RAM\n   55 type H nodes: 36 cores, 188.4G RAM\n    3 type I nodes: 36 cores,   1.5T RAM\n    2 type J nodes: 36 cores, 188.4G RAM\n

This shows how many of each letter-labelled nodetype the cluster has, then the number of cores and amount of memory the node is reporting it has. It also shows the cluster has some utility nodes - those are part of the infrastructure. The * nodes are the login nodes.

"},{"location":"howto/#how-do-i-run-a-graphical-program","title":"How do I run a graphical program?","text":"

To run a graphical program on the cluster and be able to view the user interface on your own local computer, you will need to have an X-Windows Server installed on your local computer and use X-forwarding.

"},{"location":"howto/#x-forwarding-on-linux","title":"X-forwarding on Linux","text":"

Desktop Linux operating systems already have X-Windows installed, so you just need to ssh in with the correct flags.

You need to make sure you use either the -X or -Y (look at man ssh for details) flags on all ssh commands you run to establish a connection to the cluster.

For example, connecting from outside of UCL:

ssh -X <your_UCL_user_id>@ssh-gateway.ucl.ac.uk\n

and then

ssh -X <your_UCL_user_id>@myriad.rc.ucl.ac.uk\n

A video walkthrough of running remote applications using X11, X-forwarding on compute nodes.

"},{"location":"howto/#x-forwarding-on-mac-os-x","title":"X-forwarding on Mac OS X","text":"

You will need to install XQuartz to provide an X-Window System for Mac OS X. (Previously known as X11.app).

You can then follow the Linux instructions using Terminal.app.

"},{"location":"howto/#x-forwarding-on-windows","title":"X-forwarding on Windows","text":"

You will need:

Exceed is available on Desktop@UCL machines and downloadable from the UCL software database. Xming is open source (and mentioned here without testing).

"},{"location":"howto/#exceed-on-desktopucl","title":"Exceed on Desktop@UCL","text":"
  1. Load Exceed. You can find it under Start > All Programs > Applications O-P > Open Text Exceed 14 > Exceed
  2. Open PuTTY (Applications O-P > PuTTY)
  3. In PuTTY, set up the connection with the host machine as usual:
  4. Then, from the Category menu, select Connection > SSH > X11 for 'Options controlling SSH X11 forwarding'.
  5. Return to the session menu and save these settings with a new identifiable name for reuse in future.
  6. Click 'Open' and login to the host as usual
  7. To test that X-forwarding is working, try running nedit which is a text editor in our default modules.

If nedit works, you have successfully enabled X-forwarding for graphical applications.

"},{"location":"howto/#installing-xming","title":"Installing Xming","text":"

Xming is a popular open source X server for Windows. These are instructions for using it alongside PuTTY. Other SSH clients and X servers are available. We cannot verify how well it may be working.

  1. Install both PuTTY and Xming if you have not done so already. During Xming installation, choose not to install an SSH client.
  2. Open Xming - the Xming icon should appear on the task bar.
  3. Open PuTTY
  4. Set up PuTTY as shown in the Exceed section.
"},{"location":"Background/Cluster_Computing/","title":"Cluster Computing","text":""},{"location":"Background/Cluster_Computing/#what-is-a-cluster","title":"What is a cluster?","text":"

In this context, a cluster is a collection of computers (often referred to as \"nodes\"). They're networked together with some shared storage and a scheduling system that lets people run programs on them without having to enter commands \"live\".

"},{"location":"Background/Cluster_Computing/#why-would-i-want-to-use-one","title":"Why would I want to use one?","text":"

Some researchers have programs that require a lot of compute power, like simulating weather patterns or the quantum behaviour of molecules.

Others have a lot of data to process, or need to simulate a lot of things at once, like simulating the spread of disease or assembling parts of DNA into a genome.

Often these kinds of work are either impossible or would take far too long to do on a desktop or laptop computer, as well as making the computer unavailable to do everyday tasks like writing documents or reading papers.

By running the programs on the computers in a cluster, researchers can use many powerful computers at once, without locking up their own one.

"},{"location":"Background/Cluster_Computing/#how-do-i-use-it","title":"How do I use it?","text":"

Most people use something like the following workflow:

Most people connect using a program called a Secure Shell Client (\"ssh client\" for short), but some programs, like Matlab and Comsol, run on your own computer and can be set up to send work to the cluster automatically. That can be a little tricky to get working, though.

The ssh client gives you a command prompt when you can enter text commands, but you can also tell it to pass graphical windows through the network connection, using a system called X-Forwarding. This can be useful for visualising your data without transferring all the files back, but the network connection makes it a bit slower to use than running it on your own computer. You'll need an X server on your own computer to use this: check our page on X-Forwarding for details.

Please be aware that login nodes are shared resources, so users should not be running memory intensive jobs nor jobs with long runtimes in the login node. Doing so may negatively impact the performance of the login node for other users.

"},{"location":"Clusters/Acknowledging_RC_Systems/","title":"Acknowledging the Use of RC Systems","text":"

To keep running our services, we depend on being able to demonstrate that they are used in published research.

When preparing papers describing work that has used any of our clusters or services, please use the terms below, especially the \"service@UCL\" label, so that we can easily search for them.

"},{"location":"Clusters/Acknowledging_RC_Systems/#kathleen","title":"Kathleen","text":"

\"The authors acknowledge the use of the UCL Kathleen High Performance Computing Facility (Kathleen@UCL), and associated support services, in the completion of this work.\"

"},{"location":"Clusters/Acknowledging_RC_Systems/#myriad","title":"Myriad","text":"

\"The authors acknowledge the use of the UCL Myriad High Performance Computing Facility (Myriad@UCL), and associated support services, in the completion of this work.\"

"},{"location":"Clusters/Acknowledging_RC_Systems/#aristotle","title":"Aristotle","text":"

\"The authors acknowledge the use of the UCL Aristotle Computing Facility (Aristotle@UCL), and associated support services, in the completion of this work.\"

"},{"location":"Clusters/Acknowledging_RC_Systems/#mmm-hub-young","title":"MMM Hub Young","text":"

Please find the appropriate wording at Acknowledging the use of Young in publications.

"},{"location":"Clusters/Acknowledging_RC_Systems/#mmm-hub-thomas","title":"MMM Hub Thomas","text":"

Please find the appropriate wording at Acknowledging the use of Thomas in publications.

"},{"location":"Clusters/Kathleen/","title":"Kathleen","text":"

Kathleen is a compute cluster designed for extensively parallel, multi-node batch-processing jobs, having high-bandwidth connections between each individual node. It is named after Professor Dame Kathleen Lonsdale, a pioneering chemist and activist, and was installed in December 2019. It went into service at the end of January 2020.

"},{"location":"Clusters/Kathleen/#accounts","title":"Accounts","text":"

Kathleen accounts can be applied for via the Research Computing sign up process.

As Kathleen is intended for multi-node jobs, users who specify that they will need to use multi-node jobs (e.g. with MPI) will be given access to Kathleen.

"},{"location":"Clusters/Kathleen/#logging-in","title":"Logging in","text":"

Please use your UCL username and password to connect to Kathleen with an SSH client.

ssh uccaxxx@kathleen.rc.ucl.ac.uk\n

If using PuTTY, put kathleen.rc.ucl.ac.uk as the hostname and your seven-character username (with no @ after) as the username when logging in, eg. uccaxxx. When entering your password in PuTTY no characters or bulletpoints will show on screen - this is normal.

If you are outside the UCL firewall you will need to follow the instructions for Logging in from outside the UCL firewall.

"},{"location":"Clusters/Kathleen/#logging-in-to-a-specific-node","title":"Logging in to a specific node","text":"

You can access a specific Kathleen login node by using their dedicated addresses instead of the main kathleen.rc.ucl.ac.uk address, for example:

ssh\u00a0uccaxxx@login01.kathleen.rc.ucl.ac.uk\n

The main address will unpredictably direct you to either one of these (to balance load), so if you need multiple sessions on one, this lets you do that.

"},{"location":"Clusters/Kathleen/#copying-data-onto-kathleen","title":"Copying data onto Kathleen","text":"

You will need to use an SCP or SFTP client to copy data onto Kathleen. Please refer to the page on How do I transfer data onto the system?

You can connect directly in both directions between Grace and Kathleen.

If you find you cannot connect directly from one cluster to another, this is generally because of firewalls in between and so you need to use tunnelling with the scp command.

"},{"location":"Clusters/Kathleen/#quotas","title":"Quotas","text":"

On Kathleen you have a single 250GB quota by default which covers your home and Scratch.

This is a hard quota: once you reach it, you will no longer be able to write more data. Keep an eye on it, as this will cause jobs to fail if they cannot create their .o or .e files at the start, or their output files partway through.

You can check your quota on Kathleen by running:

lquota\n

which will give you output similar to this:

     Storage        Used        Quota   % Used   Path\n      lustre  146.19 MiB   250.00 GiB       0%   /home/uccaxxx\n

You can apply for quota increases using the form at Additional Resource Requests.

Here are some tips for managing your quota and finding where space is being used.

"},{"location":"Clusters/Kathleen/#job-sizes-and-durations","title":"Job sizes and durations","text":"

Please consider that Kathleen nodes have 40 physical cores - 2 nodes is 80 cores. Jobs do not share nodes, so although asking for 41 cores is possible, it means you are wasting the other 39 cores on your second node!

For interactive and batch jobs:

Cores Max. Duration 41-240 48h 241-480 24h 481-5760 12h

These are numbers of physical cores.

If you have a workload that requires longer jobs than this, you may be able to apply to our governance group for access to a longer queue. Applications will be expected to demonstrate that their work cannot be run using techniques like checkpointing that would allow their workload to be broken up into smaller parts. Please see the section on Additional Resource Requests for more details.

"},{"location":"Clusters/Kathleen/#node-types","title":"Node types","text":"

Kathleen's compute capability comprises 192 diskless compute nodes each with two 20-core Intel Xeon Gold 6248 2.5GHz processors, 192 gigabytes of 2933MHz DDR4 RAM, and an Intel OmniPath network.

Two nodes identical to these, but with two 1 terabyte hard-disk drives added, serve as the login nodes.

"},{"location":"Clusters/Kathleen/#hyperthreading","title":"Hyperthreading","text":"

Kathleen has hyperthreading enabled and you can choose on a per-job basis whether you want to use it.

Hyperthreading lets you use two virtual cores instead of one physical core - some programs can take advantage of this.

If you do not ask for hyperthreading, your job only uses one thread per core as normal.

The -l threads= request is not a true/false setting, instead you are telling the scheduler you want one slot to block one virtual cpu instead of the normal situation where it blocks two. If you have a script with a threads request and want to override it on the command line or set it back to normal, the usual case is -l threads=2. (Setting threads to 0 does not disable hyperthreading!)

# request hyperthreading in this job\n#$ -l threads=1\n\n# request the number of virtual cores\n#$ -pe mpi 160\n\n# request 2G RAM per virtual core\n#$ -l mem=2G\n\n# set number of OpenMP threads being used per MPI process \nexport OMP_NUM_THREADS=2\n

This job would be using 80 physical cores, using 80 MPI processes each of which would create two threads (on Hyperthreads).

Note that memory requests are now per virtual core with hyperthreading enabled. If you asked for #$ -l mem=4Gon a node with 80 virtual cores and 192G RAM then you are requiring 320G RAM in total which will not fit on that node and so you would be given a sparse process layout across more nodes to meet this requirement.

# request hyperthreading in this job\n#$ -l threads=1\n\n# request the number of virtual cores\n#$ -pe mpi 160\n\n# request 2G RAM per virtual core\n#$ -l mem=2G\n\n# set number of OpenMP threads being used per MPI process\n# (a whole node's worth)\nexport OMP_NUM_THREADS=80\n

This job would still be using 80 physical cores, but would use one MPI process per node which would create 80 threads on the node (on Hyperthreads).

"},{"location":"Clusters/Kathleen/#diskless-nodes","title":"Diskless nodes","text":"

Kathleen nodes are diskless (have no local hard drives) - there is no $TMPDIR available on Kathleen, so you should not request -l tmpfs=10G in your jobscripts or your job will be rejected at submit time.

If you need temporary space, you should use somewhere in your Scratch.

"},{"location":"Clusters/Kathleen/#loading-and-unloading-modules","title":"Loading and unloading modules","text":"

Kathleen has a newer version of modulecmd which tries to manage module dependencies automatically by loading or unloading prerequisites for you whenever possible.

If you get an error like this:

[uccaxxx@login01.kathleen ~]$ module unload compilers mpi\nUnloading compilers/intel/2018/update3\n  ERROR: compilers/intel/2018/update3 cannot be unloaded due to a prereq.\n    HINT: Might try \"module unload default-modules/2018\" first.\n\nUnloading mpi/intel/2018/update3/intel\n  ERROR: mpi/intel/2018/update3/intel cannot be unloaded due to a prereq.\n    HINT: Might try \"module unload default-modules/2018\" first.\n

You can use the -f option to force the module change. It will carry it out and warn you about modules it thinks are dependent.

[uccaxxx@login01.kathleen ~]$ module unload -f compilers mpi\nUnloading compilers/intel/2018/update3\n  WARNING: Dependent default-modules/2018 is loaded\n\nUnloading mpi/intel/2018/update3/intel\n  WARNING: Dependent default-modules/2018 is loaded\n

Otherwise you will need to unload default-modules/2018 to swap compiler and MPI module, but that will leave you without gerun in your path. You can then do either of these things:

# load everything that was in default-modules except the compiler and mpi\nmodule unload default-modules/2018\nmodule load rcps-core/1.0.0\nmodule load whatever\n

or

# just reload gerun\nmodule unload default-modules/2018\nmodule load gerun\nmodule load whatever\n
"},{"location":"Clusters/Michael/","title":"Michael","text":"

Michael is an extension to the UCL-hosted Hub for Materials and Molecular Modelling, an EPSRC-funded Tier 2 system providing large scale computation to UK researchers; and delivers computational capability for the Faraday Institution, a national institute for electrochemical energy storage science and technology.

"},{"location":"Clusters/Michael/#applying-for-an-account","title":"Applying for an account","text":"

Michael accounts belong to you as an individual and are applied for via David Scanlon who is the point of contact for Michael.

You will need to supply an SSH public key, which is the only method used to log in.

"},{"location":"Clusters/Michael/#creating-an-ssh-key-pair","title":"Creating an ssh key pair","text":"

An ssh key consists of a public and a private part, typically named id_rsa and id_rsa.pub by default. The public part is what we need. You must not share your private key with anyone else. You can copy it onto multiple machines belonging to you so you can log in from all of them (or you can have a separate pair for each machine).

"},{"location":"Clusters/Michael/#creating-an-ssh-key-in-linuxunixmac-os-x","title":"Creating an ssh key in Linux/Unix/Mac OS X","text":"
ssh-keygen\u00a0-t\u00a0rsa\n

The defaults should give you a reasonable key. If you prefer to use ed25519 instead, and/or longer keys, you can. You can also tell it to create one with a different name, so it doesn't overwrite any existing key.

You will be asked to add a passphrase for your key. A blank passphrase is not recommended; if you use one please make sure that no one else ever has access to your local computer account. How often you are asked for a passphrase depends on how long your local ssh agent keeps it.

You may need to run ssh-add to add the key to your agent so you can use it. If you aren't sure what keys your agent can see, running ssh-add -L will show all the public parts of the keys it is aware of.

"},{"location":"Clusters/Michael/#creating-an-ssh-key-in-windows","title":"Creating an ssh key in Windows","text":"

Have a look at Key-Based SSH Logins With PuTTY which has step-by-step instructions. You can choose whether to use Pageant or not to manage your key. You can again pick RSA, ED25519, ECDSA etc but do not pick SSH-1 as that is a very old and insecure key type. As above, DSA is no longer accepted. The key must be at least 2048-bit.

If you are using Windows 10, then you probably have OpenSSH installed and could instead run ssh-keygen in a terminal per the Linux instructions and use the ssh command to log in instead of PuTTY.

"},{"location":"Clusters/Michael/#information-for-points-of-contact","title":"Information for Points of Contact","text":"

Points of Contact have some tools they can use to manage users and allocations, documented at MMM Points of Contact.

"},{"location":"Clusters/Michael/#logging-in","title":"Logging in","text":"

You will be assigned a personal username and your SSH key pair will be used to log in. External users will have a username in the form mmmxxxx (where xxxx is a number) and UCL users will use their central username.

You connect with ssh directly to:

michael.rc.ucl.ac.uk\n
"},{"location":"Clusters/Michael/#ssh-timeouts","title":"SSH timeouts","text":"

Idle ssh sessions will be disconnected after 7 days.

"},{"location":"Clusters/Michael/#using-the-system","title":"Using the system","text":"

Michael is a batch system. The login nodes allow you to manage your files, compile code and submit jobs. Very short (\\<15mins) and non-resource-intensive software tests can be run on the login nodes, but anything more should be submitted as a job.

"},{"location":"Clusters/Michael/#full-user-guide","title":"Full user guide","text":"

Michael has the same user environment as RC Support's other clusters, so the User guide is relevant and is a good starting point for further information about how the environment works. Any variations that Michael has should be listed on this page.

"},{"location":"Clusters/Michael/#submitting-a-job","title":"Submitting a job","text":"

Create a job script for non-interactive use and submit your jobscript using qsub. Jobscripts must begin #!/bin/bash -l in order to run as a login shell and get your login environment and modules.

A job on Michael must also specify what type of job it is (Gold, Free, Test) and the project it is being submitted for. (See Budgets and allocations below).

"},{"location":"Clusters/Michael/#memory-requests","title":"Memory requests","text":"

Note: the memory you request is always per core, not the total amount. If you ask for 128G RAM and 24 cores, that will run on 24 nodes using only one core per node. This allows you to have sparse process placement when you do actually need that much RAM per process.

"},{"location":"Clusters/Michael/#monitoring-a-job","title":"Monitoring a job","text":"

In addition to qstat, nodesforjob $JOB_ID can be useful to see what proportion of cpu/memory/swap is being used on the nodes a certain job is running on.

qexplain $JOB_ID will show you the full error for a job that is in Eqw status.

"},{"location":"Clusters/Michael/#useful-utilities","title":"Useful utilities","text":"

As well as nodesforjob, there are the following utilities which can help you find information about your jobs after they have run.

These utilities live in GitHub at https://github.com/UCL-RITS/go-clustertools and https://github.com/UCL-RITS/rcps-cluster-scripts

"},{"location":"Clusters/Michael/#software","title":"Software","text":"

Michael mounts the RC Systems software stack.

Have a look at Software Guides for specific information on running some applications, including example scripts. The list there is not exhaustive.

Access to software is managed through the use of modules.

Access to licensed software may vary based on your host institution and project.

"},{"location":"Clusters/Michael/#requesting-software-installs","title":"Requesting software installs","text":"

To request software installs, email us at the support address below or open an issue on our GitHub. You can see what software has already been requested in the Github issues and can add a comment if you're also interested in something already requested.

"},{"location":"Clusters/Michael/#installing-your-own-software","title":"Installing your own software","text":"

You may install software in your own space. Please look at Compiling for tips.

"},{"location":"Clusters/Michael/#maintaining-a-piece-of-software-for-a-group","title":"Maintaining a piece of software for a group","text":"

It is possible for people to be given central areas to install software that they wish to make available to everyone or to a select group - generally because they are the developers or if they wish to use multiple versions or developer versions. The people given install access would then be responsible for managing and maintaining these installs.

"},{"location":"Clusters/Michael/#licensed-software","title":"Licensed software","text":"

Reserved application groups exist for software that requires them. The group name will begin with leg or lg. After we add you to one of these groups, the central group change will happen overnight. You can check your groups with the groups command.

Please let us know your username when you ask to be added to a group.

"},{"location":"Clusters/Michael/#suggested-job-sizes-on-original-michael","title":"Suggested job sizes on original Michael","text":"

The target job sizes for original Michael K-type nodes are 48-120 cores (2-5 nodes). Jobs larger than this may have a longer queue time and are better suited to ARCHER, and single node jobs may be more suited to your local facilities.

"},{"location":"Clusters/Michael/#maximum-job-resources-on-original-michael","title":"Maximum job resources on original Michael","text":"Cores Max wallclock 864 48hrs

On Michael, interactive sessions using qrsh have the same wallclock limit as other jobs.

The K-type nodes in Michael are 24 cores, 128GB RAM. The default maximum jobsize is 864 cores, to remain within the 36-node 1:1 nonblocking interconnect zones.

Jobs on Michael do not share nodes. This means that if you request less than 24 cores, your job is still taking up an entire node and no other jobs can run on it, but some of the cores are idle. Whenever possible, request a number of cores that is a multiple of 24 for full usage of your nodes.

There is a superqueue for use in exceptional circumstances that will allow access to a larger number of cores outside the nonblocking interconnect zones, going across the 3:1 interconnect between blocks. A third of each CU is accessible this way, roughly approximating a 1:1 connection. Access to the superqueue for larger jobs must be applied for: contact the support address below for details.

Some normal multi-node jobs will use the superqueue - this is to make it easier for larger jobs to be scheduled, as otherwise they can have very long waits if every CU is half full.

"},{"location":"Clusters/Michael/#2020-michael-expansion","title":"2020 Michael expansion","text":"

At the end of March 2020, Michael was expanded to include a new set of nodes. The old Michael nodes are the K-type nodes, while the new ones are the A-type nodes. The node name will look like node-a14a-001 or node-k10a-001.

The Michael expansion consists of 208 compute nodes each with two 20-core Intel Xeon Gold 6248 2.5GHz processors, 192 gigabytes of 2933MHz DDR4 RAM, 1TB disk, and an Intel OmniPath network. Expansion nodes have two Hyperthreads available.

These are arranged in two 32-node CUs (a and b) and four 36-node CUs (c to f). Jobs are restricted to running either within a CU (all nodes connected to the same switch) or across CUs using only the bottom third of nodes attached to each switch. This approximates 1:1 blocking on a cluster that does not have it.

"},{"location":"Clusters/Michael/#maximum-job-resources-on-michael-expansion","title":"Maximum job resources on Michael expansion","text":"

Please consider that Michael's A-type nodes have 40 physical cores - 2 nodes is 80 cores. Jobs do not share nodes, so although asking for 41 cores is possible, it means you are wasting the other 39 cores on your second node!

Cores Max. Duration 2800 48h

These are numbers of physical cores: multiply by two for virtual cores with hyperthreads.

"},{"location":"Clusters/Michael/#hyperthreading","title":"Hyperthreading","text":"

The A-type nodes have hyperthreading enabled and you can choose on a per-job basis whether you want to use it.

Hyperthreading lets you use two virtual cores instead of one physical core - some programs can take advantage of this.

If you do not ask for hyperthreading, your job only uses one thread per core as normal.

The -l threads= request is not a true/false setting, instead you are telling the scheduler you want one slot to block one virtual cpu instead of the normal situation where it blocks two. If you have a script with a threads request and want to override it on the command line or set it back to normal, the usual case is -l threads=2. (Setting threads to 0 does not disable hyperthreading!)

# request hyperthreading in this job\n#$ -l threads=1\n\n# request the number of virtual cores\n#$ -pe mpi 160\n\n# request 2G RAM per virtual core\n#$ -l mem=2G\n\n# set number of OpenMP threads being used per MPI process\nexport OMP_NUM_THREADS=2\n

This job would be using 80 physical cores, using 80 MPI processes each of which would create two threads (on Hyperthreads).

Note that memory requests are now per virtual core with hyperthreading enabled. If you asked for #$ -l mem=4Gon a node with 80 virtual cores and 192G RAM then you are requiring 320G RAM in total which will not fit on that node and so you would be given a sparse process layout across more nodes to meet this requirement.

# request hyperthreading in this job\n#$ -l threads=1\n\n# request the number of virtual cores\n#$ -pe mpi 160\n\n# request 2G RAM per virtual core\n#$ -l mem=2G\n\n# set number of OpenMP threads being used per MPI process\n# (a whole node's worth)\nexport OMP_NUM_THREADS=80\n

This job would still be using 80 physical cores, but would use one MPI process per node which would create 80 threads on the node (on Hyperthreads).

"},{"location":"Clusters/Michael/#choosing-node-types","title":"Choosing node types","text":"

Given the difference in core count on the original and expansion Michael nodes, we strongly suggest you always specify which type of node you intend your job to run on, to avoid unintentionally wasting cores if your total number does not cleanly fit on that node size.

The old nodes are K-type while the new nodes with hyperthreading are A-type. Jobs never run across a mix of node types - it will be all K nodes or all A nodes.

To specify node type in your jobscript, add either:

# run on original 24-core nodes\n#$ -ac allow=K\n

or

# run on expansion 40-core hyperthread-enabled nodes\n#$ -ac allow=A\n
"},{"location":"Clusters/Michael/#queue-names","title":"Queue names","text":"

On Michael, users do not submit directly to queues - the scheduler assigns your job to one based on the resources it requested. The queues have somewhat unorthodox names as they are only used internally, and do not directly map to particular job types.

"},{"location":"Clusters/Michael/#preventing-a-job-from-running-cross-cu","title":"Preventing a job from running cross-CU","text":"

If your job must run within a single CU, you can request the parallel environment as -pe wss instead of -pe mpi (wss standing for 'wants single switch'). This will increase your queue times. It is suggested you only do this for benchmarking or if performance is being greatly affected by running in the superqueue.

back to top

"},{"location":"Clusters/Michael/#disk-quotas","title":"Disk quotas","text":"

You have one per-user quota, with a default amount of 250GB - this is the total across home and Scratch.

If you go over quota, you will no longer be able to create new files and your jobs will fail as they cannot write.

Quota increases may be granted without further approval, depending on size and how full the filesystem is. Otherwise they may need to go to the Thomas User Group for approval.

back to top

"},{"location":"Clusters/Michael/#budgets-and-allocations","title":"Budgets and allocations","text":"

We have enabled Gold for allocation management. Jobs that are run under a project budget have higher priority than free non-budgeted jobs. All jobs need to specify what project they belong to, whether they are paid or free.

To see the name of your project(s) and how much allocation that budget has, run the command budgets.

$ budgets  \nProject\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Machines\u00a0Balance\n--------\u00a0\u00a0\u00a0\u00a0\u00a0--------\u00a0--------\nFaraday_Test\u00a0ANY\u00a0\u00a0\u00a0\u00a0\u00a0\u00a022781.89\n
"},{"location":"Clusters/Michael/#submitting-a-job-under-a-project","title":"Submitting a job under a project","text":"

To submit a paid job that will take Gold from a particular project budget, add this to your jobscript:

#$\u00a0-P\u00a0Gold\n#$\u00a0-A\u00a0MyProject\n

To submit a free job that will not use up any Gold, use this instead:

#$\u00a0-P\u00a0Free\n#$\u00a0-A\u00a0MyProject\n

You can also submit testing jobs that will not use up any Gold, and will have higher priority than normal free jobs, but are limited to 2 nodes (48 cores) and 1 hour of walltime:

#$\u00a0-P\u00a0Test\n#$\u00a0-A\u00a0MyProject\n
"},{"location":"Clusters/Michael/#troubleshooting-unable-to-verify-membership-in-policyjsv-project","title":"Troubleshooting: Unable to verify membership in policyjsv project","text":"
Unable\u00a0to\u00a0run\u00a0job:\u00a0Rejected\u00a0by\u00a0policyjsv\nUnable\u00a0to\u00a0verify\u00a0membership\u00a0of\u00a0`<username>`\u00a0in\u00a0the\u00a0policyjsv\u00a0project\n

You asked for a Free job but didn't specify #$ -A MyProject in your jobscript.

"},{"location":"Clusters/Michael/#troubleshooting-unable-to-verify-membership-in-project-uninitialized-value","title":"Troubleshooting: Unable to verify membership in project / Uninitialized value","text":"
Unable to run job: Rejected by policyjsv  \nReason:Unable to verify sufficient material worth to submit this job:  \nUnable to verify membership of mmmxxxx in the UCL_Example project\n

This error from qsub can mean that you aren't in the project you are trying to submit to, but also happens when the Gold daemon is not running. 

Use of uninitialized value in print at /opt/gold/bin/mybalance line 60, <GBALANCE> line 1.\nFailed sending message: (Unable to connect to socket (Connection refused)).\n

If you also get this error from the budgets command, then the Gold daemon is definitely not running and you should contact rc-support.

"},{"location":"Clusters/Michael/#gold-charging","title":"Gold charging","text":"

When you submit a job, it will reserve the total number of core hours that the job script is asking for. When the job ends, the Gold will move from 'reserved' into charged. If the job doesn't run for the full time it asked for, the unused reserved portion will be refunded after the job ends. You cannot submit a job that you do not have the budget to run.

"},{"location":"Clusters/Michael/#gold-costs-of-a-type-nodes","title":"Gold costs of A-type nodes","text":"

The A-type nodes have twice the peak theoretical performance of the K-type nodes. A 24-core job lasting an hour costs 24 Gold on the K-type nodes. A 40-physical-core job lasting one hour costs 80 Gold on the A-type nodes. An 80-virtual-core job on the A-type nodes also costs 80 Gold.

"},{"location":"Clusters/Michael/#troubleshooting-unable-to-verify-sufficient-material-worth","title":"Troubleshooting: Unable to verify sufficient material worth","text":"
Unable\u00a0to\u00a0run\u00a0job:\u00a0Rejected\u00a0by\u00a0policyjsv\nReason:Unable\u00a0to\u00a0verify\u00a0sufficient\u00a0material\u00a0worth\u00a0to\u00a0submit\u00a0this\u00a0job:\nInsufficient\u00a0balance\u00a0to\u00a0reserve\u00a0job\n

This means you don't have enough Gold to cover the cores*wallclock time cost of the job you are trying to submit. You need to wait for queued jobs to finish and return unused Gold to your project, or submit a smaller/shorter job. Note that array jobs have to cover the whole cost of all the tasks at submit time.

"},{"location":"Clusters/Michael/#job-deletion","title":"Job deletion","text":"

If you qdel a submitted Gold job, the reserved Gold will be made available again. This is done by a cron job that runs every 15 minutes, so you may not see it back instantly.

"},{"location":"Clusters/Michael/#reporting-gold-usage","title":"Reporting Gold usage","text":"

There are a few commands that everyone can run that report Gold usage for their entire project, broken down by user. See Reporting from Gold.

"},{"location":"Clusters/Michael/#support","title":"Support","text":"

Email rc-support@ucl.ac.uk with any support queries. It will be helpful to include Michael in the subject along with some descriptive text about the type of problem, and you should mention your username in the body.

"},{"location":"Clusters/Myriad/","title":"Myriad","text":"

Myriad is designed for high I/O, high throughput jobs that will run within a single node rather than multi-node parallel jobs.

"},{"location":"Clusters/Myriad/#accounts","title":"Accounts","text":"

Myriad accounts can be applied for via the Research Computing sign up process.

As Myriad is our most general-purpose system, everyone who signs up for a Research Computing account is given access to Myriad.

"},{"location":"Clusters/Myriad/#logging-in","title":"Logging in","text":"

You will use your UCL username and password to ssh in to Myriad.

ssh\u00a0uccaxxx@myriad.rc.ucl.ac.uk\n

If using PuTTY, put myriad.rc.ucl.ac.uk as the hostname and your seven-character username (with no @ after) as the username when logging in, eg. uccaxxx. When entering your password in PuTTY no characters or bulletpoints will show on screen - this is normal.

If you are outside the UCL firewall you will need to follow the instructions for Logging in from outside the UCL firewall.

"},{"location":"Clusters/Myriad/#logging-in-to-a-specific-node","title":"Logging in to a specific node","text":"

You can access a specific Myriad login node with: 

ssh\u00a0uccaxxx@login12.myriad.rc.ucl.ac.uk\nssh\u00a0uccaxxx@login13.myriad.rc.ucl.ac.uk\n

The main address will redirect you on to either one of them.

"},{"location":"Clusters/Myriad/#copying-data-onto-myriad","title":"Copying data onto Myriad","text":"

You will need to use an SCP or SFTP client to copy data onto Myriad. Please refer to the page on How do I transfer data onto the system?

"},{"location":"Clusters/Myriad/#quotas","title":"Quotas","text":"

The default quotas on Myriad are 150GB for home and 1TB for Scratch.

These are hard quotas: once you reach them, you will no longer be able to write more data. Keep an eye on them, as this will cause jobs to fail if they cannot create their .o or .e files at the start, or their output files partway through.

You can check both quotas on Myriad by running: 

lquota\n

which will give you output similar to this:

     Storage        Used        Quota   % Used   Path\n        home  721.68 MiB   150.00 GiB       0%   /home/uccaxxx\n     scratch   52.09 MiB     1.00 TiB       0%   /scratch/scratch/uccaxxx\n

You can apply for quota increases using the form at Additional Resource Requests.

Here are some tips for managing your quota and finding where space is being used.

"},{"location":"Clusters/Myriad/#job-sizes","title":"Job sizes","text":"Cores Max wallclock 1 72hrs 2 to 36 48hrs

Interactive jobs run with qrsh have the same maximum wallclock time as other jobs.

"},{"location":"Clusters/Myriad/#node-types","title":"Node types","text":"

Myriad contains three main node types: standard compute nodes, high memory nodes and GPU nodes. As new nodes as added over time with slightly newer processor variants, new letters are added.

Type Cores per node RAM per node tmpfs Nodes H,D 36 192GB 1500G 342 I,B 36 1.5TB 1500G 17 J 36 + 2 P100 GPUs 192GB 1500G 2 E,F 36 + 2 V100 GPUs 192GB 1500G 19 L 36 + 4 A100 GPUs 192GB 1500G 6

You can tell the type of a node by its name: type H nodes are named node-h00a-001 etc.

Here are the processors each node type has:

(If you ever need to check this, you can include cat /proc/cpuinfo in your jobscript so you get it in your job's .o file for the exact node your job ran on. You will get an entry for every core).

"},{"location":"Clusters/Myriad/#gpus","title":"GPUs","text":"

Myriad has four types of GPU nodes: E, F, J and L.

You can include nvidia-smi in your jobscript to get information about the GPU your job ran on.

"},{"location":"Clusters/Myriad/#compute-capability","title":"Compute Capability","text":"

Compute Capability is how NVIDIA categorises its generations of GPU architectures. When code is compiled, it targets one or multiple of these and so it may only be able to run on GPUs of a specific Compute Capability.

If you get an error like this:

CUDA runtime implicit initialization on GPU:0 failed. Status: device kernel image is invalid\n

then the software you are running does not support the Compute Capability of the GPU you tried to run it on, and you probably need a newer version.

"},{"location":"Clusters/Myriad/#requesting-multiple-and-specific-types-of-gpu","title":"Requesting multiple and specific types of GPU","text":"

You can request a number of GPUs by adding them as a resource request to your jobscript: 

# For 1 GPU\n#$\u00a0-l\u00a0gpu=1\n\n# For 2 GPUs\n#$ -l gpu=2\n\n# For 4 GPUs\n#$ -l gpu=4\n

If you ask for one or two GPUs your job can run on any type of GPU since it can fit on any of the nodetypes. If you ask for four, it can only be a node that has four. If you need to specify one node type over the others because you need a particular Compute Capability, add a request for that type of node to your jobscript:

# request a V100 node only\n#$ -ac allow=EF\n\n# request an A100 node only\n#$ -ac allow=L\n

The GPU nodes page has some sample code for running GPU jobs if you need a test example.

"},{"location":"Clusters/Myriad/#tensorflow","title":"Tensorflow","text":"

Tensorflow is installed: type module avail tensorflow to see the available versions.

Modules to load for the non-MKL GPU version: 

module\u00a0load\u00a0python3/3.7\nmodule\u00a0load\u00a0cuda/10.0.130/gnu-4.9.2\nmodule\u00a0load\u00a0cudnn/7.4.2.24/cuda-10.0\nmodule\u00a0load\u00a0tensorflow/2.0.0/gpu-py37\n

Modules to load the most recent version we have installed with GPU support (2.11.0):

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load python/3.9.6-gnu-10.2.0\nmodule load cuda/11.2.0/gnu-10.2.0\nmodule load cudnn/8.1.0.77/cuda-11.2\nmodule load tensorflow/2.11.0/gpu\n
"},{"location":"Clusters/Myriad/#pytorch","title":"PyTorch","text":"

PyTorch is installed: type module avail pytorch to see the versions available.

Modules to load the most recent release we have installed (May 2022) are:

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load python3/3.9-gnu-10.2.0\nmodule load cuda/11.3.1/gnu-10.2.0\nmodule load cudnn/8.2.1.32/cuda-11.3\nmodule load pytorch/1.11.0/gpu\n

If you want the CPU only version then use:

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load python3/3.9-gnu-10.2.0\nmodule load pytorch/1.11.0/cpu\n
"},{"location":"Clusters/Thomas/","title":"MMM Thomas","text":"

Thomas is the UK National Tier 2 High Performance Computing Hub in Materials and Molecular Modelling.

Thomas final retirement

The Thomas cluster has been retired. Thomas' filesystem was getting too old and beginning to fail, so we had to retire Thomas. Notable dates are:

"},{"location":"Clusters/Thomas/#original-thomas-retirement-dates","title":"Original Thomas retirement dates:","text":"

The Thomas cluster is being retired. Notable dates are:

Thomas has now been retired as an MMM Hub machine. A portion of Thomas is running for UCL users only until the end of 2021.

7 June 2021: Thomas is now inside the UCL firewall, use VPN, or SSH via ssh-gateway.ucl.ac.uk to access.

"},{"location":"Clusters/Thomas/#thomas-technical-specs","title":"Thomas technical specs","text":"

Thomas originally consisted of 720 Lenovo Intel x86-64 nodes, giving 17.2k cores in total, with Intel OmniPath interconnect (1:1 nonblocking in 36 node blocks, 3:1 between blocks and across the system).

Each node has the following specs:

"},{"location":"Clusters/Thomas/#applying-for-an-account","title":"Applying for an account","text":"

UCL users can contact David Scanlon or Scott Woodley for access to the post-retirement portion of Thomas.

Thomas accounts belong to you as an individual and are applied for through your own institution's Point of Contact. You will need to supply an SSH public key, which is the only method used to log in.

"},{"location":"Clusters/Thomas/#creating-an-ssh-key-pair","title":"Creating an ssh key pair","text":"

An ssh key consists of a public and a private part, typically named id_rsa and id_rsa.pub by default. The public part is what we need. You must not share your private key with anyone else. You can copy it onto multiple machines belonging to you so you can log in from all of them (or you can have a separate pair for each machine).

"},{"location":"Clusters/Thomas/#creating-an-ssh-key-in-linuxunixmac-os-x","title":"Creating an ssh key in Linux/Unix/Mac OS X","text":"
ssh-keygen\u00a0-t\u00a0rsa\n

The defaults should give you a reasonable key. If you prefer to use ed25519 instead, and/or longer keys, you can. You can also tell it to create one with a different name, so it doesn't overwrite any existing key.

You will be asked to add a passphrase for your key. A blank passphrase is not recommended; if you use one please make sure that no one else ever has access to your local computer account. How often you are asked for a passphrase depends on how long your local ssh agent keeps it.

You may need to run ssh-add to add the key to your agent so you can use it. If you aren't sure what keys your agent can see, running ssh-add -L will show all the public parts of the keys it is aware of.

"},{"location":"Clusters/Thomas/#creating-an-ssh-key-in-windows","title":"Creating an ssh key in Windows","text":"

Have a look at Key-Based SSH Logins With PuTTY which has step-by-step instructions. You can choose whether to use Pageant or not to manage your key. You can again pick RSA, ED25519, ECDSA etc but do not pick SSH-1 as that is a very old and insecure key type. As above, DSA is no longer accepted. The key must be at least 2048-bit.

If you are using Windows 10, then you probably have OpenSSH installed and could instead run ssh-keygen in a terminal per the Linux instructions and use the ssh command to log in instead of PuTTY.

"},{"location":"Clusters/Thomas/#information-for-points-of-contact","title":"Information for Points of Contact","text":"

Points of Contact have some tools they can use to manage users and allocations, documented at MMM Points of Contact.

"},{"location":"Clusters/Thomas/#logging-in","title":"Logging in","text":"

You will be assigned a personal username and your SSH key pair will be used to log in. External users will have a username in the form mmmxxxx (where xxxx is a number) and UCL users will use their central username.

You ssh directly to:

thomas.rc.ucl.ac.uk\n

From inside the UCL VPN you can ssh directly into the above but from outside you can do the following:

ssh -o ProxyJump=<UCL_user_ID>@ssh-gateway.ucl.ac.uk <thomas_user_ID>@thomas.rc.ucl.ac.uk\n
"},{"location":"Clusters/Thomas/#ssh-timeouts","title":"SSH timeouts","text":"

Idle ssh sessions will be disconnected after 7 days.

"},{"location":"Clusters/Thomas/#using-the-system","title":"Using the system","text":"

Thomas is a batch system. The login nodes allow you to manage your files, compile code and submit jobs. Very short (\\<15mins) and non-resource-intensive software tests can be run on the login nodes, but anything more should be submitted as a job.

"},{"location":"Clusters/Thomas/#full-user-guide","title":"Full user guide","text":"

Thomas has the same user environment as RC Support's other clusters, so the User guide is relevant and is a good starting point for further information about how the environment works. Any variations that Thomas has should be listed on this page.

"},{"location":"Clusters/Thomas/#submitting-a-job","title":"Submitting a job","text":"

Create a jobscript for non-interactive use and submit your jobscript using qsub. Jobscripts must begin #!/bin/bash -l in order to run as a login shell and get your login environment and modules.

Jobs no longer need to specify what type of job it is (Gold, Free, Test) or the project it is being submitted for. (See Budgets and allocations below.)

"},{"location":"Clusters/Thomas/#memory-requests","title":"Memory requests","text":"

Note: the memory you request is always per core, not the total amount. If you ask for 128GB RAM and 24 cores, that will run on 24 nodes using only one core per node. This allows you to have sparse process placement when you do actually need that much RAM per process.

"},{"location":"Clusters/Thomas/#monitoring-a-job","title":"Monitoring a job","text":"

In addition to qstat, nodesforjob $JOB_ID can be useful to see what proportion of cpu/memory/swap is being used on the nodes a certain job is running on.

qexplain $JOB_ID will show you the full error for a job that is in Eqw status.

"},{"location":"Clusters/Thomas/#useful-utilities","title":"Useful utilities","text":"

As well as nodesforjob, there are the following utilities which can help you find information about your jobs after they have run.

These utilities live in GitHub at https://github.com/UCL-RITS/go-clustertools and https://github.com/UCL-RITS/rcps-cluster-scripts

"},{"location":"Clusters/Thomas/#software","title":"Software","text":"

Thomas mounts the RC Systems software stack.

Have a look at Software Guides for specific information on running some applications, including example scripts. The list there is not exhaustive.

Access to software is managed through the use of modules.

Access to licensed software may vary based on your host institution and project.

"},{"location":"Clusters/Thomas/#requesting-software-installs","title":"Requesting software installs","text":"

To request software installs, email us at the support address below or open an issue on our GitHub. You can see what software has already been requested in the Github issues and can add a comment if you're also interested in something already requested.

"},{"location":"Clusters/Thomas/#installing-your-own-software","title":"Installing your own software","text":"

You may install software in your own space. Please look at Compiling for tips.

"},{"location":"Clusters/Thomas/#maintaining-a-piece-of-software-for-a-group","title":"Maintaining a piece of software for a group","text":"

It is possible for people to be given central areas to install software that they wish to make available to everyone or to a select group - generally because they are the developers or if they wish to use multiple versions or developer versions. The people given install access would then be responsible for managing and maintaining these installs.

"},{"location":"Clusters/Thomas/#licensed-software","title":"Licensed software","text":"

Reserved application groups exist for software that requires them. The group name will begin with leg or lg. After we add you to one of these groups, the central group change will happen overnight. You can check your groups with the groups command.

Please let us know your username when you ask to be added to a group.

"},{"location":"Clusters/Thomas/#suggested-job-sizes","title":"Suggested job sizes","text":"

The target job sizes for Thomas are 48-120 cores (2-5 nodes). Jobs larger than this may have a longer queue time and are better suited to ARCHER, and single node jobs may be more suited to your local facilities.

"},{"location":"Clusters/Thomas/#maximum-job-resources","title":"Maximum job resources","text":"Cores Max wallclock 864 48hrs

On Thomas, interactive sessions using qrsh have the same wallclock limit as other jobs.

Nodes in Thomas are 24 cores, 128G RAM. The default maximum jobsize is 864 cores, to remain within the 36-node 1:1 nonblocking interconnect zones.

Jobs on Thomas do not share nodes. This means that if you request less than 24 cores, your job is still taking up an entire node and no other jobs can run on it, but some of the cores are idle. Whenever possible, request a number of cores that is a multiple of 24 for full usage of your nodes.

There is a superqueue for use in exceptional circumstances that will allow access to a larger number of cores outside the nonblocking interconnect zones, going across the 3:1 interconnect between blocks. A third of each CU is accessible this way, roughly approximating a 1:1 connection. Access to the superqueue for larger jobs must be applied for: contact the support address below for details.

Some normal multi-node jobs will use the superqueue - this is to make it easier for larger jobs to be scheduled, as otherwise they can have very long waits if every CU is half full.

"},{"location":"Clusters/Thomas/#queue-names","title":"Queue names","text":"

On Thomas, users do not submit directly to queues - the scheduler assigns your job to one based on the resources it requested. The queues have somewhat unorthodox names as they are only used internally, and do not directly map to particular job types.

"},{"location":"Clusters/Thomas/#preventing-a-job-from-running-cross-cu","title":"Preventing a job from running cross-CU","text":"

If your job must run within a single CU, you can request the parallel environment as -pe wss instead of -pe mpi (wss standing for 'wants single switch'). This will increase your queue times. It is suggested you only do this for benchmarking or if performance is being greatly affected by running in the superqueue.

back to top

"},{"location":"Clusters/Thomas/#node-types","title":"Node types","text":"

Thomas has one type of node.

Type Cores per node RAM per node tmpfs Nodes K 24 128G 100G 705

Here are the processors each node type has:

(If you ever need to check this, you can include cat /proc/cpuinfo in your jobscript so you get it in your job's .o file for the exact node your job ran on. You will get an entry for every core).

"},{"location":"Clusters/Thomas/#disk-quotas","title":"Disk quotas","text":"

Quotas were enabled on Thomas on 29 May 2019. You have one per-user quota, with a default amount of 250GB - this is the total across home and Scratch.

If you go over quota, you will no longer be able to create new files and your jobs will fail as they cannot write.

Quota increases may be granted without further approval, depending on size and how full the filesystem is. Otherwise they may need to go to the Thomas User Group for approval.

back to top

"},{"location":"Clusters/Thomas/#budgets-and-allocations","title":"Budgets and allocations","text":"

There is no more Gold on Thomas. Jobs should now be submitted without specifying any -P or -A entries in your jobscript.

Info

1 Gold unit is 1 hour of using 1 processor core.

"},{"location":"Clusters/Thomas/#troubleshooting-unable-to-verify-membership-in-project-uninitialized-value","title":"Troubleshooting: Unable to verify membership in project / Uninitialized value","text":"
Unable to run job: Rejected by policyjsv \nReason:Unable to verify sufficient material worth to submit this job: \nUnable to verify membership of mmmxxxx in the UCL_Example project\n

This error from qsub can mean that you aren't in the project you are trying to submit to, but also happens when the Gold daemon is not running.

Remove the -P and -A lines from your jobscript and submit jobs without any project specified.

Use of uninitialized value in print at /opt/gold/bin/mybalance line 60, <GBALANCE> line 1.\nFailed sending message: (Unable to connect to socket (Connection refused)).\n

The Gold database is uncontactable.

Remove the -P and -A lines from your jobscript and submit jobs without any project specified.

"},{"location":"Clusters/Thomas/#the-tier-2-safe","title":"The Tier 2 SAFE","text":"

SAFE is a service administration platform used by ARCHER and various of the Tier 2 sites. As a user, you can use it to do some management of your details and view your usage across all systems that send data to SAFE.

"},{"location":"Clusters/Thomas/#see-your-usage-data-in-safe","title":"See your usage data in SAFE","text":"

If you wish, you can claim your Thomas account as belonging to you in the Tier 2 SAFE. This lets you view some individual usage reports, and if you have other Tier 2 accounts that also use SAFE, you use the same login information.

If you do not claim your account, then SAFE only contains username/job usage information with no information about who owns that username. Points of Contact can use it to allocate Gold to your budgets.

"},{"location":"Clusters/Thomas/#update-your-ssh-key-via-safe","title":"Update your SSH key via SAFE","text":"

SAFE also gives you a second mechanism for changing your SSH key - changing the key you have associated with your SAFE account will automatically create a request for us to add that key to your Thomas account. (We still need to act on it, so it won't update immediately).

"},{"location":"Clusters/Thomas/#support","title":"Support","text":"

Email rc-support@ucl.ac.uk with any support queries. It will be helpful to include Thomas in the subject along with some descriptive text about the type of problem, and you should mention your username in the body.

"},{"location":"Clusters/Thomas/#notional-costs","title":"Notional costs","text":"

EPSRC contributed to the hardware, so there are two numbers to use for notional costs:

Non-grant-funded projects should use the second figure.

"},{"location":"Clusters/Thomas/#acknowledging-the-use-of-thomas-in-publications","title":"Acknowledging the use of Thomas in publications","text":"

All work arising from this facility should be properly acknowledged in presentations and papers with the following text:

\"We are grateful to the UK Materials and Molecular Modelling Hub for computational resources, which is partially funded by EPSRC (EP/P020194/1 and EP/T022213/1)\"

"},{"location":"Clusters/Thomas/#mcc","title":"MCC","text":"

When publishing work that benefited from resources allocated by the MCC: please include the following acknowledgment:

\"Via our membership of the UK's HEC Materials Chemistry Consortium, which is funded by EPSRC (EP/L000202), this work used the UK Materials and Molecular Modelling Hub for computational resources, MMM Hub, which is partially funded by EPSRC (EP/P020194 and EP/T022213)\"

"},{"location":"Clusters/Thomas/#ukcp","title":"UKCP","text":"

When publishing work that benefited from resources allocated by UKCP, please include:

\"We are grateful for computational support from the UK Materials and Molecular Modelling Hub, which is partially funded by EPSRC (EP/P020194 and EP/T022213), for which access was obtained via the UKCP consortium and funded by EPSRC grant ref EP/P022561/1\"

"},{"location":"Clusters/Young/","title":"MMM Hub Young","text":"

Young is the UK National Tier 2 High Performance Computing Hub in Materials and Molecular Modelling, and replacement for Thomas. Young went into pilot on 3 Aug 2020.

"},{"location":"Clusters/Young/#training-resources","title":"Training resources","text":"

The MMM Hub's Events and Training page contains useful information for new users and about specific software.

In particular the \"Software Training\" section:

"},{"location":"Clusters/Young/#mmm-hub-hpe-nvidia-gpu-training-day","title":"MMM Hub: HPE / NVIDIA GPU Training Day","text":"

The MMM Hub: HPE / NVIDIA GPU Training Day took place on 31 March 2022 and recordings of the talks are available at the link below. The day was split into two parts, \"Technology and Partner sessions (UCL, HPE & NVIDIA)\" which gave an overview of the new GPU nodes being added to Young and tools useful for GPU programming, followed by \"Materials community codes, experiences & lessons learned (invited speakers)\" which looked at how CASTEP, VASP, CP2K, GSGW and QMCpack had approached adding GPU support.

"},{"location":"Clusters/Young/#applying-for-an-account","title":"Applying for an account","text":"

Young accounts belong to you as an individual and are applied for through your own institution's Point of Contact. You will need to supply an SSH public key, which is the only method used to log in.

"},{"location":"Clusters/Young/#creating-an-ssh-key-pair","title":"Creating an ssh key pair","text":"

An ssh key consists of a public and a private part, typically named id_rsa and id_rsa.pub by default. The public part is what we need. You must not share your private key with anyone else. You can copy it onto multiple machines belonging to you so you can log in from all of them (or you can have a separate pair for each machine).

"},{"location":"Clusters/Young/#creating-an-ssh-key-in-linuxunixmac-os-x","title":"Creating an ssh key in Linux/Unix/Mac OS X","text":"
ssh-keygen\u00a0-t\u00a0rsa\n

The defaults should give you a reasonable key. If you prefer to use ed25519 instead, and/or longer keys, you can. You can also tell it to create one with a different name, so it doesn't overwrite any existing key.

You will be asked to add a passphrase for your key. A blank passphrase is not recommended; if you use one please make sure that no one else ever has access to your local computer account. How often you are asked for a passphrase depends on how long your local ssh agent keeps it.

You may need to run ssh-add to add the key to your agent so you can use it. If you aren't sure what keys your agent can see, running ssh-add -L will show all the public parts of the keys it is aware of.

"},{"location":"Clusters/Young/#creating-an-ssh-key-in-windows","title":"Creating an ssh key in Windows","text":"

Have a look at Key-Based SSH Logins With PuTTY which has step-by-step instructions. You can choose whether to use Pageant or not to manage your key. You can again pick RSA, ED25519, ECDSA etc but do not pick SSH-1 as that is a very old and insecure key type. As above, DSA is no longer accepted. The key must be at least 2048-bit.

If you are using Windows 10, then you probably have OpenSSH installed and could instead run ssh-keygen in a terminal per the Linux instructions and use the ssh command to log in instead of PuTTY.

"},{"location":"Clusters/Young/#information-for-points-of-contact","title":"Information for Points of Contact","text":"

Points of Contact have some tools they can use to manage users and allocations, documented at MMM Points of Contact.

"},{"location":"Clusters/Young/#logging-in","title":"Logging in","text":"

You will be assigned a personal username and your SSH key pair will be used to log in. External users will have a username in the form mmmxxxx (where xxxx is a number) and UCL users will use their central username.

You ssh directly to:

young.rc.ucl.ac.uk\n

Young has two login nodes and you will be round-robin assigned to one or the other each time. You can also ssh directly into a specific login node, useful if you had a tmux or screen session running there.

# replace 'mmmxxxx' with your username\nssh mmmxxxx@login01.young.rc.ucl.ac.uk\n# or\nssh mmmxxxx@login02.young.rc.ucl.ac.uk\n
"},{"location":"Clusters/Young/#ssh-timeouts","title":"SSH timeouts","text":"

Idle ssh sessions will be disconnected after 7 days.

"},{"location":"Clusters/Young/#using-the-system","title":"Using the system","text":"

Young is a batch system. The login nodes allow you to manage your files, compile code and submit jobs. Very short (\\<15mins) and non-resource-intensive software tests can be run on the login nodes, but anything more should be submitted as a job.

"},{"location":"Clusters/Young/#full-user-guide","title":"Full user guide","text":"

Young has the same user environment as RC Support's other clusters, so the User guide is relevant and is a good starting point for further information about how the environment works. Any variations that Young has should be listed on this page.

"},{"location":"Clusters/Young/#submitting-a-job","title":"Submitting a job","text":"

Create a jobscript for non-interactive use and submit your jobscript using qsub. Jobscripts must begin #!/bin/bash -l in order to run as a login shell and get your login environment and modules.

A job on Young must also specify what type of job it is (Gold, Free, Test) and the project it is being submitted for. (See Budgets and allocations below.)

"},{"location":"Clusters/Young/#memory-requests","title":"Memory requests","text":"

Note: the memory you request is always per core, not the total amount. If you ask for 192GB RAM and 40 cores, that may run on 40 nodes using only one core per node. This allows you to have sparse process placement when you do actually need that much RAM per process.

Young also has high memory nodes, where a job like this may run.

If you want to avoid sparse process placement and your job taking up more nodes than you were expecting, the maximum memory request you can make when using all the cores in a standard node is 4.6G.

"},{"location":"Clusters/Young/#monitoring-a-job","title":"Monitoring a job","text":"

In addition to qstat, nodesforjob $JOB_ID can be useful to see what proportion of cpu/memory/swap is being used on the nodes a certain job is running on.

qexplain $JOB_ID will show you the full error for a job that is in Eqw status.

"},{"location":"Clusters/Young/#useful-utilities","title":"Useful utilities","text":"

As well as nodesforjob, there are the following utilities which can help you find information about your jobs after they have run.

These utilities live in GitHub at https://github.com/UCL-RITS/go-clustertools and https://github.com/UCL-RITS/rcps-cluster-scripts

"},{"location":"Clusters/Young/#known-problems","title":"Known problems","text":""},{"location":"Clusters/Young/#slowness-in-vim","title":"Slowness in vim","text":"

Use of vim has frequent lags which can make editing (or viewing) files in it annoying. This is caused by how often vim autosaves status (every few seconds by default) which causes a lot of metadata accesses. It might include saving things like the current cursor position into .viminfo). When Young's filesystem is being particularly slow, this can make use of vim impossible.

You can turn off autosaves entirely:

:set noswapfile\n

or you can set the autosave frequency in milliseconds:

:set updatetime=\n

You can save these in your .vimrc.

"},{"location":"Clusters/Young/#software","title":"Software","text":"

Young mounts the RC Systems software stack.

Have a look at Software Guides for specific information on running some applications, including example scripts. The list there is not exhaustive.

Access to software is managed through the use of modules.

Access to licensed software may vary based on your host institution and project.

"},{"location":"Clusters/Young/#loading-and-unloading-modules","title":"Loading and unloading modules","text":"

Young has a newer version of modulecmd which tries to manage module dependencies automatically by loading or unloading prerequisites for you whenever possible.

If you get an error like this:

[uccaxxx@login01 ~]$ module unload compilers mpi\nUnloading compilers/intel/2018/update3\n  ERROR: compilers/intel/2018/update3 cannot be unloaded due to a prereq.\n    HINT: Might try \"module unload default-modules/2018\" first.\n\nUnloading mpi/intel/2018/update3/intel\n  ERROR: mpi/intel/2018/update3/intel cannot be unloaded due to a prereq.\n    HINT: Might try \"module unload default-modules/2018\" first.\n

You can use the -f option to force the module change. It will carry it out and warn you about modules it thinks are dependent.

[uccaxxx@login01 ~]$ module unload -f compilers mpi\nUnloading compilers/intel/2018/update3\n  WARNING: Dependent default-modules/2018 is loaded\n\nUnloading mpi/intel/2018/update3/intel\n  WARNING: Dependent default-modules/2018 is loaded\n
"},{"location":"Clusters/Young/#requesting-software-installs","title":"Requesting software installs","text":"

To request software installs, email us at the support address below or open an issue on our GitHub. You can see what software has already been requested in the Github issues and can add a comment if you're also interested in something already requested.

"},{"location":"Clusters/Young/#installing-your-own-software","title":"Installing your own software","text":"

You may install software in your own space. Please look at Compiling for tips.

"},{"location":"Clusters/Young/#maintaining-a-piece-of-software-for-a-group","title":"Maintaining a piece of software for a group","text":"

It is possible for people to be given central areas to install software that they wish to make available to everyone or to a select group - generally because they are the developers or if they wish to use multiple versions or developer versions. The people given install access would then be responsible for managing and maintaining these installs.

"},{"location":"Clusters/Young/#licensed-software","title":"Licensed software","text":"

Reserved application groups exist for software that requires them. The group name will begin with leg or lg. After we add you to one of these groups, the central group change will happen overnight. You can check your groups with the groups command.

Please let us know your username when you ask to be added to a group.

"},{"location":"Clusters/Young/#suggested-job-sizes","title":"Suggested job sizes","text":"

The target job sizes for Young are 2-5 nodes. Jobs larger than this may have a longer queue time and are better suited to ARCHER, and single node jobs may be more suited to your local facilities.

"},{"location":"Clusters/Young/#maximum-job-resources","title":"Maximum job resources","text":"Job type Cores GPUs Max wallclock Gold CPU job, any 5120 0 48hrs Free CPU job, any 5120 0 24hrs Free GPU job, any 320 40 48hrs Free GPU fast interactive 64 8 6hrs

CPU jobs or GPU jobs can be run on Young, and there are different nodes dedicated for each.

These are numbers of physical cores: multiply by two for virtual cores with hyperthreads on the CPU nodes.

On Young, interactive sessions using qrsh have the same wallclock limit as other jobs.

CPU jobs on Young do not share nodes, whereas GPU jobs do. This means that if you request less than 40 cores for a CPU job, your job is still taking up an entire node and no other jobs can run on it, but some of the cores are idle. Whenever possible, request a number of cores that is a multiple of 40 for full usage of your CPU nodes.

There is a superqueue for use in exceptional circumstances that will allow access to a larger number of cores outside the nonblocking interconnect zones, going across the interconnect between blocks. A third of each CU is accessible this way, roughly approximating a 1:1 connection. Access to the superqueue for larger jobs must be applied for: contact the support address below for details.

Some normal multi-node jobs will use the superqueue - this is to make it easier for larger jobs to be scheduled, as otherwise they can have very long waits if every CU is half full.

"},{"location":"Clusters/Young/#preventing-a-job-from-running-cross-cu","title":"Preventing a job from running cross-CU","text":"

If your job must run within a single CU, you can request the parallel environment as -pe wss instead of -pe mpi (wss standing for 'wants single switch'). This will increase your queue times. It is suggested you only do this for benchmarking or if performance is being greatly affected by running in the superqueue.

back to top

"},{"location":"Clusters/Young/#node-types","title":"Node types","text":"

Young has four types of node: standard nodes, big memory nodes, really big memory nodes and GPU nodes. Note those last two have different processors and number of CPU cores per node.

Type Cores per node RAM per node tmpfs Nodes Memory request necessary GPU C 40 192G None 576 Any None Y 40 1.5T None 3 mpi: mem >=19G, smp: >186G total None Z 36 3.0T None 3 mpi: mem >=42G, smp: >1530G total None X 64 1T 200G 6 Any 8 x Nvidia 40G A100

These are numbers of physical cores: multiply by two for virtual cores with hyperthreading.

The 'memory request necessary' column shows what memory requests a job needs to make to be eligible for that node type. For MPI jobs it looks at the memory per slot requested. For SMP jobs they will go on the node that their total memory request (slots * mem) fits on.

Here are the processors each node type has:

(If you ever need to check this, you can include cat /proc/cpuinfo in your jobscript so you get it in your job's .o file for the exact node your job ran on. You will get an entry for every core).

"},{"location":"Clusters/Young/#gpu-nodes","title":"GPU nodes","text":"

Now available for general use, for Free jobs only. There will be separate GPU Gold budgets in future.

How to use the GPU nodes.

"},{"location":"Clusters/Young/#restricting-to-one-node-type","title":"Restricting to one node type","text":"

The scheduler will schedule your job on the relevant nodetype based on the resources you request, but if you really need to specify the nodetype yourself, use:

# Only run on Z-type nodes\n#$ -ac allow=Z\n
"},{"location":"Clusters/Young/#hyperthreading","title":"Hyperthreading","text":"

Young has hyperthreading enabled and you can choose on a per-job basis whether you want to use it.

Hyperthreading lets you use two virtual cores instead of one physical core - some programs can take advantage of this.

If you do not ask for hyperthreading, your job only uses one thread per core as normal.

The -l threads= request is not a true/false setting, instead you are telling the scheduler you want one slot to block one virtual cpu instead of the normal situation where it blocks two. If you have a script with a threads request and want to override it on the command line or set it back to normal, the usual case is -l threads=2. (Setting threads to 0 does not disable hyperthreading!)

# request hyperthreading in this job\n#$ -l threads=1\n\n# request the number of virtual cores\n#$ -pe mpi 160\n\n# request 2G RAM per virtual core\n#$ -l mem=2G\n\n# set number of OpenMP threads being used per MPI process\nexport OMP_NUM_THREADS=2\n

This job would be using 80 physical cores, using 80 MPI processes each of which would create two threads (on Hyperthreads).

Note that memory requests are now per virtual core with hyperthreading enabled. If you asked for #$ -l mem=4Gon a node with 80 virtual cores and 192G RAM then you are requiring 320G RAM in total which will not fit on that node and so you would be given a sparse process layout across more nodes to meet this requirement.

# request hyperthreading in this job\n#$ -l threads=1\n\n# request the number of virtual cores\n#$ -pe mpi 160\n\n# request 2G RAM per virtual core\n#$ -l mem=2G\n\n# set number of OpenMP threads being used per MPI process\n# (a whole node's worth)\nexport OMP_NUM_THREADS=80\n

This job would still be using 80 physical cores, but would use one MPI process per node which would create 80 threads on the node (on Hyperthreads).

"},{"location":"Clusters/Young/#diskless-nodes","title":"Diskless nodes","text":"

Young nodes are diskless (have no local hard drives) - there is no $TMPDIR available, so you should not request -l tmpfs=10G in your jobscripts or your job will be rejected at submit time.

If you need temporary space, you should use somewhere in your Scratch.

"},{"location":"Clusters/Young/#disk-quotas","title":"Disk quotas","text":"

You have one per-user quota, with a default amount of 250GB - this is the total across home and Scratch.

If you go over quota, you will no longer be able to create new files and your jobs will fail as they cannot write.

Quota increases may be granted without further approval, depending on size and how full the filesystem is. Otherwise they may need to go to the MMM Hub User Group for approval.

back to top

"},{"location":"Clusters/Young/#budgets-and-allocations","title":"Budgets and allocations","text":"

We have enabled Gold for allocation management. Jobs that are run under a project budget have higher priority than free non-budgeted jobs. All jobs need to specify what project they belong to, whether they are paid or free.

To see the name of your project(s) and how much allocation that budget has, run the command budgets.

budgets  \nProject\u00a0\u00a0Machines\u00a0Balance\u00a0  \n--------\u00a0--------\u00a0--------  \nUCL_Test\u00a0ANY\u00a0\u00a0\u00a0\u00a0\u00a0\u00a022781.89\n

Pilot users temporarily have access to a project for their institution, eg. Imperial_pilot. These will be deactivated after the pilot and no longer show up.

Info

1 Gold unit is 1 hour of using 1 virtual processor core (= 0.5 physical core).

Since Young has hyperthreading, a job asking for 40 physical cores and one asking for 80 virtual cores with hyperthreading on both cost the same amount: 80 Gold.

"},{"location":"Clusters/Young/#subprojects","title":"Subprojects","text":"

You might be in a subproject that does not itself have an allocation, but instead takes allocation from a different project:

Project\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Machines\u00a0Balance\n--------\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0--------\u00a0--------\nUCL_physM\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0ANY\u00a0\u00a0\u00a0474999.70  \nUCL_physM_Bowler\u00a0ANY\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a00.00\n

In this case, you submit jobs using the subproject (UCL_physM_Bowler here) even though it says it has 0 budget and it takes Gold from the superproject.

"},{"location":"Clusters/Young/#submitting-a-job-under-a-project","title":"Submitting a job under a project","text":"

To submit a paid job that will take Gold from a particular project budget, add this to your jobscript:

#$\u00a0-P\u00a0Gold\n#$\u00a0-A\u00a0MyProject\n

To submit a free job that will not use up any Gold, use this instead:

#$\u00a0-P\u00a0Free\n#$\u00a0-A\u00a0MyProject\n

You can also submit testing jobs that will not use up any Gold, and will have higher priority than normal free jobs, but are limited to 2 nodes (80 cores) and 1 hour of walltime:

#$\u00a0-P\u00a0Test\n#$\u00a0-A\u00a0MyProject\n
"},{"location":"Clusters/Young/#troubleshooting-unable-to-verify-membership-in-policyjsv-project","title":"Troubleshooting: Unable to verify membership in policyjsv project","text":"
Unable\u00a0to\u00a0run\u00a0job:\u00a0Rejected\u00a0by\u00a0policyjsv\nUnable\u00a0to\u00a0verify\u00a0membership\u00a0of\u00a0`<username>`\u00a0in\u00a0the\u00a0policyjsv\u00a0project\n

You asked for a Free job but didn't specify #$ -A MyProject in your jobscript.

"},{"location":"Clusters/Young/#troubleshooting-unable-to-verify-membership-in-project-uninitialized-value","title":"Troubleshooting: Unable to verify membership in project / Uninitialized value","text":"
Unable to run job: Rejected by policyjsv \nReason:Unable to verify sufficient material worth to submit this job: \nUnable to verify membership of mmmxxxx in the UCL_Example project\n

This error from qsub can mean that you aren't in the project you are trying to submit to, but also happens when the Gold daemon is not running. 

Use of uninitialized value in print at /opt/gold/bin/mybalance line 60, <GBALANCE> line 1.\nFailed sending message: (Unable to connect to socket (Connection refused)).\n

If you also get this error from the budgets command, then the Gold daemon is definitely not running and you should contact rc-support.

"},{"location":"Clusters/Young/#gold-charging","title":"Gold charging","text":"

When you submit a job, it will reserve the total number of core hours that the job script is asking for. When the job ends, the Gold will move from 'reserved' into charged. If the job doesn't run for the full time it asked for, the unused reserved portion will be refunded after the job ends. You cannot submit a job that you do not have the budget to run.

"},{"location":"Clusters/Young/#troubleshooting-unable-to-verify-sufficient-material-worth","title":"Troubleshooting: Unable to verify sufficient material worth","text":"
Unable\u00a0to\u00a0run\u00a0job:\u00a0Rejected\u00a0by\u00a0policyjsv\nReason:Unable\u00a0to\u00a0verify\u00a0sufficient\u00a0material\u00a0worth\u00a0to\u00a0submit\u00a0this\u00a0job:\nInsufficient\u00a0balance\u00a0to\u00a0reserve\u00a0job\n

This means you don't have enough Gold to cover the cores \u2a09 wallclock time cost of the job you are trying to submit. You need to wait for queued jobs to finish and return unused Gold to your project, or submit a smaller/shorter job. Note that array jobs have to cover the whole cost of all the tasks at submit time.

"},{"location":"Clusters/Young/#job-deletion","title":"Job deletion","text":"

If you qdel a submitted Gold job, the reserved Gold will be made available again. This is done by a cron job that runs every 15 minutes, so you may not see it back instantly.

"},{"location":"Clusters/Young/#reporting-gold-usage","title":"Reporting Gold usage","text":"

There are a few commands that everyone can run that report Gold usage for their entire project, broken down by user. See Reporting from Gold.

Specifically, gstatement can show you a summary of who in your budget used what amount during a given time period:

# summarise per user usage of the MyProject budget from 00:00 on these dates\ngstatement -p MyProject -s 2023-01-01 -e 2023-04-01 --summarize\n
"},{"location":"Clusters/Young/#requests-for-longer-wallclock-time","title":"Requests for longer wallclock time","text":"

You can apply for access to a 96-hour queue for Gold jobs only using this form:

The request must be sent to your local MMM Hub point of contact (PoC) who will evaluate the request and approve or reject it, as appropriate. The list of PoCs may be found on the MMM Hub website.

Please complete the form and send it to your PoC.

The request must include a clear and compelling justification of why the usual 48-hour wall time is insufficient, how a 96-hour wall time would alleviate the problem and what it will enable you to achieve scientifically. Justifications that are solely based on simplifying your workflow and job management will not be approved - clear reasoning must be provided for why it is either very difficult or impossible to work within a 48-hour wall time.

"},{"location":"Clusters/Young/#support","title":"Support","text":"

Email rc-support@ucl.ac.uk with any support queries. It will be helpful to include Young in the subject along with some descriptive text about the type of problem, and you should mention your username in the body.

"},{"location":"Clusters/Young/#acknowledging-the-use-of-young-in-publications","title":"Acknowledging the use of Young in publications","text":"

All work arising from this facility should be properly acknowledged in presentations and papers with the following text:

\"We are grateful to the UK Materials and Molecular Modelling Hub for computational resources, which is partially funded by EPSRC (EP/T022213/1, EP/W032260/1 and EP/P020194/1)\"

"},{"location":"Clusters/Young/#mcc","title":"MCC","text":"

When publishing work that benefited from resources allocated by the MCC: please include the following acknowledgment:

\"Via our membership of the UK's HEC Materials Chemistry Consortium, which is funded by EPSRC (EP/L000202), this work used the UK Materials and Molecular Modelling Hub for computational resources, MMM Hub, which is partially funded by EPSRC (EP/T022213/1, EP/W032260/1 and EP/P020194/1)\"

"},{"location":"Clusters/Young/#ukcp","title":"UKCP","text":"

When publishing work that benefited from resources allocated by UKCP, please include:

\"We are grateful for computational support from the UK Materials and Molecular Modelling Hub, which is partially funded by EPSRC (EP/T022213/1, EP/W032260/1 and EP/P020194/1), for which access was obtained via the UKCP consortium and funded by EPSRC grant ref EP/P022561/1\"

"},{"location":"Installed_Software_Lists/module-packages/","title":"General Software Lists","text":"

Our clusters have a wide range of software installed, available by using the modules system.

The module files are organised by name, version, variant (where applicable) and, if relevant, the compiler version used to build the software. If no compiler version is given, either no compiler was required, or only the base system compiler (/usr/bin/gcc) and libraries were used.

When we install applications, we try to install them on all of our clusters, but sometimes licence restrictions prevent it. If something seems to be missing, it may be because we are not able to provide it. Please contact us for more information if this is hindering your work.

The lists below were last updated at 15:05:45 (+0100) on 09 Sep 2023, and are generated from the software installed on the Myriad cluster.

"},{"location":"Installed_Software_Lists/module-packages/#bundles","title":"Bundles","text":"

Some applications or tools depend on a lot of other modules, or have some awkward requirements. For these, we sometimes make a \"bundle\" module in this section, that loads all the dependencies.

For Python and R in particular, we also have recommended bundles that load the module for a recent version of Python or R, along with a collection of packages for it that have been requested by users, and the modules those packages require.

The lists of Python and R packages installed for those bundles are on separate pages:

We'll sometimes include /new and /old versions of these bundles, if we've recently made a version switch or are intending to make one soon. We send out emails to the user lists about version changes, so if you use these bundles, you should look out for those.

Module Description beta-modules This module adds the beta module space to your environment. bioperl/recommended Loads all the modules needed to use BioPerl. blic-modules Adds Cancer Biology supported modules to your environment. brunel-modules Adds Brunel licensed software module space to module avail. cancerit/20190218 adds UCL set of cancerit packages to your environment variables cancerit/recommended adds UCL recommended set of cancerit packages to your environment variables chemistry-modules Adds Chemistry Department supported modules to your environment. climate-tools/recommended Adds set of default applications to the environment for climate science users. deep_earth Sets up VASP, Gnuplot etc for Earth Sciences default-modules-aristotle Adds default Aristotle modules to your environment. default-modules/2015 Adds default modules to your environment. default-modules/2017 Adds default modules to your environment. default-modules/2018 Adds default modules to your environment. economics-modules Adds Economics Department modules to your environment. farr-modules Adds FARR supported modules to your environment. farr/recommended Adds set of default applications to the environment for FARR users. gmt/new Adds set of default modules to the environment for GMT users. gmt/old Adds set of default modules to the environment for gmt users. gmt/recommended Adds set of default modules to the environment for gmt users. imperial-modules Adds Imperial College licensed software module space to module avail. naglib/mark27-intel-2019 adds the NAG Library Mark 27 and required modules to your environment. octave/recommended Octave is an open source competitor to Matlab. personal-modules Adds personal modules to your environment. physics-modules Adds Pysics Department supported modules to your environment. pypy3/3.5-compat Adds UCL recommended set of Pypy3 python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/pypy-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/pypy-3.list python2/recommended Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-2.list python3/3.4 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list python3/3.5 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list python3/3.6 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list python3/3.7 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list python3/3.8 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list python3/3.9 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list python3/3.11 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list python3/recommended Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list r/new adds UCL recommended set of R packages for R 4.2.0 to your environment variables r/old adds UCL recommended set of R packages to your environment for R 3.6.0 r/r-3.5.1_bc-3.7 adds UCL recommended set of R packages to your environment variables r/r-3.6.0_bc-3.9 adds UCL recommended set of R packages to your environment variables r/recommended adds UCL recommended set of R packages for R 4.2.0 to your environment variables rsd-modules Adds Research Software Development supported modules to your environment. thermo-modules Adds modules for Molecular Thermodynamics to your environment. torch-deps Loads the dependencies for Torch and makes a quick-install alias. workaround-modules This module adds the workarounds module space to your environment."},{"location":"Installed_Software_Lists/module-packages/#applications","title":"Applications","text":"Module Description abaqus/2017 Adds Abaqus 2017 to your environment. abaqus/2017-intelmpi Adds Abaqus 2017 to your environment. abinit/9.6.2/intel-2018-update3 adds ABINIT Version 9.6.2 compiled using Intel 2018 to your environment. adf/2014.10 Adds ADF 2014.10 to your environment. afni/22.2.05 Adds AFNI to your environment. afni/23.0.02 Adds AFNI to your environment. afni/20151030 Adds AFNI to your environment. afni/20181011 Adds AFNI to your environment. amber/14/mpi/intel-2015-update2 Adds AMBER 14 to your environment amber/14/openmp/intel-2015-update2 Adds AMBER 14 to your environment amber/14/serial/intel-2015-update2 Adds AMBER 14 to your environment amber/16/mpi/gnu-4.9.2 Adds AMBER 16 to your environment amber/16/mpi/intel-2015-update2 Adds AMBER 16 to your environment amber/16/openmp/gnu-4.9.2 Adds AMBER 16 to your environment amber/16/openmp/intel-2015-update2 Adds AMBER 16 to your environment amber/16/serial/gnu-4.9.2 Adds AMBER 16 to your environment amber/16/serial/intel-2015-update2 Adds AMBER 16 to your environment ampliconarchitect/1.3r1 Amplicon Architect is a tool for finding amplicons in Illumina reads. ams/2023.101 Adds the Amsterdam Modeling Suite Version 2023.101 to your environment. ansys/17.2 Adds Ansys CFX/Fluent etc to your environment ansys/18.0 Adds Ansys CFX/Fluent etc to your environment ansys/19.1 Adds Ansys CFX/Fluent, EM etc to your environment ansys/2019.r3 Adds Ansys CFX/Fluent, EM etc to your environment ansys/2021.r2 Adds Ansys CFX/Fluent, EM etc to your environment ants/2.1.0 Adds ANTs 2.1.0 (Advanced Normalization Tools) to your environment. ANTs is popularly considered a state-of-the-art medical image registration and segmentation toolkit. approxwf/gnu-4.9.2 Adds ApproxWF to your environment. arrayfire/3.5.0/gnu-4.9.2 Adds ArrayFire 3.5.0 to your environment. asp/2.6.2 Adds NASA Ames Stereo Pipeline (ASP) 6.2.2 to your environment. autodock/4.2.6 Adds AutoDock and AutoGrid 4.2.6 to your environment. AutoDock is a suite of automated docking tools. It is designed to predict how small molecules, such as substrates or drug candidates, bind to a receptor of known 3D structure. bamtools/2.4.0/gnu-4.9.2 Adds BamTools 2.4.0 to your environment. BamTools provides both a programmer's API and an end-user's toolkit for handling BAM files. bcftools/1.2/gnu-4.9.2 Adds BCFtools 1.2 to your environment. Reading/writing BCF2/VCF/gVCF files and calling/filtering/summarising SNP and short indel sequence variants bcftools/1.3.1/gnu-4.9.2 Adds BCFtools 1.3.1 to your environment. Reading/writing BCF2/VCF/gVCF files and calling/filtering/summarising SNP and short indel sequence variants bcftools/1.11/gnu-4.9.2 Tools for reading/writing BCF2/VCF/gVCF files and calling/filtering/summarising SNP and short indel sequence variants. bcftools/2.1/gnu-4.9.2 Adds BCFtools 1.2 to your environment. Reading/writing BCF2/VCF/gVCF files and calling/filtering/summarising SNP and short indel sequence variants bcl2fastq/1.8.4 Adds bcl2fastq 1.8.4 to your environment. bcl2fastq2/2.19.1 Adds bcl2fastq2 2.19.1 to your environment. bcl2fastq2/2.20.0-rpm Adds bcl2fastq2 2.20.0.422 from rpm to your environment. beast/2.3.0 Adds BEAST 2.3.0 with addons to your PATH. bedtools/2.25.0 Adds bedtools 2.25.0 to your environment. The bedtools utilities are a swiss-army knife of tools for a wide-range of genomics analysis tasks. bgen/1.1.4 Adds BGen 1.1.4 to your environment. blast+/2.2.30/intel-2015-update2 This module adds the BLAST+ 2.2.30 package to your environment. blast+/2.13.0/gnu-7.3.0 This module adds the BLAST+ 2.13.0 package to your environment. blast/2.2.26 Adds Blast 2.2.26 to your environment. blender/2.79 Adds Blender Version 2.79 to your environment. boltztrap/1.2.5/intel-2018 Adds boltztrap 1.2.5 to your environment. bowtie/1.1.2 Adds Bowtie 1.1.2 to your environment. bowtie2/2.2.5 Adds Bowtie2 2.2.5 to your environment. bwa/0.6.2/gnu-4.9.2 Adds BWA 0.7.12 to your environment. BWA is a software package for mapping DNA sequences against a large reference genome, such as the human genome. bwa/0.7.12/gnu-4.9.2 Adds BWA 0.7.12 to your environment. BWA is a software package for mapping DNA sequences against a large reference genome, such as the human genome. caffe/1.0/cpu Adds Caffe 1.0 for CPU to your environment. caffe/1.0/cudnn Adds Caffe 1.0 for CUDA+CudNN to your environment. caffe/1.0/gpu Adds Caffe 1.0 for CUDA to your environment. cancerit/20190218-python-2.7.12/gnu-4.9.2 Adds CancerIT program versions as of 20190218 to your environment. The CancerIT Suite is a collection of linked bioinformatics tools. cancerit/gnu-4.9.2 Adds the cancer it suite to your environment. castep/17.2/intel-2017 Adds castep 17.2 to your environment. CASTEP is a program that uses density functional theory to calculate the properties of materials from first principles. castep/17.21/intel-2017 Adds castep 17.21 to your environment. CASTEP is a program that uses density functional theory to calculate the properties of materials from first principles. castep/19.1.1/intel-2019 CASTEP is a program that uses density functional theory to calculate the properties of materials from first principles. cctools/5.4.1/gnu-4.9.2 Adds cctools 5.4.1 to your environment. cctools/7.0.11/gnu-4.9.2 Adds cctools 7.0.11 to your environment. cdo/2.0.6/gnu-10.2.0 This module adds the CDO 2.0.6 package to your environment. cellranger/5.0.1 Adds Cell Ranger 5.0.1 to your environment. Cell Ranger is a set of analysis pipelines that process Chromium single-cell RNA-seq output to align reads, generate feature-barcode matrices and perform clustering and gene expression analysis. cellranger/6.0.1 Adds Cell Ranger 6.0.1 to your environment. Cell Ranger is a set of analysis pipelines that process Chromium single-cell RNA-seq output to align reads, generate feature-barcode matrices and perform clustering and gene expression analysis. cesm/1.0.6/intel-2015-update2 Adds CESM 1.0.6 to your environment. cesm/1.2.2/intel-2015-update2 Adds CESM 1.2.2 to your environment. cfd-ace/2014.1 Adds CFD-ACE+ to your execution path. Only on Kathleen and Myriad. cfd-ace/2018.0 Adds CFD-ACE+ to your execution path. Only on Kathleen and Myriad. chemshell/3.7.1/mpi/gulp4.5 This is a modulefile for ChemShell 3.7.1, MPI+GULP version. Can be used to run other packages if you load a module for those. chemshell/3.7.1/standalone This is a modulefile for ChemShell 3.7.1, standalone serial version. Can be used to run GULP and other packages if you load a module for those. clustal-omega/1.2.1 Adds Clustal Omega 1.2.1 to your environment. clustal-w/2.1 Adds Clustal W 2.1 to your environment. cmg/2017.101 Adds CMG Reservoir Simulation Software Version 2017.101 to your environment. cmg/2018.101 Adds CMG Reservoir Simulation Software Version 2018.101 to your environment. cmg/2019.101 Adds CMG Reservoir Simulation Software Version 2019.101 to your environment. collectl/4.0.2 [collectl/4.0.2] collectl is a tool for tracking and monitoring various node usage statistics. compucell3d/3.7.4 Adds CompuCell3D to your environment comsol/6.0 COMSOL is a general-purpose macroscopic physics simulation package. comsol/6.1 COMSOL is a general-purpose macroscopic physics simulation package. comsol/6.1-eee COMSOL is a general-purpose macroscopic physics simulation package. Installed for EEE Dept comsol/52 Adds the COMSOL 52 binaries to your environment. COMSOL Multiphysics\u00ae is a general-purpose software platform, based on advanced numerical methods, for modeling and simulating physics-based problems. Module must be loaded once from a login node prior to running jobs. comsol/52a Adds the COMSOL 52a binaries to your environment. COMSOL Multiphysics\u00ae is a general-purpose software platform, based on advanced numerical methods, for modeling and simulating physics-based problems. Module must be loaded once from a login node prior to running jobs. comsol/53a Adds COMSOL Multiphysics Version 53a to your environment. comsol/56 Adds COMSOL Multiphysics Version 56 to your environment. cosi-corr/oct14 Adds COSI-Corr Version Oct14 for use with ENVI 5.5.2/5.5.3 to your environment. covid-19-spatial-sim/0.8.0/intel-2020 SpatialSim COVID-19 pandemic modelling tool from Imperial College. covid-19-spatial-sim/0.9.0/gnu-4.9.2 SpatialSim COVID-19 pandemic modelling tool from Imperial College. covid-19-spatial-sim/0.13.0/gnu-4.9.2 SpatialSim COVID-19 pandemic modelling tool from Imperial College. covid-19-spatial-sim/0.14.0/gnu-4.9.2 SpatialSim COVID-19 pandemic modelling tool from Imperial College. covid-19-spatial-sim/0.14.0/intel-2020 SpatialSim COVID-19 pandemic modelling tool from Imperial College. covid-19-spatial-sim/0.15.0/gnu-4.9.2 SpatialSim COVID-19 pandemic modelling tool from Imperial College. covid-19-spatial-sim/0.15.0/intel-2020 SpatialSim COVID-19 pandemic modelling tool from Imperial College. cp2k/4.1/ompi/gnu-4.9.2 Adds CP2K to your environment. cp2k/5.1/ompi-plumed/gnu-4.9.2 Adds CP2K to your environment. cp2k/5.1/ompi/gnu-4.9.2 Adds CP2K to your environment. cp2k/6.1/ompi/gnu-4.9.2 Adds CP2K to your environment. cp2k/7.1/ompi/gnu-4.9.2 Adds CP2K to your environment. cpmd/4.1/intel-2017 Adds CPMD 4.1 to your environment. crystal14/v1.0.3 Adds Crystal14 v1.0.3 to your environment. crystal14/v1.0.4 Adds Crystal14 v1.0.4 to your environment. crystal14/v1.0.4_2017 Adds Crystal14 v1.0.4 to your environment. crystal17/v1.0.1 Adds Crystal17 v1.0.1 to your environment. crystal17/v1.0.2/intel-2017 The CRYSTAL program computes the electronic structure of periodic systems within Hartree Fock, density functional or various hybrid approximations. cuba/4.2/gnu-4.9.2 adds Cuba Numerical Integration Package Version 4.2 to your environment. cufflinks/2.2.1 Adds Cufflinks 2.2.1 to your environment. curl/7.47.1/gnu-4.9.2 Adds curl 7.47.1 to your environment. curl/7.86.0/gnu-4.9.2 Adds curl 7.86.0 to your environment. dakota/6.12.0-bindist Dakota is a package for performing parametric analysis with other software packages. datamash/1.4 This is a module with no description string. deeptools/3.0.2 Adds deeptools to your environment. delly/0.7.8-bindist Delly is an integrated structural variant (SV) prediction method that can discover, genotype and visualize deletions, tandem duplications, inversions and translocations at single-nucleotide resolution in short-read massively parallel sequencing data. dftbplus/17.1/intel-2017 DFTB+ is a quantum mechanical simulation software package, based on the Density Functional Tight Binding (DFTB) method. dftbplus/18.2/intel-2018 DFTB+ is a software package for carrying out fast quantum mechanical atomistic calculations based on the Density Functional Tight Binding method. dftbplus/19.1/intel-2018 DFTB+ is a software package for carrying out fast quantum mechanical atomistic calculations based on the Density Functional Tight Binding method. dftbplus/dev/d07f92e/intel-2017 DFTB+ is a quantum mechanical simulation software package, based on the Density Functional Tight Binding (DFTB) method. dl_monte/2.07/intel-2018 DL_MONTE is an open-source program for applying Monte Carlo molecular simulation to a wide range of systems. dl_poly/4.07/intel-2015-update2 Adds DL_POLY 4.07 to your environment dl_poly/4.08-plumed-2.3.1/intel-2017 Adds dl_poly 4.08 to your environment. DL_POLY is a general purpose classical molecular dynamics (MD) simulation software developed at Daresbury Laboratory. This version has been linked against the PLUMED metadynamics library. dl_poly/4.08/intel-2015-update2 Adds DL_POLY 4.08 to your environment. dl_poly/4.09/intel-2018 Adds DL_POLY 4.09 to your environment. dl_poly/4.10.0/intel-2018 Adds DL_POLY 4.10.0 to your environment. DL_POLY is a general purpose classical molecular dynamics (MD) simulation software. dl_poly/5.0.0/intel-2018 DL_POLY is a general-purpose classical molecular dynamics (MD) simulation software package. dl_poly/classic/1.9/intel-2015-update2 Adds DL_POLY Classic 1.9 to your environment dock/6.9-impi/intel-2018 The DOCK suite of programs is designed to find favorable orientations of a ligand in a receptor. This is the Intel MPI build, intended for high-performance parallel runs. dock/6.9-reference/gnu-4.9.2 The DOCK suite of programs is designed to find favorable orientations of a ligand in a receptor. This is a reference build intended to be close to the version of the software the developers test with: a serial build using the GNU compilers. dos2unix/7.3 Adds dos2unix 7.3 to your environment. Text format converters dos2unix, unix2dos, mac2unix, unix2mac. dssp/3.0.0/gnu-4.9.2 Adds dssp 3.0.0 to your environment. DSSP calculates DSSP entries from Protein Databank (PDB) entries. dymola/2020.1-1 Dymola is a commercial modeling and simulation environment based on the open Modelica modeling language. ea-utils/822 Adds ea-utils to your environment. easylausanne/55c7bf0 Adds Easy Lausanne to your environment. eigensoft/6.1.1/gnu-4.9.2 Adds EIGENSOFT 6.1.1 to your environment. Population genetics methods and EIGENSTRAT stratification correction method. elk/4.0.15/intel-2018 Adds Elk 4.0.15 to your environment. Binary is elk. elk/4.0.15/intel-2018+wa Adds Elk 4.0.15 to your environment. Binary is elk. elk/4.3.6/intel-2017 Adds Elk 4.3.6 to your environment. Binary is elk. elk/4.3.6/intel-2017+wa Adds Elk 4.3.6 to your environment. Binary is elk. elk/5.2.14/intel-2018 Elk is an all-electron full-potential linearised augmented-planewave (FP-LAPW) code. elk/6.8.4/intel-2018 An all-electron full-potential linearised augmented-planewave (FP-LAPW) code. energyplus/8.9.0-bindist EnergyPlus\u2122 is a whole building energy simulation program that engineers, architects, and researchers use to model both energy consumption\u2014for heating, cooling, ventilation, lighting and plug and process loads\u2014and water use in buildings. energyplus/9.1.0-bindist EnergyPlus\u2122 is a whole building energy simulation program that engineers, architects, and researchers use to model both energy consumption\u2014for heating, cooling, ventilation, lighting and plug and process loads\u2014and water use in buildings. envi/5.5.2 Adds ENVI 5.5.2 with IDL 8.7.2 to your environment. envi/5.5.3 Adds ENVI 5.5.3 with IDL 8.7.3 to your environment. epacts/3.3.0/gnu-4.9.2 Adds EPACTS 3.3.0 to your environment. examl/8dcf2cc/gnu-4.9.2 Adds ExaML to your environment. exonerate/2.2.0 Adds Exonerate to your environment. fasta/36.3.8d/gnu-4.9.2 Adds the cancer it suite to your environment. fastqc/0.11.5 Adds FastQC 0.11.5 to your environment. A quality control application for high throughput sequence data. fastqc/0.11.8 Adds FastQC 0.11.8 to your environment. A quality control application for high throughput sequence data. ffmpeg/4.1/gnu-4.9.2 FFmpeg is a framework for encoding, decoding, muxing, demuxing, encoding, transcoding, streaming, filtering, and playing many types of audio and video media. fgbio/0.5.1 Adds fgbio to your environment. fgbio is a command line toolkit for working with genomic and particularly next generation sequencing data. fgbio/0.6.1 Adds fgbio to your environment. fgbio is a command line toolkit for working with genomic and particularly next generation sequencing data. figtree/1.4.2 Adds Figtree 1.4.2. foldx/4 Adds FoldX Suite 4.0 to your environment. foldx/5 Adds FoldX Suite 5.0 to your environment. freesurfer/5.3.0 Adds FreeSurfer 5.3.0 to your environment. FreeSurfer is a set of automated tools for reconstruction of the brain's cortical surface from structural MRI data, and overlay of functional MRI data onto the reconstructed surface. freesurfer/6.0.0 Adds FreeSurfer 6.0.0 to your environment. FreeSurfer is a set of automated tools for reconstruction of the brain's cortical surface from structural MRI data, and overlay of functional MRI data onto the reconstructed surface. freesurfer/7.2.0 Adds FreeSurfer 7.2.0 to your environment. FreeSurfer is a set of automated tools for reconstruction of the brain's cortical surface from structural MRI data, and overlay of functional MRI data onto the reconstructed surface. fsl/5.0.9 Adds FSL 5.0.9 (FMRIB Software Library) to your environment. FSL is a comprehensive library of analysis tools for FMRI, MRI and DTI brain imaging data. fsl/5.0.10 Adds FSL 5.0.10 (FMRIB Software Library) to your environment. FSL is a comprehensive library of analysis tools for FMRI, MRI and DTI brain imaging data. fsl/6.0.0 Adds FSL 6.0.0 (FMRIB Software Library) to your environment. FSL is a comprehensive library of analysis tools for FMRI, MRI and DTI brain imaging data. fsl/6.0.0_cuda Adds FSL 6.0.0 CUDA (FMRIB Software Library) to your environment. FSL is a comprehensive library of analysis tools for FMRI, MRI and DTI brain imaging data. fsl/6.0.4 Adds FSL 6.0.4 CUDA (FMRIB Software Library) to your environment. FSL is a comprehensive library of analysis tools for FMRI, MRI and DTI brain imaging data. gamess/5Dec2014_R1/intel-2015-update2 Adds GAMESS 5Dec2014_R1 to your environment, built for Intel MPI. Uses ~/Scratch/gamess for USERSCR. You can override by exporting GAMESS_USERSCR as another path. gatk/3.4.46 Adds GATK 3.4.46 to your environment. The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyze high-throughput sequencing data. Website: https://www.broadinstitute.org/gatk/index.php gatk/3.8.0 Adds GATK 3.8.0 to your environment. The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyze high-throughput sequencing data. Website: https://www.broadinstitute.org/gatk/index.php gatk/4.0.3.0 Adds GATK 4.0.3.0 to your environment. The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyze high-throughput sequencing data. Website: https://www.broadinstitute.org/gatk/index.php gatk/4.0.8.0 Adds GATK 4.0.8.0 to your environment. The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyze high-throughput sequencing data. Website: https://www.broadinstitute.org/gatk/index.php gatk/4.2.1.0 Adds GATK 4.2.1.0 to your environment. The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyze high-throughput sequencing data. Website: https://www.broadinstitute.org/gatk/index.php gatk/4.2.5.0 Adds GATK 4.2.1.0 to your environment. The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyze high-throughput sequencing data. Website: https://www.broadinstitute.org/gatk/index.php gaussian/g09-c01_linda/pgi-2013.9 Adds Gaussian 09 Revision C01 and GaussView 5 to your environment. gaussian/g09-d01/pgi-2015.4 Adds Gaussian G09-D01 to your environment and also includes Linda and Gaussview 5. gaussian/g09-d01/pgi-2015.7 Adds Gaussian G09-D01 to your environment and also includes Linda and Gaussview 5 gaussian/g16-a03/pgi-2016.5 Adds Gaussian G16-A03 to your environment and also includes Linda and Gaussview 6. gaussian/g16-c01/pgi-2018.10 Adds Gaussian G16-C01 to your environment and also includes Linda and Gaussview 6. gcta/1.93.2beta GCTA: a tool for Genome-wide Complex Trait Analysis. gdal/2.0.0 Adds GDAL 2.0.0 to your environment variables. Works with Python 2. gdal/2.1.1 adds GDAL 2.1.1 with PROJ.4 4.9.1 to your environment variables. Works with Python 2. gdal/2.1.4 adds GDAL 2.1.4 with PROJ.4 6.1.0 to your environment variables. Works with Python 2. gdal/3.0.4/gnu-4.9.2 adds GDAL 3.0.4 with PROJ.4 6.1.0 to your environment variables. Works with Python 2. gephi/0.9.1 Adds Gephi Version 0.9.1 to your environment. gftp/2.9.1b gFTP is a file transfer client that supports a wide range of FTP-like protocols. ghostscript/9.16/gnu-4.9.2 Adds Ghostscript 9.16 to your environment. ghostscript/9.19/gnu-4.9.2 Adds Ghostscript 9.19 to your environment. gmsh/2.12.0-bindist Adds gmsh 2.12.0 to your environment. Gmsh is a free 3D finite element grid generator with a build-in CAD engine and post-processor. gmt/5.1.2 adds GMT 5.1.2 to your environment variables gmt/5.3.1 adds GMT 5.3.1 to your environment variables gmt/5.4.5 adds GMT 5.4.5 to your environment variables gnuplot/5.0.1 Adds gnuplot 5.0.1 to your environment. Gnuplot is a portable command-line driven graphing utility. grace/5.1.25 Adds Grace 5.1.25 to your environment. Grace is a 2D plotting tool. graphicsmagick/1.3.21 adds GraphicsMagick 1.3.21 to your environment variables graphviz/2.38.0/gnu-4.9.2 This module adds the Graphviz 2.38.0 package to your environment. Graphviz is open source graph visualization software. graphviz/2.40.1/gnu-4.9.2 This module adds the Graphviz 2.40.1 package to your environment. Graphviz is open source graph visualization software. groff/1.22.3/gnu-4.9.2 Adds GNU groff Version 1.22.3 to your environment. gromacs/5.0.4/intel-2015-update2 Adds GROMACS 5.0.4 to your environment, built using MKL gromacs/5.0.4/plumed/intel-2015-update2 Adds GROMACS 5.0.4 with Plumed 2.1.2 to your environment. Note: Plumed will always run in double precision even if GROMACS is single-precision, so only use that combination if you need it and are aware of the effects. gromacs/5.1.1/intel-2015-update2 Adds GROMACS 5.1.1 to your environment, built using MKL gromacs/5.1.1/plumed/intel-2015-update2 Adds GROMACS 5.1.1 with Plumed 2.2 to your environment. Note: Plumed will always run in double precision even if GROMACS is single-precision, so only use that combination if you need it and are aware of the effects. gromacs/5.1.3/plumed/intel-2015-update2 GROMACS 5.1.3 molecular dynamics package, built with Intel 2015u2 compilers, PLUMED 2.2.3 patches (including libmatheval), and OpenBLAS 0.2.14. gromacs/2016.3/intel-2017-update1 Adds GROMACS 2016.3 to your environment, built using MKL gromacs/2016.3/plumed/intel-2017-update1 GROMACS 2016.3 molecular dynamics package, built with Intel 2017u1 compilers, PLUMED 2.3.1 patches (including libmatheval), and OpenBLAS 0.2.14. gromacs/2016.4/plumed/intel-2017 GROMACS 2016.4 molecular dynamics package, built with Intel 2017u4 compilers, PLUMED 2.4.1 patches (including libmatheval) with hrex, and OpenBLAS 0.2.14. gromacs/2018.2/intel-2018 Adds gromacs 2018 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2018.3/intel-2018 Adds gromacs 2018 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2018.3/plumed/intel-2018 GROMACS 2018.3 molecular dynamics package, built with Intel 2018u3 compilers, PLUMED 2.4.3 patches (including libmatheval). gromacs/2018/intel-2017 Adds gromacs 2018 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2019.3/cuda-10 Adds gromacs 2019 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2019.3/intel-2018 Adds gromacs 2019 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2019.3/plumed/intel-2018 GROMACS 2019.3 molecular dynamics package, built with Intel 2018u3 compilers, PLUMED 2.5.2 patches (including libmatheval). gromacs/2021.3/plumed/gnu-10.2.0 GROMACS 2021.3 molecular dynamics package, built with GNU 10.2.0 compilers, PLUMED 2.7.2 patches (including libmatheval). gulp/4.5/intel-2018 Adds GULP 4.5 to your environment. Built with FoX and without plumed. GULP is a materials simulation code. gulp/5.1.1/intel-2018 Adds GULP 5.1.1 to your environment. Built with FoX and without plumed. GULP is a materials simulation code. gurobi/7.5.1 Adds Gurobi 7.5.1 to your environment. gurobi/8.1.1 Adds Gurobi 8.1.1 to your environment. gurobi/9.1.2 Adds Gurobi 9.1.2 to your environment. h5utils/1.12.1 Adds h5utils 1.12.1 to your environment. h5utils is a set of utilities for visualization and conversion of scientific data in HDF5 format. hammock/1.0.5 Loads the dependencies for Hammock 1.0.5 to your environment and makes a quick-install alias, do-hammock-install. Run as java -Xmx2g -jar $HAMMOCKPATH/Hammock.jar mode param1 param2 -d outputpath. Will use Scratch for temporary files. hhsuite/3.0-beta.1/gnu-4.9.2 Adds hhsuite 3.0-beta.1 to your environment. hisat2/2.2.1/gnu-4.9.2 Adds HISAT2 Version 2.2.1 to your environment. hmmer/3.1b2 Adds HMMER 3.1b2 to your environment. hmri/0.4.0/spm12.jan2020 Adds the hMRI Toolbox for use with SPM12 to your environment hoomd-blue/2.4.2 Adds HOOMD-blue to your environment. hopspack/2.0.2/gnu-4.9.2 Adds HOPSPACK 2.0.2 to your environment hopspack/2.0.2/intel-2017 Adds HOPSPACK 2.0.2 to your environment icommands/4.1.7 [icommands/4.1.7] The iRODS iCommands are the command-line clients to an iRODS system. idl/8.4.1 Adds IDL 8.4.1 to your environment. idl/8.7.3 Adds IDL 8.7.3 to your environment. idl/8.8.3 Adds IDL 8.8.3 to your environment. illustrate/20190807 adds Illustrate to your environment variables impute2/2.3.2 adds Impute2 V2.3.2 to your environment. inetutils/1.9.4 GNU inetutils is a package of utilities for performing a range of network tasks including FTP and telnet clients. intltool/0.51.0 Adds intltool 0.51.0 to your environment. intltool is a set of tools to centralize translation of many different file formats using GNU gettext-compatible PO files. iva/0.11.6 Adds IVA 0.11.6 to your environment. iva/1.0.0 Adds IVA 1.0.0 to your environment. jags/3.4.0/gnu.4.9.2-atlas Adds JAGS 3.4.0 to your environment. jags/3.4.0/gnu.4.9.2-openblas Adds JAGS 3.4.0 to your environment. jags/4.2.0/gnu.4.9.2-openblas Adds JAGS 4.2.0 to your environment. jq/1.5/gnu-4.9.2 adds jq for GCC 4.9.2 to your environment. jq/1.6-bindist jq is a lightweight and flexible command-line JSON processor. kallisto/v0.42.5 Adds Kallisto v0.42.5 to your environment. kallisto/v0.46.1 Adds Kallisto v0.46.1 to your environment. keras/2.2.4 Adds Keras to your environment. kmc/2.1.1/gnu-4.9.2 Adds KMC 2.1.1 to your environment. KMC is a disk-based program for counting k-mers from FASTQ/FASTA files. knitro/12.0.0/gnu-4.9.2 Adds Knitro solver 12.0.0 to your environment. knitro/12.4.0/gnu-4.9.2 Adds Knitro solver 12.4.0 to your environment. knitro/13.1.0/gnu-4.9.2 Adds Knitro solver 13.1.0 to your environment. lammps/3Mar20/plumed-colvars/intel-2018 Adds LAMMPS 3Mar20 to your environment. LAMMPS is a GPL molecular dynamics code which shows exceptional scaling on a wide variety of machines. Binary is lmp_mpi or lmp_default. This version was built with packages kspace, manybody, molecule, rigid, lib-linalg, user-colvars and user-plumed. lammps/7Aug19/basic/intel-2018 Adds LAMMPS 7Aug19 to your environment. Binary is lmp_default. lammps/7Aug19/gpu/intel-2018 Adds LAMMPS 7Aug19 to your environment. Binary is lmp_default. lammps/7Aug19/userintel/intel-2018 Adds LAMMPS 7Aug19 to your environment. Binary is lmp_default. lammps/8Dec15/intel-2015-update2 Adds LAMMPS 8Dec15 to your environment. Binary is lmp_default. lammps/10Feb15/intel-2015-update2 Adds LAMMPS 10Feb15 to your environment. Binary is lmp_default. lammps/13Apr17/intel-2017 Adds LAMMPS 13Apr17 to your environment. Binary is lmp_default. lammps/16Mar18/basic/intel-2018 Adds LAMMPS 16Mar18 to your environment. Binary is lmp_default. lammps/16Mar18/gpu/intel-2018 Adds LAMMPS 16Mar18 to your environment. Binary is lmp_default. lammps/16Mar18/intel-2017 Adds LAMMPS 16Mar18 to your environment. Binary is lmp_default. lammps/16Mar18/userintel/intel-2018 Adds LAMMPS 16Mar18 to your environment. Binary is lmp_default. lynx/2.8.9 Adds Lynx Version 2.8.9 to your environment. mathematica/10.1.0 Adds Mathematica 10.1.0 to your environment. mathematica/10.2.0 Adds Mathematica 10.2.0 to your environment. mathematica/10.4.0 Adds Mathematica 10.4.0 to your environment. mathematica/11.0.1 Adds Mathematica 11.0.1 to your environment. mathematica/11.2.0 Adds Mathematica 11.2 to your environment. mathematica/11.3.0 Adds Mathematica 11.3 to your environment. mathematica/12.2.0 Adds Mathematica 12.2 to your environment. mathematica/13.1.0 Adds Mathematica 13.1 to your environment. matlab/full/r2015a/8.5 Adds Matlab R2015a for SPM to your environment. matlab/full/r2015b/8.6 Adds Matlab R2015b to your environment. matlab/full/r2016b/9.1 Adds Matlab R2016b to your environment. matlab/full/r2017a/9.2 Adds Matlab R2017a to your environment. matlab/full/r2018a/9.4 Adds Matlab R2018a to your environment. matlab/full/r2018b/9.5 Adds Matlab R2018b to your environment. matlab/full/r2019b/9.7 Adds Matlab R2019b to your environment. matlab/full/r2021a/9.10 Adds Matlab R2021a to your environment. matlab/full/r2023a/9.14 Adds Matlab R2023a to your environment. mcl/14-137 Adds MCL 14-137 your environment. meep/1.3-ompi/gnu-4.9.2 Adds meep 1.3-ompi to your environment. meep/1.3/gnu-4.9.2 Adds meep 1.3 to your environment. meep/1.11.0-ompi/gnu-4.9.2 Adds meep 1.11.0-ompi to your environment. MEEP is a package for electromagnetics simulation via the finite-diffe rence time-domain (FDTD) method. meme/4.10.1_4 Adds MEME Suite 4.10.1_4 to your environment. The MEME Suite: Motif-based sequence analysis tools. This install is for the command-line tools and connects to their website for further analysis. Web: http://meme-suite.org mgltools/1.5.6 Adds MGLTools 1.5.6 to your environment. Applications for visualization and analysis of molecular structures. Contains AutoDockTools (ADT), Python Molecular Viewer (PMV) and Vision. mirdeep/2.0.0.7 Adds mirdeep 2.0.0.7 to your environment. molden/5.2.2 Adds Molden 5.2.2 to your environment. molpro/2012.1.25/gnu-4.9.2 Adds Molpro to your environment molpro/2015.1.3 Adds Molpro 2015.1.3 binary (no Infiniband support) to your environment. molpro/2015.1.5/intel-2015-update2 Adds Molpro 2015.1.5 built from source with MPI to your environment. molpro/2020.1/openmp Adds Molpro 2020.1 mpi binary to your environment. mosek/8.1.0.83 The MOSEK Optimization Suite is a software package for solving large optimization problems with many constraints and variables. mosek/9.1.12 Adds Mosek 9.1.12 to your environment. mothur/1.41.3-bindist Mothur is an expandable, multi-purpose bioinformatics tool aimed at microbial ecology. mpb/1.5-ompi/gnu-4.9.2 Adds mpb 1.5 to your environment. mpb/1.5/gnu-4.9.2 Adds mpb 1.5 to your environment. mpb/1.9.0-hdf5-ompi/gnu-4.9.2 Adds serial mpb 1.9.0 to your environment. Built with HDF5-ompi for use by parallel MEEP. mrbayes/3.2.5/mpi/intel-2015-update2 Adds MrBayes 3.2.5 to your environment mrbayes/3.2.5/serial/intel-2015-update2 Adds MrBayes 3.2.5 to your environment mrtrix/0.3.12/nogui Adds MRtrix3 0.3.12 to your environment. MRtrix3 provides a set of tools to perform analysis of diffusion MRI data, based around the concept of spherical deconvolution and probabilistic tractography. Note: mrview and shview cannot be run over a remote X11 connection so are not usable. mrtrix/0.3.16/gnu-4.9.2/nogui MRtrix provides a set of tools to perform various advanced diffusion MRI analyses, including constrained spherical deconvolution (CSD), probabilistic tractography, track-density imaging, and apparent fibre density. mrtrix/3.0.4/gnu-10.2.0/nogui Adds MRtrix 3.0.4 to your environment. mrtrix/3.0rc3/gnu-4.9.2/nogui Adds MRtrix 3.0RC3 to your environment. mstor/2013/gnu-4.9.2 MSTor is a program for calculating partition functions, free energies, enthalpies, entropies, and heat capacities of complex molecules including torsional anharmonicity. mumax/3.9.3 Adds Mumax 3.9.3 to your environment. mumax/3.10-bindist mumax3 is a GPU-accelerated micromagnetic simulation program. mummer/3.23/gnu-4.9.2 Adds MUMmer 3.23 to your environment. MUMmer is a system for rapidly aligning entire genomes, whether in complete or draft form. muscle/3.8.31 Adds MUSCLE 3.8.31 to your environment. mutect/1.1.7 Adds MuTect 1.1.7 to your environment. MuTect is a GATK-based variant caller specialized for somatic/cancer variants. namd/2.10/intel-2015-update2 Adds NAMD 2.10 to your environment namd/2.11/intel-2015-update2 Adds NAMD 2.11 to your environment namd/2.12/intel-2015-update2 Adds NAMD 2.12 to your environment namd/2.12/intel-2017-update1 Adds NAMD 2.12 to your environment namd/2.12/intel-2018-update3 Adds NAMD 2.12 to your environment namd/2.13/intel-2018-update3 Adds NAMD 2.13 to your environment namd/2.13/plumed/intel-2018-update3 Adds NAMD 2.13 to your environment namd/2.14/multicore-gpu NAMD is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. This is NAMD's Linux-x86_64-multicore-CUDA binary. namd/2.14/ofi-smp-gpu/intel-2019 Adds NAMD 2.14 OFI-SMP-GPU version to your environment. This version should be run using charmrun. namd/2.14/ofi-smp/intel-2019 Adds NAMD 2.14 OFI-SMP version to your environment. This version should be run using charmrun. namd/2.14/ofi/intel-2019 Adds NAMD 2.14 OFI version to your environment. This version should be run using charmrun. ncdu/2.2.1-bindist Ncdu is a disk usage analyzer with an ncurses interface. nco/4.5.0 Adds nco to your environment. nektar++/4.3.5-impi/intel-2017-update1 Adds Nektar++ Version 4.3.5 to your environment nektar++/4.3.5-ompi/gnu-4.9.2 Adds Nektar++ Version 4.3.5 to your environment ngsutils/0.5.9 Adds a set of python scripts for handling various NGS tasks to your environment. nighres/1.1.0b Adds Nighres to your environment. nonmem/7.3.0/gnu-4.9.2 Adds NONMEM 7.3.0 using GCC Fortran 4.9.2 to your environment. nonmem/7.3.0/intel-2015-update2 Adds NONMEM 7.3.0 using Intel Fortran 2015 to your environment. novocraft/3.04.06 Adds novocraft 3.04.06 to your environment. Novocraft is a set of tools for bioinformatics, including Novoalign for short-read mapping. nwchem/6.5-r26243/atlas/intel-2015-update2 Adds NWChem 6.5 revision 26243 to your PATH, creates symlink to global .nwchemrc. You may need to alter/remove any old ~/.nwchemrc. Built with Python 2.7 interface and ATLAS. Global .nwchemrc: /shared/ucl/apps/nwchem/6.5-r26243-atlas/intel-2015-update2.nwchemrc nwchem/6.5-r26243/intel-2015-update2 Adds NWChem 6.5 revision 26243 to your PATH, creates symlink to global .nwchemrc. You may need to alter/remove any old ~/.nwchemrc. Built with Python 2.7 interface and MKL with ScaLAPACK. Global .nwchemrc: /shared/ucl/apps/nwchem/6.5-r26243/intel-2015-update2/.nwchemrc nwchem/6.6-r27746/intel-2015-update2 Adds NWChem 6.6 revision 27746 patched 2016-01-20 to your PATH, creates symlink to global .nwchemrc. You may need to alter/remove any old ~/.nwchemrc. Built with Python 2.7 interface and MKL with ScaLAPACK. Global .nwchemrc: /shared/ucl/apps/nwchem/6.6-r27746/intel-2015-update2/.nwchemrc nwchem/6.6-r27746/intel-2017 Adds NWChem 6.6 revision 27746 patched 2016-01-20 to your PATH, creates symlink to global .nwchemrc. You may need to alter/remove any old ~/.nwchemrc. Built with Python 2.7 interface and MKL with ScaLAPACK. Global .nwchemrc: /shared/ucl/apps/nwchem/6.6-r27746/intel-2017/.nwchemrc nwchem/6.8-47-gdf6c956/intel-2017 Adds NWChem 6.8 47-gdf6c956 to your PATH, creates symlink to global .nwchemrc. You may need to alter/remove any old ~/.nwchemrc. Built with Python 2.7 interface and MKL with ScaLAPACK. Global .nwchemrc: /shared/ucl/apps/nwchem/6.8-47-gdf6c956/intel-2017/.nwchemrc oasislmf/1.2.4 Oasis LMF 1.2.4 oasislmf/1.26.3 Oasis LMF 1.26.3 oasislmf/ktools/3.0.3/gnu-4.9.2 OasisLMF ktools package built with the GNU compilers oasislmf/ktools/3.4.1/gnu-4.9.2 OasisLMF ktools package built with the GNU compilers oasislmf/ktools/3.9.5/gnu-4.9.2 OasisLMF ktools package built with the GNU compilers oasislmf/ktools/f92a41f/gnu-4.9.2 OasisLMF ktools package built with the GNU compilers octave/4.4.1 Octave is an open source competitor to Matlab which is mostly compatible with Matlab. octopus/4.1.2-impi/intel-2015-update2 Adds octopus 4.1.2 to your environment. octopus/4.1.2/intel-2015-update2 Adds octopus 4.1.2 to your environment. octopus/5.0.1-ompi/gnu-4.9.2 Adds octopus 5.0.1 to your environment. octopus/5.0.1/gnu-4.9.2 Adds octopus 5.0.1 to your environment. octopus/6.0-ompi/gnu-4.9.2 Adds octopus 6.0 to your environment. octopus/6.0/gnu-4.9.2 Adds octopus 6.0 to your environment. openbabel/2.4.1/gnu-4.9.2 OpenBabel is a library and command-line tool for manipulating and converting between various chemistry file formats. opencv/2.4.13/gnu-4.9.2 Adds OpenCV 2.4.13 to your environment. Open Source Computer Vision Library. opencv/3.4.1/gnu-4.9.2 Adds OpenCV 3.4.1 to your environment. Open Source Computer Vision Library. openfoam/2.3.1/intel-2015-update2 Adds OpenFOAM 2.3.1 to your environment openfoam/2.4.0/intel-2017-update1 Adds OpenFOAM 2.4.0 to your environment openfoam/7.20200120/gnu-7.3.0 Adds OpenFOAM 7 to your environment openfoamplus/v1706/gnu-4.9.2 Adds OpenFOAMplus v1706 to your environment openfoamplus/v1906/gnu-7.3.0 Adds OpenFOAMplus v1906 to your environment openfoamplus/v1906/gnu-7.3.0-64 Adds OpenFOAMplus v1906 with 64 bit labels to your environment openfoamplus/v2112/gnu-7.3.0-64 Adds OpenFOAMplus v2112 with 64 bit labels to your environment openmm/7.3.1/cuda-10 Adds OpenMM to your environment. openmm/7.3.1/gnu-4.9.2 Adds OpenMM to your environment. openmx/3.8.3 Adds OpenMX 3.8.3 to your environment. optimet/1.0.1/gnu-4.9.2 Adds Optimet to your environment. orca/4.2.1-bindist/gnu-4.9.2 ORCA is an ab initio, DFT, and semiempirical SCF-MO package. orca/5.0.4-sbindist ORCA is an ab initio, DFT, and semiempirical SCF-MO package. This installation is the official statically-linked binary distribution. p7zip/15.09/gnu-4.9.2 Adds p7zip 15.09 to your environment. To expand 7z files: 7za x archive.7z p7zip/16.02/gnu-4.9.2 p7zip is a port of the command-line version of the 7-zip file compression tool to UNIX-like systems. pandoc/1.19.2.1 Adds pandoc Version 1.19.2.1 to your environment. parallel/20181122 GNU parallel is a shell tool for executing jobs in parallel using one or more computers. paraview/5.3.0 This module adds the ParaView 5.3.0 binaries to your environment. ParaView is an open-source, multi-platform data analysis and visualization application. paraview/5.10.1 This module adds the ParaView 5.10.1 binaries to your environment. ParaView is an open-source, multi-platform data analysis and visualization application. parmed/3.2.0 Adds ParmEd to your environment. petsc/3.12.1/gnu-4.9.2 Adds Petsc 3.12.1 to your environment phon/1.39/gnu-4.9.2 Adds Phon 1.3.9 with addons to your PATH. phon/1.43/gnu-4.9.2 Adds Phon 1.43 with addons to your PATH. picard-tools/1.136 Adds Picard Tools 1.136 to your environment. If using the java -jar command, you should pass TMP_DIR=$TMPDIR to Picard. picard-tools/2.18.9 Adds Picard Tools to your environment. If using the java -jar command, you should pass TMP_DIR=$TMPDIR to Picard. platypus/3e72641 Adds Platypus to your environment. plink/1.07 Adds Plink 1.07 with addons to your PATH. plink/1.90b3.40 Adds PLINK 1.90b3.40 to your environment. A comprehensive update to the PLINK association analysis toolset. plink/2.0alpha-git Adds PLINK 2.0 alpha to your environment. A comprehensive update to the PLINK association analysis toolset. plumed/2.1.2/intel-2015-update2 Adds PLUMED 2.1.2 to your environment, built using OpenBLAS plumed/2.2.3/intel-2015-update2 Adds PLUMED 2.2.3 to your environment, built using OpenBLAS and libmatheval plumed/2.2/intel-2015-update2 Adds PLUMED 2.2 to your environment, built using OpenBLAS plumed/2.3.1/intel-2017-update1 Adds PLUMED 2.3.1 to your environment, built using OpenBLAS and libmatheval plumed/2.4.1/gnu-4.9.2 Adds PLUMED 2.4.1 to your environment, built using OpenBLAS and libmatheval plumed/2.4.1/intel-2017-update4 Adds PLUMED 2.4.1 to your environment, built using OpenBLAS and libmatheval plumed/2.4.3/intel-2018 Adds PLUMED 2.4.3 to your environment, built using MKL and libmatheval plumed/2.5.2/intel-2018 Adds PLUMED 2.5.2 to your environment, built using MKL and libmatheval plumed/2.6.0/intel-2018 Adds PLUMED 2.6.0 to your environment, built using MKL and libmatheval plumed/2.7.2/gnu-10.2.0 Adds PLUMED 2.7.2 to your environment, built using GCC, OpenBLAS and libmatheval. PLUMED is a plugin that works with a large number of molecular dynamics codes. plumed/2.7.2/intel-2020 Adds PLUMED 2.7.2 to your environment, built using MKL and libmatheval. PLUMED is a plugin that works with a large number of molecular dynamics codes. poppler/22.10.0/gnu-9.2.0 Adds Poppler 22.10.0 to your environment. postgres+postgis/9.5.3+2.2.2/gnu-4.9.2 Adds postgres+postgis 9.5.3+2.2.2 to your environment. PostgreSQL is a relational database, and PostGIS is a geographical information enhancement for PostgreSQL. postgresql/9.5.3/gnu-4.9.2 Adds postgresql 9.5.3 to your environment. PostgreSQL is a relational database. primer3/2.3.6 This module adds the primer3 package to your environment. probabel/0.4.5/gnu-4.9.2 Adds ProbABEL to your environment. proj.4/4.9.1 Adds the PROJ.4 Cartographic Projections library to your environment. proj.4/5.2.0 Adds the PROJ.4 Cartographic Projections library to your environment. proj.4/6.0.0 Adds the PROJ.4 Cartographic Projections library to your environment. proj.4/6.1.0 Adds the PROJ.4 Cartographic Projections library to your environment. proovread/2.13.11-8Jan2016-f6a856a Adds proovread 2.13.11-8Jan2016-f6a856a to your environment. f6a856a is the commit for this version. pymol/1.7.7.2 Adds PyMol to your environment. pymol/1.8.2.1 Adds PyMol to your environment. pyrosetta/release-73 Adds PyRosetta to your environment. pytorch/1.2.0/cpu Adds PyTorch 1.2.0 to your environment. pytorch/1.2.0/gpu Adds PyTorch 1.2.0 to your environment. pytorch/1.11.0/cpu Adds PyTorch 1.11.0 to your environment. pytorch/1.11.0/gpu Adds PyTorch 1.11.0 to your environment. qctool/2/beta/ba5eaa44a62f This module adds qctool v2 beta to your environment. quantum-espresso/5.2.0-impi/intel-2015-update2 Adds quantum-espresso 5.2.0 to your environment. quantum-espresso/6.1-impi/intel2017 Adds quantum-espresso 6.1 to your environment. quantum-espresso/6.3-impi/thermo_pw-1.0.9/intel-2018 Adds quantum-espresso 6.3 + thermo_pw 1.0.9 to your environment. quantum-espresso/6.4.1-impi/intel-2018 Adds quantum-espresso 6.4.1 to your environment. quantum-espresso/6.5-impi/intel-2018 Adds quantum-espresso 6.5 to your environment. quantum-espresso/6.5-impi/thermo_pw-1.2.1/intel-2018 Adds quantum-espresso 6.5 + thermo_pw 1.2.1 to your environment. quantum-espresso/7.0-impi/intel-2018 Adds quantum-espresso 7.0 to your environment. r/3.2.0-atlas/gnu-4.9.2 Adds R 3.2.0 and Bioconductor 3.2 to your environment. r/3.2.2-openblas/gnu-4.9.2 Adds R 3.2.2 and Bioconductor 3.2 to your environment. r/3.3.0-openblas/gnu-4.9.2 Adds R 3.3.0 and Bioconductor 3.3 to your environment. r/3.3.2-openblas/gnu-4.9.2 Adds R 3.3.2 and Bioconductor 3.4 to your environment. r/3.4.0-openblas/gnu-4.9.2 Adds R 3.4.0 and Bioconductor 3.5 to your environment. r/3.4.2-openblas/gnu-4.9.2 Adds R 3.4.2 and Bioconductor 3.6 to your environment. r/3.5.0-openblas/gnu-4.9.2 Adds R 3.5.0 and Bioconductor 3.7 to your environment. r/3.5.1-openblas/gnu-4.9.2 Adds R 3.5.1 and Bioconductor 3.7 to your environment. r/3.5.3-openblas/gnu-4.9.2 Adds R 3.5.3 and Bioconductor 3.8 to your environment. r/3.6.0-openblas/gnu-4.9.2 Adds R 3.6.0 and Bioconductor 3.9 to your environment. r/4.2.0-openblas/gnu-10.2.0 Adds R 4.2.0 and Bioconductor 3.15 to your environment. r/4.2.2-openblas/gnu-10.2.0 Adds R 4.2.2 and Bioconductor 3.16 to your environment. r/4.2.3-openblas/gnu-10.2.0 Adds R 4.2.3 and Bioconductor 3.16 to your environment. randfold/2.0/gnu-4.9.2 Adds randfold 2.0 to your environment. rclone/1.51.0 RClone is a command-line program intended to download and upload files from and to various storage services and providers. rclone/1.61.0 RClone is a command-line program intended to download and upload files from and to various storage services and providers. repast-hpc/2.1/gnu-4.9.2 Adds Repast HPC 2.1 compiled with GCC 4.9.2 and OpenMPI to your environment. root/5.34.30/gnu-4.9.2 Adds ROOT 5.34.30 to your environment. root/5.34.30/gnu-4.9.2-fftw-3.3.6 Adds ROOT 5.34.30 to your environment. root/5.34.36/gnu-4.9.2-fftw-3.3.6 Adds ROOT 5.34.36 to your environment. root/5.34.36/gnu-4.9.2-fftw-3.3.6-gsl-2.4 Adds ROOT 5.34.36 to your environment. root/6.04.00/gnu-4.9.2 Adds ROOT 6.04.00 to your environment. rosetta/2015.31.58019 Adds Rosetta 2015.31.58019 to your environment. rosetta/2015.31.58019-mpi Adds Rosetta 2015.31.58019 with MPI to your environment. rosetta/2018.48.60516 Adds Rosetta 2018.48.60516 serial version to your environment. rosetta/2018.48.60516-mpi Adds Rosetta 2018.48.60516 MPI version to your environment. rsem/1.2.31 Adds RSEM 1.2.31 to your environment. rvtests/2.1.0 Rvtests: Rare variant test software for next generation sequencing data sac/101.6a Adds SAC 101.6a to your environment. sac/102.0 Adds SAC 102.0 to your environment. salmon/1.9.0 Adds Salmon 1.9.0 to your environment. sambamba/0.6.7-bindist A tool for extracting information from SAM/BAM files. samblaster/0.1.24/gnu-4.9.2 samblaster is a program for marking duplicates in read-id grouped paired-end SAM files. samsrf/5.84/matlab.r2019b Adds the SamSrf Matlab toolbox to your environment samtools/0.1.19 This module adds the Samtools 0.1.19 package to your environment. samtools/1.2/gnu-4.9.2 Adds SAMtools 1.2 to your environment. Reading/writing/editing/indexing/viewing SAM/BAM/CRAM format. samtools/1.3.1/gnu-4.9.2 Adds SAMtools 1.3.1 to your environment. Reading/writing/editing/indexing/viewing SAM/BAM/CRAM format. samtools/1.9/gnu-4.9.2 Adds SAMtools 1.9 to your environment. Reading/writing/editing/indexing/viewing SAM/BAM/CRAM format. samtools/1.11/gnu-4.9.2 Tools for reading/writing/editing/indexing/viewing SAM/BAM/CRAM formatted data. sas/9.4-M6/64 Adds SAS 9.4 (9.04.01M6) 64 bit to your environment sas/9.4-m7/64 Adds SAS 9.4 (9.04.01M7) 64 bit to your environment sas/9.4/64 Adds SAS 9.4 64 bit to your environment sc/7.16 Adds sc 7.16 to your environment. siesta/4.0.1/intel-2017 Adds SIESTA 4.0.1 to your environment. siesta/5.0-dev/intel-2018 Adds SIESTA 5.0 Development Version to your environment. skewer/0.2.2 Adds skewer 0.2.2 to your environment. slim/4.0.1/gnu-4.9.2 This module adds SLiM 4.0.1 evolutionary simulation framework to your environment. smalt/0.7.6/gnu-4.9.2 Adds SMALT 0.7.6 to your environment. SMALT aligns DNA sequencing reads with a reference genome. Compiled with bambamc support for SAM/BAM input and BAM output. snpAD/0.3.4-bindist snpAD is an ancient DNA aware genotype caller. snptest/2.5.4-beta3 Adds SNPtest 2.5.4-beta3 to your environment. sod/3.2.7 Adds SOD 3.2.7 to your environment. SOD is a program that automates tedious data selection, downloading, and routine processing tasks in seismology. sod/3.2.10 Adds SOD 3.2.10 to your environment. SOD is a program that automates tedious data selection, downloading, and routine processing tasks in seismology. spaceranger/1.2.2 Adds Space Ranger 1.2.2 to your environment. Space Ranger is a set of analysis pipelines that process Visium spatial RNA-seq output and brightfield and fluorescence microscope images. spm/8/r6313/matlab.r2015a Adds SPM8 to your environment spm/12/jan2020/matlab.r2019b Adds SPM12 to your environment spm/12/r6470/matlab.r2015a Adds SPM12 to your environment spss/25 Adds SPSS 25 to your environment spss/26 Adds SPSS 26 to your environment sqlite/3.31.1/gnu-9.2.0 Adds SQLite Version 3.31.1 to your environment. sqlite/3.41.2 SQLite is a C-language library that implements a small, fast, self-contained, high-reliability, full-featured, SQL database engine. stacks/2.54/gnu-4.9.2 Stacks is a software pipeline for building loci from short-read sequences, such as those generated on the Illumina platform. Stacks was developed to work with restriction enzyme-based data, such as RAD-seq, for the purpose of building genetic maps and conducting population genomics and phylogeography. star-ccm+/9.06.011 Adds STAR-CCM+ and STAR-View to your environment. star-ccm+/11.04.010-R8 Adds STAR-CCM+ and STAR-View to your environment. star-ccm+/12.04.010 Adds STAR-CCM+ and STAR-View to your environment. star-ccm+/13.02.011 Adds STAR-CCM+ and STAR-View to your environment. star-ccm+/13.06.012 Adds STAR-CCM+ and STAR-View to your environment. star-ccm+/14.06.013 Adds STAR-CCM+ and STAR-View to your environment. star-cd/4.22.058 Adds STAR-CD 4.22.058 to your environment. star-cd/4.26.011 Adds STAR-CD 4.26.011 using Intel 2016 compiler suite to your environment. STAR-CD is a code for performing CFD simulations. It is designed for modelling fluid flow, heat transfer, mass transfer and chemical reactions. star-cd/4.26.022 Adds STAR-CD 4.26.022 using Intel 2016 compiler suite to your environment. STAR-CD is a code for performing CFD simulations. It is designed for modelling fluid flow, heat transfer, mass transfer and chemical reactions. star-cd/4.28.050 Adds STAR-CD 4.28.050 using Intel 2016 compiler suite to your environment. STAR-CD is a code for performing CFD simulations. It is designed for modelling fluid flow, heat transfer, mass transfer and chemical reactions. star/2.5.2a Adds STAR 2.5.2a to your environment. star/2.7.3a Adds STAR 2.7.3a to your environment. star/2.7.10b/sbindist The Spliced Transcripts Alignment to a Reference (STAR) package for RNA sequence alignment. This installation uses the statically-linked binaries by default. stata/14 Adds Stata/MP 14 to your environment. stata/15 Adds Stata/MP 15 to your environment. supermagic/1.2/intel-2017 Adds supermagic 1.2 to your environment. Supermagic is a simple MPI sanity test. taup/2.1.2 adds TauP 2.1.2 to your environment variables tensorflow/1.4.1/cpu Adds Tensorflow 1.4.1 to your environment. tensorflow/1.4.1/gpu Adds Tensorflow 1.4.1 to your environment. tensorflow/1.4.1/mkl Adds Tensorflow 1.4.1 to your environment. tensorflow/1.8.0/cpu Adds Tensorflow 1.8.0 to your environment. tensorflow/1.8.0/gpu Adds Tensorflow 1.8.0 to your environment. tensorflow/1.8.0/mkl Adds Tensorflow 1.8.0 to your environment. tensorflow/1.12.0/cpu Adds Tensorflow 1.12.0 to your environment. tensorflow/1.12.0/gpu Adds Tensorflow 1.12.0 to your environment. tensorflow/1.12.0/mkl Adds Tensorflow 1.12.0 to your environment. tensorflow/1.13.1/cpu Adds Tensorflow 1.13.1 to your environment. tensorflow/1.13.1/gpu Adds Tensorflow 1.13.1 to your environment. tensorflow/1.13.1/mkl Adds Tensorflow 1.13.1 to your environment. tensorflow/1.14.0/cpu Adds Tensorflow 1.14.0 to your environment. tensorflow/1.14.0/gpu Adds Tensorflow 1.14.0 to your environment. tensorflow/1.14.0/mkl Adds Tensorflow 1.14.0 to your environment. tensorflow/2.0.0/gpu-py37 Adds Tensorflow 2.0.0 to your environment. tensorflow/2.0.0/gpu-py37-cudnn75 Adds Tensorflow 2.0.0 to your environment. tensorflow/2.0.0/mkl-py37 Adds Tensorflow 2.0.0 to your environment. tensorflow/2.8.0/python-3.8.6 Adds Tensorflow 2.8.0 to your environment. tensorflow/2.11.0/cpu Adds Tensorflow 2.11.0 to your environment. tensorflow/2.11.0/gpu Adds Tensorflow 2.11.0 to your environment. tephra2/2.0/gnu-4.9.2 Adds Tephra2 version 2.0 to your environment. tephra2/normal/r149 Adds Tephra2 version r149 to your environment. tesseract/3.05.01 Adds Tesseract 3.05.01 to your environment. texinfo/5.2/gnu-4.9.2 Adds GNU texinfo 5.2 to your environment. texinfo/6.6/gnu-4.9.2 Adds GNU texinfo 6.6 to your environment. texlive/2014 Adds TeX Live 2014 to your environment. texlive/2015 Adds TeX Live 2015 to your environment. texlive/2019 Adds TeX Live 2019 to your environment. textract/1.5.0 Adds textract 1.5.0 to your environment. textract extracts text from a wide range of document types. tmux/2.2 This module adds the tmux 2.2 package to your environment. tmux/3.2a This module adds the tmux 3.2a package to your environment. tmux/3.3a This module adds the tmux 3.3a package to your environment. tophat/2.1.0 Adds Tophat 2.1.0 to your environment. tracer/1.6 Adds Tracer 1.6. tractor/3.2.5 Adds TractoR 3.2.5 to your environment. tree/1.7.0 Adds tree 1.7.0 to your environment. This shows your directory structure as a tree. trim_galore/0.4.1 Adds Trim Galore 0.4.1 to your environment. A wrapper tool around Cutadapt and FastQC to consistently apply quality and adapter trimming to FastQ files. trim_galore/0.6.10 Adds Trim Galore 0.6.10 to your environment. A wrapper tool around Cutadapt and FastQC to consistently apply quality and adapter trimming to FastQ files. trimmomatic/0.33 Adds Trimmomatic 0.33 to your environment. A flexible read trimming tool for Illumina NGS data. turbomole/6.4/mpi Adds turbomole 6.4 (using MPI) to your environment. turbomole/6.4/serial Adds turbomole 6.4 (serial) to your environment. turbomole/6.4/smp Adds turbomole 6.4 (using SMP) to your environment. turbomole/6.5/mpi Adds turbomole 6.5 (using MPI) to your environment. turbomole/6.5/serial Adds turbomole 6.5 (serial) to your environment. turbomole/6.5/smp Adds turbomole 6.5 (using SMP) to your environment. turbomole/6.6/mpi Adds turbomole 6.6 (using MPI) to your environment. turbomole/6.6/serial Adds turbomole 6.6 (serial) to your environment. turbomole/6.6/smp Adds turbomole 6.6 (using SMP) to your environment. ubpred/1-bin32dist UbPred is a random forest-based predictor of potential ubiquitination sites in proteins. udunits/2.2.19 Adds udunits to your environment. udunits/2.2.20/gnu-4.9.2 adds the UDUNITS-2 package to your environment. udunits/2.2.26/gnu-4.9.2 adds the UDUNITS-2 package to your environment. varscan/2.3.9 Adds VarScan v2.3.9 to your environment. VarScan is a platform-independent mutation caller for targeted, exome, and whole-genome resequencing data generated on Illumina, SOLiD, Life/PGM, Roche/454, and similar instruments. vasp/5.4.1-05feb16-p2/intel-2015-update2 The VASP Quantum Chemistry package, version 5.4.1-05feb16 with patches 1 and 2. vasp/5.4.1-24jun15-p2-vtst-r160/intel-2015-update2 Adds VASP 5.4.1 built with VTST r160 to your environment. vasp/5.4.1-24jun15-p08072015/intel-2015-update2 The VASP Quantum Chemistry package, version 5.4.1-24jun15 with patch 08072015. vasp/5.4.4-18apr2017-libbeef/intel-2017-update1 Adds VASP 5.4.4 with BEEF-vdW functionals to your environment. vasp/5.4.4-18apr2017-vtst-r178/intel-2017-update1 Adds VASP 5.4.4 built with VTST r178 to your environment. vasp/5.4.4-18apr2017/intel-2017-update1 Adds VASP 5.4.4 to your environment. vasp/5.4.4-18apr2017/intel-2019 Adds VASP 5.4.4 to your environment. Built with Intel 2019. vasp/6.1.2-01dec2020/intel-2019-update5 Adds VASP 6.1.2 to your environment. vasp/6.2.1-19Jan2022/intel-2019-update5 Adds VASP 6.2.1 to your environment. vasp/6.2.1-19Jan2022/nvidia-22.1-gpu Adds VASP 6.2.1 FOR GPUS ONLY to your environment. vasp/6.3.0-24Jan2022/intel-2019-update5 Adds VASP 6.3.0 to your environment. vasp/6.3.0-24Jan2022/nvidia-22.1-gpu Adds VASP 6.3.0 FOR GPUS ONLY to your environment. vcftools/0.1.15/gnu-4.9.2 Adds VCFtools version 0.1.15 to your environment. Tools for working with VCF files. velvet/1.2.10 Adds Velvet 1.2.10 to your environment. vep/95.0 Adds VEP 95.0 to your environment. vesta/3.4.6-bindist VESTA is a 3D visualization program for structural models, volumetric data such as electron/nuclear densities, and crystal morphologies. vg/1.11.0 This is a module with no description string. viennarna/2.1.9/gnu-4.9.2 Adds viennarna 2.1.9 to your environment. vinalc/1.1.2/gnu-4.9.2 Adds VinaLC to your environment. visit/2.9.2 This package adds VisIt 2.9.2 to your environment. VisIt is a distributed, parallel visualization and graphical analysis tool for data defined on two- and three-dimensional (2D and 3D) meshes. Visit will create a ~/.visit directory and a ~/Scratch/visit directory. Jobfiles created by the GUI will go in the latter. Legion hostfile: /shared/ucl/apps/visit/2.9.2/gnu-4.9.2/host_legion.xml Web: https://wci.llnl.gov/simulation/computer-codes/visit/ vmd/1.9.3/GL+CUDA VMD is a molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3-D graphics and built-in scripting. vmd/1.9.3/text-only The binary, text only version of VMD 1.9.3 vmd/1.9.4/a55-gl+cuda Adds VMD 1.9.4 Development Version a55 to your environment. vt/2018-08-01/gnu-4.9.2 [ref:f6d2b5dab73c] A tool set for short variant discovery in genetic sequence data. xmds/2.2.2 Adds XMDS 2.2.2 (GNU/ATLAS/Intel MPI/FFTW toolchain) to your environment. xmds/3.0.0 Adds XMDS 3.0.0 (GNU/ATLAS/Intel MPI/FFTW toolchain) to your environment. xtalopt/r12.1/gnu-4.9.2 Adds XtalOpt r12.1 to your environment. xulrunner/3.6.28/gnu-4.9.2 Adds XULRunner 3.6.28 to your environment. XULRunner is a Mozilla runtime package that can be used to bootstrap XUL+XPCOM applications. This version was built including javaxpcom. xulrunner/10.0.2 Adds the XULRunner 3.6.28 64-bit runtime binaries to your environment. XULRunner is a Mozilla runtime package that can be used to bootstrap XUL+XPCOM applications. yambo/4.1.4/intel-2017 This is a module with no description string."},{"location":"Installed_Software_Lists/module-packages/#libraries","title":"Libraries","text":"

Modules in this section set up your environment to use specific C, C++, or Fortran libraries. This can include being able to use them with other languages, like Python.

Module Description apr-util/1.5.4 adds APR-util 1.5.4 to your environment variables apr-util/1.6.1 adds APR-util 1.6.1 to your environment variables apr/1.5.2 adds APR 1.5.2 to your environment variables apr/1.7.0 adds APR 1.7.0 to your environment variables argtable/2.13 Adds argtable 2.13 to your environment. armadillo/7.400.3/intel-2015-update2 Adds armadillo 7.400.3 to your environment. Armadillo is a linear alebra library for C++, aiming to balance speed and ease of use. arpack-ng/3.4.0/intel-2015-update2 Adds arpack-ng 3.4.0 to your environment. ARPACK-NG is a collection of Fortran77 subroutines designed to solve large scale eigenvalue problems. arpack-ng/3.5.0/gnu-4.9.2-serial Adds arpack-ng 3.5.0 to your environment. ARPACK-NG is a collection of Fortran77 subroutines designed to solve large scale eigenvalue problems. arpack-ng/3.5.0/gnu-4.9.2-threaded Adds arpack-ng 3.5.0 to your environment. ARPACK-NG is a collection of Fortran77 subroutines designed to solve large scale eigenvalue problems. arpack-ng/3.5.0/intel-2017 Adds arpack-ng 3.5.0 to your environment. ARPACK-NG is a collection of Fortran77 subroutines designed to solve large scale eigenvalue problems. atlas/3.10.2/gnu-4.9.2 adds ATLAS 3.10.2 for GCC 4.9.2 compilers to your environment variables atlas/3.10.2/intel-2015-update2 adds ATLAS 3.10.2 for Intel 15 compilers to your environment variables bambamc/0.0.50/gnu-4.9.2 Adds bambamc 0.0.50 to your environment. bambamc is a lightweight C implementation of the read name collation code from the larger libmaus/biobambam C++ project. boost/1.75.0/gnu-4.9.2 Boost is a collection of miscellaneous C++ libraries. This build does not include Python bindings or MPI support, and is multi-threaded. boost/1_54_0/gnu-4.9.2 Adds Boost 1.54.0 with Python libraries to your environment. boost/1_54_0/mpi/gnu-4.9.2 Adds Boost 1.54.0 with Python and MPI libraries to your environment. boost/1_54_0/mpi/gnu-4.9.2-ompi-1.10.1 Adds Boost 1.54.0 with Python and MPI libraries to your environment. boost/1_54_0/mpi/intel-2015-update2 Adds Boost 1.54.0 with Python and MPI libraries to your environment. boost/1_63_0/gnu-4.9.2 Adds Boost 1.63.0 with Python libraries to your environment. boost/1_63_0/mpi/gnu-4.9.2 Adds Boost 1.63.0 with Python and MPI libraries to your environment. boost/1_63_0/mpi/intel-2017-update1 Adds Boost 1.63.0 with Python and Intel MPI libraries to your environment. cernlib/2006-35 Adds the CERN Program library to your environment. 2006-35 EL6 RPM binaries. cernlib/2006/gnu-4.9.2 Adds the CERN Program library to your environment cfitsio/3370/gnu-4.9.2 Adds cfitsio 3370 to your environment. cfitsio/3370/intel-2015-update2 Adds cfitsio 3370 to your environment. cgal/4.9/gnu-4.9.2 Adds CGAL 4.9 with Qt5 to your environment. The Computational Geometry Algorithms Library. clusteringsuite/2.6.6/bindist Adds clusteringsuite 2.6.6 to your environment. Clustering Suite is a set of tools to automatically expose the main performance trends in applications' computation structure. cppunit/1.15.1/gnu-4.9.2 Adds CppUnit 1.15.1 to your environment. CppUnit is a package for writing and running unit tests in C++. cppunit/1.15.1/gnu-10.2.0 Adds CppUnit 1.15.1 built wuth GNU 10.2.0 to your environment. CppUnit is a package for writing and running unit tests in C++. cunit/2.1-3/gnu-4.9.2 Adds cunit 2.1-3 to your environment. CUnit is a package for writing and running unit tests in C. cvmfs/2.2.1/gnu-4.9.2 Adds libcvmfs 2.2.1 to your environment. dyninst/9.3.2/gnu-4.9.2 Adds dyninst 9.3.2 to your environment. DynInst is a library for performing dynamic instrumentation of executables. eigen/3.2.5/gnu-4.9.2 adds Eigen for GCC 4.9.2 compilers to your environment variables eigen/3.3.9/gnu-4.9.2 Adds Eigen 3.3.9 to your environment. Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms. eigen/3.4.0/gnu-10.2.0 Adds Eigen 3.4.0 to your environment. Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms. elfutils/0.170/gnu-4.9.2 Adds elfutils 0.170 to your environment. Elfutils provides utilities for manipulating binary ELF files, and is one possible provider of libelf. fftw/2.1.5/gnu-4.9.2 adds FFTW 2.1.5 for GCC 4.9.2 compilers to your environment variables fftw/2.1.5/intel-2015-update2 adds FFTW 2.1.5 for Intel compilers to your environment variables fftw/3.3.4-impi/gnu-4.9.2 Adds fftw 3.3.4 (built with Intel MPI) to your environment. fftw/3.3.4-impi/intel-2017-update1 Adds fftw 3.3.4 (built with Intel MPI) to your environment. fftw/3.3.4-ompi-1.10.1/gnu-4.9.2 Adds fftw 3.3.4 (built with OpenMPI) to your environment. fftw/3.3.4-ompi/gnu-4.9.2 Adds fftw 3.3.4 (built with OpenMPI) to your environment. fftw/3.3.4-threads/gnu-4.9.2 adds FFTW 3.3.4 for GCC 4.9.2 compilers to your environment variables fftw/3.3.4/gnu-4.9.2 adds FFTW 3.3.4 for GCC 4.9.2 compilers to your environment variables fftw/3.3.4/intel-2015-update2 adds FFTW 3.3.4 for Intel compilers to your environment variables fftw/3.3.6-pl2/gnu-4.9.2 Adds FFTW 3.3.6 pl2 for GCC 4.9.2 compilers to your environment variables. Includes single and double precision only on Legion, plus long-double and quad on Grace/Thomas. Includes OpenMP and POSIX threads libraries. fftw/3.3.6-pl2/intel-2017 Adds FFTW 3.3.6 pl2 for Intel compilers to your environment variables. Includes single and double precision versions on Legion, plus long-double on Grace/Thomas. Includes OpenMP and POSIX threads libraries. fftw/3.3.8-impi/intel-2018 Adds fftw fftw (built with Intel MPI) to your environment. fftw/3.3.8-ompi/gnu-4.9.2 Adds fftw 3.3.8 (built with OpenMPI) to your environment. fftw/3.3.10/nvidia-22.1 Adds fftw 3.3.10 to your environment. forge/1.0.0/gnu-4.9.2 Adds forge 1.0.0 to your environment. freeimage/3.17.0/gnu-4.9.2 Adds FreeImage 3.17.0 to your environment. freetype/2.8.1/gnu-4.9.2 Adds freetype 2.8.1 to your environment. FreeType is a freely available software library to render fonts. ga/5.7-8BInts/intel-2018 Global Arrays (GA) is a library that provides a Partitioned Global Address Space (PGAS) programming model. This version has been compiled with 8-byte integers in the Fortran code. ga/5.7/intel-2018 Global Arrays (GA) is a library that provides a Partitioned Global Address Space (PGAS) programming model. gcc-libs/4.9.2 adds GCC 4.9.2 runtime to your evironment. geos/3.5.0/gnu-4.9.2 Adds geos 3.5.0 to your environment. GEOS (Geometry Engine, Open Source) is a library for performing various spatial operations, especially for boolean operations on GIS data. Note this version does not include the SWIG, Python, Ruby, or PHP bindings. gflags/2.2.1 Adds Google gflags 2.2.1 to your environment. giflib/5.1.1 Adds giflib 5.1.1 to your environment. A library and utilities for processing gifs. glbinding/2.1.2/gnu-4.9.2 Adds glbinding 2.1.2 to your environment. glew/1.13.0/gnu-4.9.2 Adds GLEW The OpenGL Extension Wrangler Library 1.11.0 to your environment. glew/2.1.0/gnu-4.9.2 Adds GLEW -- The OpenGL Extension Wrangler Library -- 2.1.0 to your environment. glfw/3.2.1/gnu-4.9.2 Adds GLFW 3.2.1 to your environment. glog/0.3.5 Adds Google glog 0.3.5 to your environment. glpk/4.60/gnu-4.9.2 Adds the GNU Linear Programming Kit Version 4.60 for GCC 4.9.2 to your environment. gsl/1.16/gnu-4.9.2 adds GSL 1.16 for GCC 4.9.2 to your environment. gsl/1.16/intel-2015-update2 Adds gsl 1.16 to your environment. gsl/2.4/gnu-4.9.2 adds GSL 2.4 for GCC 4.9.2 to your environment. gsl/2.4/intel-2017 adds GSL 2.4 for Intel 2017 to your environment. gstreamer/1.12.0 GStreamer is a library for constructing graphs of media-handling components, including codecs for various audio and video formats. gulp/4.5/libgulp/intel-2018 Adds GULP 4.5 library version to your environment. Built libgulp only, without FoX, for programs such as ChemShell to link. GULP is a materials simulation code. h5py/2.10.0-ompi/gnu-4.9.2 Adds h5py 2.10.0-ompi for Python 3.7 to your environment. harminv/1.4.1/gnu-4.9.2 Adds harminv 1.4.1 to your environment. harminv/1.4/gnu-4.9.2 Adds harminv 1.4 to your environment. hdf/5-1.8.15-p1-impi/intel-2015-update2 Adds hdf5 1.8.5-p1 (built with Fortran and IntelMPI options) to your environment. hdf/5-1.8.15-p1-ompi/gnu-4.9.2 Adds hdf5 1.8.5-p1 (built with Fortran and OpenMPI options) to your environment. hdf/5-1.8.15/gnu-4.9.2 adds HDF5 1.8.15 (Serial) for GCC 4.9.2 to your environment. hdf/5-1.8.15/intel-2015-update2 adds HDF5 1.8.15 (Serial) for Intel 2015 to your environment. hdf/5-1.10.2-impi/intel-2018 adds HDF5 1.10.2 (Parallel) for Intel 2018 to your environment. hdf/5-1.10.2/intel-2018 adds HDF5 1.10.2 (Serial) for Intel 2018 to your environment. hdf/5-1.10.5-ompi/gnu-4.9.2 Adds hdf 5-1.10.5-ompi to your environment. Built with OpenMPI and GNU. hdf/5-1.10.5/gnu-4.9.2 Adds hdf 5-1.10.5 to your environment. Serial version built with GNU. htslib/1.2.1 This module adds the HTSlib 1.2.1 package to your environment. HTSlib is an implementation of a unified C library for accessing common file formats, such as SAM, CRAM and VCF, used for high-throughput sequencing data. htslib/1.3.1 This module adds the HTSlib 1.3.1 package to your environment. HTSlib is an implementation of a unified C library for accessing common file formats, such as SAM, CRAM and VCF, used for high-throughput sequencing data. htslib/1.7 This module adds the HTSlib 1.7 package to your environment. HTSlib is an implementation of a unified C library for accessing common file formats, such as SAM, CRAM and VCF, used for high-throughput sequencing data. htslib/1.11/gnu-4.9.2 A C library for reading/writing high-throughput sequencing data. hwloc/1.11.12 The Portable Hardware Locality (hwloc) software package provides a portable abstraction (across OS, versions, architectures, ...) of the hierarchical topology of modern architectures, including NUMA memory nodes, sockets, shared caches, cores and simultaneous multithreading. This installation includes the optional libnuma dependency. hypre/2.11.2/openmpi-3.0.0/intel-2017 Adds HYPRE 2.11.2 to your environment. hypre/2.11.2/openmpi-3.1.1/intel-2018 Adds HYPRE 2.11.2 to your environment. ipopt/3.14.2/intel-2018 Ipopt (Interior Point OPTimizer, pronounced eye-pea-Opt) is a software package for large-scale nonlinear optimization. jansson/2.11 This is a module with no description string. json-c/0.12/gnu-4.9.2 Adds json-c 0.12 to your environment. JSON-C is a library for converting between JSON-formatted strings and C representations of the equivalent objects. lapack/3.8.0/gnu-4.9.2 LAPACK is a reference library of routines for Linear Algebra. It is not recommended for use, as its ABI is replicated in the much higher-performance libraries OpenBLAS, MKL, or ATLAS instead. leptonica/1.74.4 Adds Leptonica 1.74.4 to your environment. leveldb/1.20 Adds Google leveldb 1.20 to your environment. libbdwgc/7.4.2/gnu-4.9.2 Adds libbdwgc (a garbage-collector library) to your environment. libbeef/0.1.3/intel-2018 Library for Bayesian error estimation functionals for use in density functional theory codes: libbeef 0.1.3 commit 2822afe libctl/3.2.2/gnu.4.9.2 Adds libctl (built using Intel compilers) to your environment. libctl/4.3.0/gnu-4.9.2 Adds libctl 4.3.0 to your environment. libdwarf/20170709/gnu-4.9.2 Adds libdwarf 20170709 to your environment. libdwarf is a library for interacting with debugging info in the DWARF 2, 3, and 5 formats. libelf/0.8.13/gnu-4.9.2 Adds libelf 0.8.13 to your environment. libetsfio/1.0.4/gnu-4.9.2 Adds libetsfio 1.0.4 to your environment. libetsfio/1.0.4/intel-2015-update2 Adds libetsfio 1.0.4 to your environment. libflac/1.3.1/gnu-4.9.2 Adds libflac 1.3.1 to your environment. libFLAC is the Xiph library for handling their lossless audio codec. libgd/2.1.1/gnu-4.9.2 Adds libgd 2.1.1 to your environment. libgd/2.1.1/intel-2015-update2 Adds libgd 2.1.1 to your environment. libgdsii/0.21/gnu-4.9.2 Adds libgdsii 0.21 to your environment. C++ library and command-line utility for reading GDSII geometry files. libint/1.1.4/gnu-4.9.2 Adds libint 1.1.4 to your environment. Libint is required for CP2K. libmatheval/1.1.11 Adds libmatheval 1.1.11 to your environment. GNU libmatheval is a library (callable from C and Fortran) to parse and evaluate symbolic expressions input as text. libsodium/1.0.6/gnu-4.9.2 Adds libsodium 1.0.6 to your environment. libsodium is a crypto library primarily used by ZeroMQ. libsox/14.4.2/gnu-4.9.2 Adds libsox 14.4.2 to your environment. SoX is a library for reading, writing, and converting a variety of sound file formats. If you require support for a file format that is not installed, contact rc-support and the library can be rebuilt. libuuid/1.0.3/gnu-4.9.2 Adds a static libuuid 1.0.3 to your environment. libxc/2.1.2/intel-2015-update2 Adds libxc 2.1.2 to your environment. libxc/2.2.2/gnu-4.9.2 Adds libxc 2.2.2 to your environment. libxc/2.2.2/intel-2015-update2 Adds libxc 2.2.2 to your environment. libxc/3.0.0/gnu-4.9.2 Adds libxc 3.0.0 to your environment. libxc/3.0.0/intel-2015-update2 Adds libxc 3.0.0 to your environment. libxc/4.2.3/intel-2018 libxc is a library of routines implementing a range of exchange-correlation functionals for density-functional theory calculations. libxml2/2.9.4/gnu-4.9.2 Adds libxml2 2.9.4 to your environment. Libxml2 is an XML C parser and toolkit. Includes Python (2.7.9) bindings. llvm/3.3 This module adds the LLVM 3.3 package to your environment. The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. The LLVM Core libraries provide a modern source- and target-independent optimizer, along with code generation support for many popular CPUs. llvm/3.9.1 This module adds the LLVM 3.9.1 package to your environment. The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. The LLVM Core libraries provide a modern source- and target-independent optimizer, along with code generation support for many popular CPUs. llvm/6.0.1 This module adds the LLVM 6.0.1 package to your environment. The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. The LLVM Core libraries provide a modern source- and target-independent optimizer, along with code generation support for many popular CPUs. llvm/8.0.0/gnu-4.9.2 The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. This installation includes clang, a C compiler based on LLVM. lmdb/0.9.22 Adds LMDB 0.9.22 to your environment. lz4/1.8.3 This is a module with no description string. magma/2.4.0 This is a module with no description string. med/4.0.0/gnu-4.9.2 Adds med 4.0.0 to your environment. Allows reading and writing of MED format files. mesa/6.5/gnu-4.9.2 Adds mesa 6.5 to your environment. Mesa is an open-source implementation of the OpenGL specification - a system for rendering interactive 3D graphics. This is an old version installed to satisfy a particular dependency: please do not use for new builds. mesa/10.6.3 Adds Mesa 10.6.3 to your environment. Mesa is an open-source implementation of the OpenGL specification. (Built for offscreen rendering: OSMesa, Xlib GLX, no Gallium, no EGL, no llvm, no DRI). mesa/10.6.9/gnu-4.9.2 Adds Mesa 10.6.9 to your environment. Mesa is an open-source implementation of the OpenGL specification. (Built for offscreen rendering: OSMesa, Xlib GLX, no Gallium, no EGL, no llvm, no DRI). mesa/13.0.6/gnu-4.9.2 Adds Mesa 13.0.6 to your environment. Mesa is an open-source implementation of the OpenGL specification. (Built options: Gallium, LLVM, no EGL, no DRI, no GLX). The default driver is llvmpipe. You can use \"export GALLIUM_DRIVER\" to explicitly choose llvmpipe, softpipe, or swr metis/5.1.0/gnu-4.9.2 Adds metis 5.1.0 to your environment. METIS is a set of serial programs for partitioning graphs, partitioning finite element meshes, and producing fill reducing orderings for sparse matrices. metis/5.1.0/intel-2015-update2 Adds metis 5.1.0 to your environment. METIS is a set of serial programs for partitioning graphs, partitioning finite element meshes, and producing fill reducing orderings for sparse matrices. metis/5.1.0/intel-2018 Adds metis 5.1.0 to your environment. METIS is a set of serial programs for partitioning graphs, partitioning finite element meshes, and producing fill reducing orderings for sparse matrices. mpi/intel/2013/update1/intel adds Intel MPI 4.1.3.048 to your environment variables mpi/intel/2015/update3/gnu-4.9.2 adds Intel MPI to your environment variables mpi/intel/2015/update3/intel adds Intel MPI to your environment variables mpi/intel/2017/update1/gnu-4.9.2 adds Intel MPI to your environment variables mpi/intel/2017/update1/intel adds Intel MPI to your environment variables mpi/intel/2017/update2/gnu-4.9.2 adds Intel MPI to your environment variables configured to use GCC 4.9.2 mpi/intel/2017/update2/intel adds Intel MPI to your environment variables mpi/intel/2017/update3/gnu-4.9.2 [Intel MPI/2017.3.196] This is Intel's MPI implementation, version 2017.3.196, which is bundled with compiler package version 2017.Update4. This module sets up the compiler wrappers to use GCC 4.9.2 underneath. mpi/intel/2017/update3/intel [Intel MPI/2017.3.196] This is Intel's MPI implementation, version 2017.3.196, which is bundled with compiler package version 2017.Update4. mpi/intel/2018/update3/intel [Intel MPI/2018.3.222] This is Intel's MPI implementation, version 2018.3.222, which is bundled with compiler package version 2018.Update3. mpi/intel/2019/update4/intel [Intel MPI/2019.4.243] This is Intel's MPI implementation, version 2019.4.243, which is bundled with compiler package version 2019.Update4. mpi/intel/2019/update5/intel [Intel MPI/2019.5.281] This is Intel's MPI implementation, version 2019.5.281, which is bundled with compiler package version 2019.Update5. mpi/intel/2019/update6/intel [Intel MPI/2019.6.166] This is Intel's MPI implementation, version 2019.6.166, which is bundled with compiler package version 2020. mpi/intel/2021.6.0/intel [Intel MPI/2021.6.0] This is Intel's MPI implementation, version 2021.6.0, which is bundled with compiler package version 2022.2. mpi/openmpi/1.8.4/gnu-4.9.2 adds OpenMPI 1.8.4 for GCC 4.9.2 compilers to your environment variables mpi/openmpi/1.8.4/intel-2015-update2 adds OpenMPI 1.8.4 for Intel 2015 update 2 compilers to your environment variables mpi/openmpi/1.10.1/gnu-4.9.2 adds OpenMPI 1.10.1 for GCC 4.9.2 compilers to your environment variables mpi/openmpi/1.10.1/intel-2015-update2 adds OpenMPI 1.10.1 for Intel 2015 update 2 compilers to your environment variables mpi/openmpi/2.0.2/gnu-4.9.2 Adds openmpi 2.0.2 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/2.0.2/intel-2017 Adds openmpi 2.0.2 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/2.1.2/gnu-4.9.2 Adds openmpi 2.1.2 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/2.1.2/intel-2017 Adds openmpi 2.1.2 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.0.0/gnu-4.9.2 Adds openmpi 3.0.0 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.0.0/intel-2017 Adds openmpi 3.0.0 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.1.1/gnu-4.9.2 Adds openmpi 3.1.1 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.1.1/intel-2018 Adds openmpi 3.1.1 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.1.4/gnu-4.9.2 Adds openmpi 3.1.4 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.1.4/intel-2018 Adds openmpi 3.1.4 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.1.6/gnu-4.9.2 Adds openmpi 3.1.6 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/4.0.3/gnu-4.9.2 Adds openmpi 4.0.3 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/4.1.1/gnu-4.9.2 Adds openmpi 4.1.1 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. Note: we set OMPI_MCA_btl=vader on OmniPath clusters for this version. As a result it will work multi-node but be slower than other transports. mpi4py/2.0.0/python2 Adds Python2 mpi4py 2.0.0 to your environment. MPI for Python. mpi4py/2.0.0/python3 Adds Python3 mpi4py 2.0.0 to your environment. MPI for Python. mpi4py/3.0.0/python3 Adds Python3 mpi4py 3.0.0 to your environment. MPI for Python. mpi4py/3.0.2/gnu-4.9.2 Adds mpi4py 3.0.2 for Python 3.7 to your environment. mpi4py/3.1.4/gnu-4.9.2 Adds mpi4py 3.1.4 for Python 3.9 to your environment. multinest/3.10b/gnu-4.9.2 Adds MultiNest 3.10b to your environment. mumps-thirdparty/3.0.0/intel-2018 The COIN-OR Tools project ThirdParty-Mumps can be used to download the MUMPS code and build a MUMPS library that is recognized by Ipopt. This version of ThirdParty-Mumps retrieves and builds MUMPS 5.4.0. mumps/5.2.1/intel-2018 Adds mumps 5.2.1 to your environment. Sequential version built with Intel and METIS. mysql-connector-python/2.0.4/python-3.5.2 Adds mysql-connector-python 2.0.4 to your environment. This is Oracle's python-only MySQL connector for Python3 mysql-connector-python/2.0.4/python-3.6.3 Adds mysql-connector-python 2.0.4 to your environment. This is Oracle's python-only MySQL connector for Python3 mysql-connector-python/2.0.4/python-3.7.4 Adds mysql-connector-python 2.0.4 to your environment. This is Oracle's python-only MySQL connector for Python3 mysql-connector-python/2.0.4/python-3.8.0 Adds mysql-connector-python 2.0.4 to your environment. This is Oracle's python-only MySQL connector for Python3 mysql-connector-python/8.0.22/python-3.8.6 Adds mysql-connector-python 8.0.22 to your environment. This is Oracle's python-only MySQL connector for Python3 mysql-connector-python/8.0.22/python-3.9.0 Adds mysql-connector-python 8.0.22 to your environment. This is Oracle's python-only MySQL connector for Python3 mysql-connector-python/8.0.22/python-3.9.6 Adds mysql-connector-python 8.0.22 to your environment. This is Oracle's python-only MySQL connector for Python3 mysql-connector-python/8.0.28/python-3.9.10 Adds mysql-connector-python 8.0.28 to your environment. This is Oracle's python-only MySQL connector for Python3 nag/fortran/mark22/gnu-4.9.2 Adds NAG Fortran Library Mark 22 for GCC to your environment. nag/fortran/mark24/gnu-4.9.2 Adds NAG Fortran Library Mark 24 for GCC to your environment. nag/fortran/mark24/nag-6.0.1044 Adds NAG Fortran Library Mark 24 for NAG Fortran to your environment. nag/fortran/mark25/intel-2015-update2 Adds NAG Fortran Library Mark 25 for Intel 2015 to your environment. nag/fortran/mark26/gnu-4.9.2 Adds NAG Fortran Library Mark 26 for GCC to your environment. nag/fortran/mark26/intel-2017 Adds NAG Fortran Library Mark 26 for Intel 2017 to your environment. nag/fortran/mark26/nag-6.1.6106 Adds NAG Fortran Library Mark 26 for NAG Fortran to your environment. nag/fortran/mark26/nag-6.2.6223 Adds NAG Fortran Library Mark 26 for NAG Fortran to your environment. nag/mark27/intel-2019 Adds NAG Library Mark 27 for Intel 2019 to your environment. netcdf-c++/4.2/gnu-4.9.2 adds NetCDF C++ 4.2 for GCC to your environment. netcdf-c++/4.2/intel-2015-update2 adds NetCDF C++ 4.2 for Intel 2015 to your environment. netcdf-c++4/4.2/gnu-4.9.2 adds NetCDF C++ 4.2 for GCC to your environment. netcdf-c++4/4.2/intel-2015-update2 adds NetCDF C++ 4.2 for Intel 2015 to your environment. netcdf-fortran/4.4.1/gnu-4.9.2 adds NetCDF 4.4.1 for GCC to your environment. netcdf-fortran/4.4.1/intel-2015-update2 adds NetCDF 4.4.1 for Intel 2015 to your environment. netcdf-fortran/4.5.4/intel-2018-update3 adds NetCDF Fortran 4.5.4 for Intel 2018 to your environment. netcdf-fortran/4.6.1/gnu-10.2.0 adds NetCDF 4.6.1 for GCC to your environment. netcdf/4.3.3.1/gnu-4.9.2 adds NetCDF 4.3.3.1 for GCC 4.9.2 to your environment. netcdf/4.3.3.1/intel-2015-update2 adds NetCDF 4.3.3.1 for Intel 2015 to your environment. netcdf/4.9.0/intel-2018-update3 adds NetCDF 4.9.0 for Intel 2018 to your environment. netcdf/4.9.2/gnu-10.2.0 adds NetCDF 4.9.2 for GCC 10.2.0 to your environment. numactl/2.0.12 numactl provides NUMA policy support, as well as tools and a library to display NUMA allocation statistics and debugging information. openblas/0.2.14-threads/gnu-4.9.2 adds OpenBLAS 0.2.14 for GCC 4.9.2 compilers to your environment variables openblas/0.2.14/gnu-4.9.2 adds OpenBLAS 0.2.14 for GCC 4.9.2 compilers to your environment variables openblas/0.2.14/intel-2015-update2 adds OpenBLAS 0.2.14 for Intel 2015 update 2compilers to your environment variables openblas/0.3.2-native-threads/gnu-4.9.2 This is a module with no description string. openblas/0.3.2-openmp/gnu-4.9.2 This is a module with no description string. openblas/0.3.2-serial/gnu-4.9.2 This is a module with no description string. openblas/0.3.7-native-threads/gnu-4.9.2 This is a module with no description string. openblas/0.3.7-openmp/gnu-4.9.2 This is a module with no description string. openblas/0.3.7-serial/gnu-4.9.2 This is a module with no description string. openjpeg/2.4.0/gnu-4.9.2 OpenJPEG is an open-source JPEG 2000 codec written in C language. openslide/3.4.1/gnu-4.9.2 adds OpenSlide library 3.4.1 to your environment variables openssl/1.1.1t OpenSSL is a widely-used toolkit for general-purpose cryptography and secure communication. openssl/1.1.1u OpenSSL is a widely-used toolkit for general-purpose cryptography and secure communication. papi/5.5.1/gnu-4.9.2 Adds PAPI 5.5.1 to your environment. PAPI is a library for working with performance counters, often used in profiling applications. parmetis/4.0.3/intel-2015-update2 Adds parmetis 4.0.3 to your environment. ParMETIS is an MPI-based parallel library that implements a variety of algorithms for partitioning unstructured graphs, meshes, and for computing fill-reducing orderings of sparse matrices. pcre2/10.21/gnu-4.9.2 Adds pcre2 10.21 to your environment. PCRE (Perl-compatible regular expressions) is a C library implementing regular expression pattern-matching using the same semantics as Perl 5. pgplot/5.2.2/intel-2017 Adds PGPlot 5.2.2 to your environment. pgplot/5.2.2/intel-2018 Adds PGPlot 5.2.2 to your environment. pillow-simd/6.0.0.post0/python-3.7.4 Adds Pillow-SIMD to your environment. protobuf/3.5.1/gnu-4.9.2 adds Google Protocol Buffers for GCC 4.9.2 to your environment. protobuf/12-2017/gnu-4.9.2 adds Google Protocol Buffers for GCC 4.9.2 to your environment. psm2/11.2.185/gnu-4.9.2 Adds psm2 11.2.185 to your environment. The PSM2 messaging API, libpsm2. A low-level user-level communications interface for the Intel(R) OPA. family of products. pstreams/1.0.1/gnu-4.9.2 Adds pstreams 1.0.1 to your environment. PStreams is a C++ wrapper for process control and streaming using popen and pclose. pygsl/2.1.1-python3.6/gnu-4.9.2 Adds pygsl 2.1.1 to your environment. PyGSL provides Python bindings for the GNU Scientific Library. pyngl/1.4.0 Adds PyNGL to your environment. pynio/1.4.1 Adds PyNIO to your environment. quip/18c5440-threads/gnu-4.9.2 Adds QUIP to your environment. QUIP is required for CP2K. quip/18c5440/gnu-4.9.2 Adds QUIP to your environment. QUIP is required for CP2K. qutip/4.1.0/python-2.7.12 Adds qutip to your environment. scalapack/2.0.2/gnu-4.9.2/openblas Adds ScaLAPACK 2.0.2 to your environment, built with GCC, OpenBLAS and OpenMPI. Static and shared libraries. scalapack/2.0.2/gnu-4.9.2/openblas-0.3.2 Adds ScaLAPACK 2.0.2 to your environment, built with GCC, OpenBLAS and OpenMPI. Static and shared libraries. scalapack/2.0.2/gnu-4.9.2/openblas-0.3.7 Adds ScaLAPACK 2.0.2 to your environment, built with GCC, OpenBLAS and OpenMPI. Static and shared libraries. snappy/1.1.7 Adds Google snappy 1.1.7 to your environment. spark/3.1.1-bin-hadoop2.7 Apache Spark is an analytics engine for data processing. sparskit2/2009.11.18/gnu-4.9.2 Adds sparskit2 2009.11.18 to your environment. sparskit2/2009.11.18/intel-2015-update2 Adds sparskit2 2009.11.18 to your environment. spectral/3.4.0/bindist Adds spectral 3.4.0 to your environment. Spectral is a set of tools for performing spectral analysis on traces produced by the BSC profiling toolkit. spglib/1.7.4/gnu-4.9.2 Adds spglib 1.7.4 to your environment. Spglib is a library for finding and handling crystal symmetries written in C. squid/1.9g/gnu-4.9.2 Adds squid 1.9g to your environment. suitesparse/4.5.5/gnu-4.9.2-serial Adds suitesparse 4.5.5 to your environment. SuiteSparse is a suite of sparse matrix algorithms. suitesparse/4.5.5/gnu-4.9.2-threaded Adds suitesparse 4.5.5 to your environment. SuiteSparse is a suite of sparse matrix algorithms. suitesparse/4.5.5/intel-2017-update1 Adds suitesparse 4.5.5 to your environment. SuiteSparse is a suite of sparse matrix algorithms. superlu-dist/5.1.0/intel-2015-update2 Adds superlu-dist 5.1.0 to your environment. SuperLU_DIST is the distributed-memory parallel version of SuperLU, a general purpose library for the direct solution of large, sparse, nonsymmetric systems of linear equations. superlu/5.2.1/intel-2015-update2 Adds superlu 5.2.1 to your environment. SuperLU is a general purpose library for the direct solution of large, sparse, nonsymmetric systems of linear equations. szip/2.1 Adds szip to your environment. ucx/1.8.0/gnu-4.9.2 Adds ucx 1.8.0 to your environment. Unified Communication X (UCX) provides an optimized communication layer for Message Passing (MPI), PGAS/OpenSHMEM libraries and RPC/data-centric applications. ucx/1.9.0/gnu-4.9.2 Adds ucx 1.9.0 to your environment. Unified Communication X (UCX) provides an optimized communication layer for Message Passing (MPI), PGAS/OpenSHMEM libraries and RPC/data-centric applications. unixodbc/2.3.7 Unix ODBC driver vtk/5.10.1/gnu-4.9.2 adds VTK 5.10.1 for GCC 4.9.2 to your environment. vtk/6.2.0/gnu-4.9.2 adds VTK 6.2.0 for GCC 4.9.2 to your environment. wavpack/5.1.0/gnu-4.9.2 WavPack is a completely open audio compression format providing lossless, high-quality lossy, and a unique hybrid compression mode. webkitgtk/2.2.4-1 Adds the webkitgtk-1 with webkitgtk-devel library to your environment. 2.2.4-1 EL7 RPM binaries. webkitgtk/2.4.9-1 Adds the webkitgtk with webkitgtk-devel library to your environment. 2.4.9-1 EL7 RPM binaries. zeromq/4.1.4/gnu-4.9.2 Adds zeromq 4.1.4 to your environment. ZeroMQ is a distributed messaging library that supports many message-passing patterns and methods."},{"location":"Installed_Software_Lists/module-packages/#development-tools","title":"Development Tools","text":"

This section is for modules for programs that are used in software development, profiling, or troubleshooting.

It also contains language interpreters, like Python, Ruby, and Java.

Module Description armforge/20.1.2 ArmForge is a collection of parallel profiling and debugging tools for x86_64 and ARM processors. autoconf/2.69 Adds GNU Autoconf Version 2.69 to your environment. autogen/5.18.12/gnu-4.9.2 AutoGen is a tool designed to simplify the creation and maintenance of programs that contain large amounts of repetitious text. automake/1.16.1 Adds GNU Automake Version 1.16.1 to your environment. bazel/0.7.0 Adds bazek to your environment. bazel/0.14.1/gnu-4.9.2 Adds bazek to your environment. bazel/0.21.0/gnu-4.9.2 Adds bazek to your environment. bazel/0.24.0/gnu-4.9.2 Adds bazek to your environment. bazel/0.24.1/gnu-4.9.2 Adds bazek to your environment. bazel/0.26.1/gnu-4.9.2 Adds bazek to your environment. bazel/0.27.1/gnu-4.9.2 Adds bazek to your environment. bazel/4.2.1 Adds bazek to your environment. binutils/2.29.1/gnu-4.9.2 Adds binutils 2.29.1 to your environment. The GNU binutils are a collection of tools for working with binary files and assembling and disassembling machine instructions. bison/3.0.4/gnu-4.9.2 Adds Bison 3.0.4 to your environment. Bison is a general-purpose parser generator. chicken/4.13.0 adds Chicken 4.13.0 to your environment variables clojure/1.10.0.411 This is a module with no description string. clojure/1.10.3.1069 This is a module with no description string. cmake/3.2.1 adds Cmake 3.2.1 compilers to your environment variables cmake/3.7.2 adds Cmake 3.7.2 compilers to your environment variables cmake/3.13.3 adds Cmake 3.13.3 compilers to your environment variables cmake/3.19.1 adds Cmake 3.19.1 to your environment variables cmake/3.21.1 adds Cmake 3.21.1 to your environment variables cmake/3.27.3 CMake is an open-source, cross-platform family of tools designed to build, test and package software. cmdstan/2.24.1/gnu-4.9.2 Adds CmdStan 2.24.1 to your environment. cuda/7.5.18/gnu-4.9.2 Adds cuda 7.5.18 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/8.0.61-patch2/gnu-4.9.2 Adds cuda 8.0.61 patch2 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/9.0.176-patch4/gnu-4.9.2 Adds cuda 9.0.176 patch4 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/10.0.130/gnu-4.9.2 Adds cuda 10.0.130 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/10.1.243/gnu-4.9.2 Adds cuda 10.1.243 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cudnn/5.1/cuda-7.5 Adds cuDNN to your environment. cudnn/5.1/cuda-8.0 Adds cuDNN to your environment. cudnn/6.0/cuda-7.5 Adds cuDNN to your environment. cudnn/6.0/cuda-8.0 Adds cuDNN to your environment. cudnn/7.0.4/cuda-8.0 Adds cuDNN to your environment. cudnn/7.1.4/cuda-9.0 Adds cuDNN to your environment. cudnn/7.4.2.24/cuda-9.0 Adds cuDNN to your environment. cudnn/7.4.2.24/cuda-10.0 Adds cuDNN to your environment. cudnn/7.5.0.56/cuda-10.0 Adds cuDNN to your environment. cudnn/7.5.0.56/cuda-10.1 Adds cuDNN to your environment. cudnn/7.6.5.32/cuda-10.0 Adds cuDNN to your environment. cudnn/7.6.5.32/cuda-10.1 Adds cuDNN to your environment. ddt/6.0.4 This module adds DDT 6.0.4 with MIC support to your environment. depot_tools/788d9e0d adds depot_tools to your environment variables depot_tools/c03a9cf adds depot_tools to your environment variables dimemas/5.3.3/bindist Adds dimemas 5.3.3 to your environment. Dimemas is an abstracted network simulator for message-passing programs. dotnet-sdk/7.0.203 Microsoft .NET SDK doxygen/1.8.14 This is a module with no description string. emacs/24.5 Adds Emacs 24.5 to your environment. An extensible text editor. emacs/26.3 Adds Emacs 26.3 to your environment. An extensible text editor. emacs/28.1 Adds Emacs 28.1 to your environment. An extensible text editor. extrae/3.5.2/intel-2017 Adds extrae 3.5.2 to your environment. Extrae is an instrumentation framework to generate execution traces of the most used parallel runtimes. f2c/2013-09-26/gnu-4.9.2 Adds f2c 2013-09-26 to your environment. f2c is a source-to-source translator from Fortran 77 to C. It is not standards-compliant and is not recommended for use under any circumstances. flex/2.5.39 adds Flex 2.4.39 to your environment variables git/2.3.5 adds Git 2.3.5 to your environment variables git/2.10.2 adds Git 2.10.2 to your environment variables git/2.19.1 adds Git 2.19.1 to your environment variables git/2.32.0 adds Git 2.32.0 to your environment variables git/2.41.0-lfs-3.3.0 Git is a free and open source distributed version control system. This installation includes git-lfs pre-configured. gperf/3.0.4/gnu-4.9.2 Adds gperf 3.0.4 to your environment. GNU gperf is a perfect hash function generator. guile/2.0.11/gnu-4.9.2 Adds guile 2.0.11 to your environment. haskellplatform/2014.2.0.0 adds Haskell Platform to your environment variables htop/1.0.3/gnu-4.9.2 Adds htop 1.0.3 to your environment. htop/3.2.2 htop is a cross-platform interactive process viewer. java/1.8.0_45 adds Oracle JDK 1.8.0_45 compilers to your environment variables java/1.8.0_92 adds Oracle JDK 1.8.0_92 compilers to your environment variables java/openjdk-8/8u212/hotspot adds Oracle JDK 8 compilers to your environment variables java/openjdk-8/8u212/openj9 adds Oracle JDK 8 compilers to your environment variables java/openjdk-11/11.0.1 adds Oracle JDK 11.0.1 compilers to your environment variables java/openjdk-11/11.0.3u7/hotspot adds Oracle JDK 11.0.3 compilers to your environment variables java/openjdk-11/11.0.3u7/openj9 adds Oracle JDK 11.0.3 compilers to your environment variables java/semeru-8/8u322-b06 adds IBM Semeru 8u322-b06 compilers to your environment variables java/semeru-11/11.0.14.1+1 adds IBM Semeru 11.0.14.1+1 compilers to your environment variables java/semeru-17/17.0.2+8 adds IBM Semeru 17.0.2+8 compilers to your environment variables java/temurin-8/8u322-b06 adds Adoptium Temurin 8u322-b06 compilers to your environment variables java/temurin-11/11.0.14+9 adds Adoptium Temurin 11.0.14+9 compilers to your environment variables java/temurin-11/11.0.14.1+1 adds Adoptium Temurin 11.0.14.1+1 compilers to your environment variables java/temurin-17/17.0.2+8 adds Adoptium Temurin 17.0.2+8 compilers to your environment variables julia/0.3.10 adds Julia 0.3.10 to your environment variables julia/0.4.0 adds Julia 0.4.0 to your environment variables julia/0.4.7 adds Julia 0.4.7 to your environment variables julia/0.5.0 adds Julia 0.5.0 to your environment variables julia/0.6.0 adds Julia 0.6.0 to your environment variables julia/0.7.0 adds Julia 0.7.0 to your environment variables julia/1.0.0 adds Julia 1.0.0 to your environment variables julia/1.1.0 adds Julia 1.1.0 to your environment variables julia/1.2.0 adds Julia 1.2.0 to your environment variables julia/1.3.1 adds Julia 1.3.1 to your environment variables julia/1.5.0 adds Julia 1.5.0 to your environment variables julia/1.6.0 adds Julia 1.6.0 to your environment variables julia/1.6.2 adds Julia 1.6.2 to your environment variables julia/1.7.0 adds Julia 1.7.0 to your environment variables julia/1.7.1 adds Julia 1.7.1 to your environment variables julia/1.8.5 adds Julia 1.8.5 to your environment variables julia/1.9.0 adds Julia 1.9.0 to your environment variables julia/1.9.1 adds Julia 1.9.1 to your environment variables julia/1.9.2 adds Julia 1.9.2 to your environment variables julia/1.9.3 adds Julia 1.9.3 to your environment variables libtool/2.4.6 Adds libtool 2.4.6 to your environment. GNU libtool is a generic library support script. Libtool hides the complexity of using shared libraries behind a consistent, portable interface. libxkbcommon/1.5.0 Adds xkbcommon to your environment. ltrace/0.7.3/gnu-4.9.2 Adds ltrace 0.7.3 to your environment. lua/5.3.1 This module adds the Lua 5.3.1 package to your environment. Lua is a powerful, fast, lightweight, embeddable scripting language. mc/4.8.14 This module adds Midnight Commander 4.8.14 to your environment. mono/3.12.1 adds Mono 3.12.1 compilers to your environment variables mono/5.20.1.27/gnu-4.9.2 This is a module with no description string. nano/2.4.2 Adds nano 2.4.2 to your environment. A simple text editor. nano/4.9 The nano text editor. nano/6.1 The nano text editor. nasm/2.13.01 The Netwide Assembler, NASM, is an 80x86 and x86-64 assembler. ncl/6.0.0 adds NCL 6.0.0 to your environment variables ncl/6.3.0 adds NCL 6.3.0 to your environment variables nedit/5.6-aug15 Adds the NEdit GUI text editor to your environment. netlogo/6.1.0 adds NetLogo tooklit compilers to your environment variables ninja/1.11.1 Adds Ninja Version 1.11.1 to your environment. paraver/4.6.4.rc1/bindist Adds paraver 4.6.4.rc1 to your environment. Paraver is a trace visualizer for post-mortem trace analysis. perl/5.16.0 This module adds adds Perl 5.16.0 to your environment. perl/5.22.0 This module adds adds Perl 5.22.0 to your environment. perlbrew/0.73 This module adds the Perlbrew 0.73 package to your environment. Use Perlbrew to manage your own Perls and Perl modules pigz/2.4 pigz is a fully functional replacement for gzip that exploits multiple processors and multiple cores when compressing data. pkg-config/0.29.2 Adds pkg-config 0.29.2 to your environment variables. pycuda/2017.1/python2 Adds Python2 PyCuda to your environment. MPI for Python. pycuda/2017.1/python3 Adds Python3 PyCuda to your environment. MPI for Python. pypy3/6.0.0/gnu-4.9.2 Pypy is a JIT-ing interpreter for the Python language. This is the version intended to be compatible with CPython 3.5. python/2.7.9 adds Python 2.7.9 with pip and virtualenv to your environment variables python/2.7.12 adds Python 2.7.12 with pip and virtualenv to your environment variables python/3.4.3 adds Python 3.4.3 with pip and virtualenv to your environment variables python/3.5.2 adds Python 3.5.2 with pip and virtualenv to your environment variables python/3.6.1/gnu-4.9.2 Adds Python 3.6.1 with pip and virtualenv to your environment variables. python/3.6.3 Adds Python 3.6.3 with pip and virtualenv to your environment variables. python/3.7.0 Adds Python 3.7.0 with pip and virtualenv to your environment variables. python/3.7.2 Adds Python 3.7.2 with pip and virtualenv to your environment variables. python/3.7.4 Adds Python 3.7.4 with pip and virtualenv to your environment variables. python/3.8.0 Adds Python 3.8.0 with pip and virtualenv to your environment variables. python/3.8.6 Adds Python 3.8.6 with pip and virtualenv to your environment variables. python/3.9.0 Adds Python 3.9.0 with pip and virtualenv to your environment variables. python/3.9.1 Adds Python 3.9.1 with pip and virtualenv to your environment variables. python/3.9.6 Adds Python 3.9.6 with pip and virtualenv to your environment variables. python/3.9.10 Adds Python 3.9.10 with pip and virtualenv to your environment variables. python/3.11.3 Adds Python 3.11.3 with pip and virtualenv to your environment variables. python/3.11.4 Adds Python 3.11.4 with pip and virtualenv to your environment variables. python/idp3/2019/3.6.8 Adds Intel Distribution for Python to your environment variables. python/miniconda3/4.5.11 Adds Miniconda 4.5.11 to your environment variables. python/miniconda3/4.10.3 Adds Miniconda 4.10.3 to your environment variables. qt/4.8.6/gnu-4.9.2 Adds Qt 4.8.6 to your environment. Qt is a cross-platform development tool. qt/5.4.2/gnu-4.9.2 Adds Qt 5.4.2 to your environment. Qt is a cross-platform development tool. qt/5.12.1/gnu-4.9.2 Adds Qt 5.12.1 to your environment. Qt is a cross-platform development tool. qwt/6.1.4/gnu-4.9.2 Adds Qwt 6.1.4 to your environment. racket/6.8 Adds Racket 6.8 to your enviroment. rappture/20130903 Adds the Rappture toolkit to your environment. rstudio-ide/1.4.1717 RStudio is an integrated development environment (IDE) for R ruby/2.2.2 Ruby 2.2.2 with RubyGems 2.4.8 and libffi 3.2.1 ruse/1.0.1 A command-line utility to periodically measure the memory use of a process and its subprocesses. ruse/2.0 Ruse is a command-line utility that periodically measures the resource use of a process and its subprocesses. sbcl/1.3.19 Adds Steelbank Common LISP 1.3.19 to your environment. sbcl/2.1.6 Adds Steelbank Common LISP 2.1.6 to your environment. sbt/1.6.1 This is a module with no description string. scala/2.13.8 This is a module with no description string. scala/3.1.1 This is a module with no description string. scons/2.3.4 adds scons 2.3.4 to your environment variables shellcheck/0.8.0 Shellcheck is a tool for performing static analysis of shell scripts. strace/4.12 Adds strace 4.12 to your environment. Trace system calls and signals. subversion/1.8.13 adds Subversion 1.8.13 to your environment variables subversion/1.14.1 adds Subversion 1.14.1 to your environment variables swig/3.0.5/gnu-4.9.2 This module adds the SWIG 3.0.5 package to your environment. SWIG is an interface compiler that connects programs written in C and C++ with scripting languages such as Perl, Python, Ruby, and Tcl. swig/3.0.7/gnu-4.9.2 This module adds the SWIG 3.0.7 package to your environment. SWIG is an interface compiler that connects programs written in C and C++ with scripting languages such as Perl, Python, Ruby, and Tcl. tcl/8.6.8 This is a modulefile for Tcl/Tk 8.6.8 v8/3.15 adds v8 to your environment variables v8/5.6 adds v8 to your environment variables valgrind/3.11.0/gnu-4.9.2 Adds valgrind 3.11.0 to your environment. Valgrind is a framework for building dynamic analysis tools. It includes the memgrind and cachegrind tools. xbae/4.60.4 Adds the Xbae Matrix Widget to your environment. xcb-util/0.4.0 Adds xcb-util packages to your environment. xorg-utils/X11R7.7 Adds xorg-utils from X11R7.7 to your environment. Includes util-macros-1.17, makedepend-1.0.5 libXdmcp-1.1.1 and libXScrnSaver-1.2.2 and imake-1.0.7."},{"location":"Installed_Software_Lists/module-packages/#core-modules","title":"Core Modules","text":"

These modules refer to groups of system tools, rather than applications. They're intended to help you use the system, and some are loaded by default.

Module Description gerun adds gerun wrapper to your environment variables lm-utils/1.0 adds utilities to check license manager status to your environment. mrxvt/0.5.4 Adds Mrxvt a multi-tabbed xterm replacement to your environment. ops-tools/1.0.0 Tools for Ops work ops-tools/1.1.0 Tools for Ops work ops-tools/2.0.0 Tools for Ops work pipe-gifts/1.0.0 A tool for transferring files between users on the same node: 'pipe-give' and 'pipe-receive'. pv/1.6.6 Pipe Viewer (pv) is a tool for monitoring the progress of data through a pipeline. rcps-core/1.0.0 adds a core set of applications and libraries to your environment. rlwrap/0.43 adds rlwrap 0.43 to your environment variables screen/4.2.1 adds Screen 4.2.1 to your environment variables screen/4.8.0-ucl1 adds Screen 4.8.0 to your environment variables screen/4.9.0 adds Screen 4.9.0 to your environment variables singularity-env/1.0.0 Sets up the environment and cache directories for the Singularity container runtime. userscripts/1.0.0 Adds userscripts dir to your path. Provides jobhist among other utilities. userscripts/1.1.0 Adds userscripts dir to your path. Provides jobhist among other utilities. userscripts/1.2.0 Adds userscripts dir to your path. Provides jobhist among other utilities. userscripts/1.3.0 Adds userscripts dir to your path. Provides jobhist among other utilities. userscripts/1.4.0 Adds userscripts dir to your path. Provides jobhist among other utilities. GATK Some GATK tools require conda and associated libraries."},{"location":"Installed_Software_Lists/module-packages/#beta-modules","title":"Beta Modules","text":"

This section is for modules we're still trying out. They may or may not work with applications from other sections.

Module Description amber/20/mpi/gnu-10.2.0 Adds AMBER 20 and AmberTools 21 to your environment amber/20/openmp/gnu-10.2.0 Adds AMBER 20 and AmberTools 21 to your environment. OpenMP threaded version. amber/20/serial/gnu-10.2.0 Adds AMBER 20 and AmberTools 21 to your environment armadillo/10.4.0/gnu-10.2.0 Adds Armadillo Version 10.4.0 C++ library to your environment. arpack-ng/3.8.0-threaded/gnu-10.2.0 Adds arpack-ng 3.8.0 to your environment. ARPACK-NG is a collection of Fortran77 subroutines designed to solve large scale eigenvalue problems. atlas-adna/0.9_2021-04-16/gnu-10.2.0 Adds ATLAS Ancient DNA Analysis package to your environment. atlas-adna/0.9_2021-08-13/gnu-10.2.0 Adds ATLAS Ancient DNA Analysis package to your environment. bedtools/2.30.0/gnu-10.2.0 Adds bedtools 2.30.0 to your environment. The bedtools utilities are a swiss-army knife of tools for a wide-range of genomics analysis tasks. binutils/2.36.1/gnu-10.2.0 Adds binutils 2.36.1 to your environment. The GNU binutils are a collection of tools for working with binary files and assembling and disassembling machine instructions. compilers/chapel/1.26.0 Chapel 1.26.0 for GCC compilers/gnu/7.3.0 The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages (libstdc++,...). compilers/gnu/8.3.0 The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages (libstdc++,...). compilers/gnu/9.2.0 The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages (libstdc++,...). compilers/gnu/10.2.0 The GNU Compiler Collection includes front ends for C, C++, Objective-C, and Fortran, as well as libraries for these languages (libstdc++,...). cp2k/8.2/ompi/gnu-10.2.0 Adds CP2K to your environment. cuda/10.1.243/gnu-7.3.0 Adds cuda 10.1.243 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/11.1.1/gnu-10.2.0 Adds cuda 11.1.1 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/11.2.0/gnu-10.2.0 Adds cuda 11.2.0 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/11.3.1/gnu-10.2.0 Adds cuda 11.3.1 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/11.8.0/gnu-10.2.0 Adds cuda 11.8.0 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cudnn/8.1.0.77/cuda-11.2 Adds cuDNN 8.1.0 to your environment. cudnn/8.2.1.32/cuda-11.3 Adds cuDNN 8.2.1 to your environment. fftw/3.3.8/gnu-7.3.0 Adds FFTW 3.3.8 for GCC 7.3.0 compilers to your environment variables. Includes single and double precision, plus long-double and quad. Includes OpenMP and POSIX threads libraries. fftw/3.3.8/gnu-9.2.0 Adds FFTW 3.3.8 for GCC 9.2.0 compilers to your environment variables. Includes single and double precision, plus long-double and quad. Includes OpenMP and POSIX threads libraries. fftw/3.3.9/gnu-10.2.0 Adds FFTW 3.3.9 for GCC 10.2.0 compilers to your environment variables. Includes single and double precision, plus long-double and quad. Includes OpenMP and POSIX threads libraries. gcc-libs/7.3.0 Base module for gcc 7.3.0 -- does not set the standard compiler environment variables. The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages (libstdc++,...). gcc-libs/8.3.0 Base module for gcc 8.3.0 -- does not set the standard compiler environment variables. The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages (libstdc++,...). gcc-libs/9.2.0 Base module for gcc 9.2.0 -- does not set the standard compiler environment variables. The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages (libstdc++,...). gcc-libs/10.2.0 Base module for gcc 10.2.0 -- does not set the standard compiler environment variables. The GNU Compiler Collection includes front ends for C, C++, Objective-C, and Fortran, as well as libraries for these languages (libstdc++,...). Patch 95889 for __has_include applied. gdal/3.0.4/gnu-9.2.0 adds GDAL 3.0.4 with PROJ.4 7.0.0 to your environment variables. gdal/3.1.3/gnu-9.2.0 adds GDAL 3.1.3 with PROJ.4 7.0.0 to your environment variables. gdal/3.3.2/gnu-10.2.0 adds GDAL 3.3.2 with PROJ.4 8.1.1 to your environment variables. geos/3.8.1/gnu-9.2.0 Adds geos 3.8.1 to your environment. GEOS (Geometry Engine, Open Source) is a library for performing various spatial operations, especially for boolean operations on GIS data. Note this version does not include the SWIG, Python, Ruby, or PHP bindings. geos/3.9.1/gnu-10.2.0 Adds geos 3.9.1 to your environment. GEOS (Geometry Engine, Open Source) is a library for performing various spatial operations, especially for boolean operations on GIS data. Note this version does not include the SWIG, Python, Ruby, or PHP bindings. gmt/6.0.0/gnu-9.2.0 adds GMT 6.0.0 to your environment variables gmt/6.2.0/gnu-10.2.0 adds GMT 6.2.0 to your environment variables grass/8.0-dev/gnu-10.2.0 Adds GRASS GIS 8.0 Development Version to your environment. grass/8.0dev adds UCL set of modules for GRASS GIS 8.0 Dev to your environment gromacs/2020.1/cuda-10.1 Adds gromacs 2020.1 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2020.1/intel-2020 Adds gromacs 2020.1 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2020.4/cuda-10.1 Adds gromacs 2020.4 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2020.4/intel-2020 Adds gromacs 2020.4 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2021.2/gnu-7.3.0 Adds gromacs 2021.2 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2021.5/cuda-11.3 Adds gromacs 2021.5 built with CUDA 11.3 to your environment. ThreadMPI and OpenMPI versions included. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gsl/2.6/gnu-9.2.0 adds GSL 2.6 for GCC 9.2.0 to your environment. gsl/2.7/gnu-10.2.0 adds GSL 2.7 for GCC 10.2.0 to your environment. hdf/5-1.10.5/gnu-9.2.0 Adds hdf 5-1.10.5 to your environment. Serial version built with GNU. hdf/5-1.10.6/gnu-10.2.0 Adds hdf 5-1.10.6 to your environment. Serial version built with GNU 10.2.0. jags/4.3.0/gnu-10.2.0-openblas Adds JAGS 4.3.0 to your environment. jags/4.3.1/gnu-10.2.0-openblas Adds JAGS 4.3.1 to your environment. jags/4.3.2/gnu-10.2.0-openblas Adds JAGS 4.3.2 to your environment. jagurs/gnu-10.2.0-ompi adds JAGURS V0516 MPI version to your environment variables jagurs/mpi/recommended Adds set of default modules to the environment for JAGURS users. lammps/2aug23/basic-fftw/gnu-10.2.0 Adds LAMMPS 2 August 2023 to your environment. Binary is lmp_mpi. lammps/28mar23/basic-fftw/gnu-10.2.0 Adds LAMMPS 28 March 2023 to your environment. Binary is lmp_mpi. lammps/29sep21up2/basic-fftw/gnu-10.2.0 Adds LAMMPS 29Sep2021 Update 2 to your environment. Binary is lmp_mpi. lammps/29sep21up2/basic/gnu-10.2.0 Adds LAMMPS 29Sep2021 Update 2 to your environment. Binary is lmp. lammps/29sep21up2/basic/gnu-10.2.0-aristotle Adds LAMMPS 29Sep2021 Update 2 to your environment. Binary is lmp_aristotle. lammps/29sep21up2/basic/intel-2020 Adds LAMMPS 29Sep2021 Update 2 to your environment. Binary is lmp_mpi or lmp_default. lammps/29sep21up2/gpu/gnu-10.2.0 Adds LAMMPS 29Sep2021 Update 2 to your environment. Binary is lmp. lammps/29sep21up2/gpu/intel-2020 Adds LAMMPS 29Sep2021 Update 2 with gpu package to your environment. Binary is lmp_gpu or lmp_default. lammps/29sep21up2/userintel/intel-2020 Adds LAMMPS 29Sep2021 Update 2 with INTEL package to your environment. Binary is lmp_mpi or lmp_default. libpng/1.6.37/gnu-9.2.0 Adds libpng 1.6.37 to your environment. matlab/full/r2018a/9.4-prefdir-fix Adds Matlab R2018a to your environment. med/4.0.0/gnu-9.2.0 Adds med 4.0.0 to your environment. Allows reading and writing of MED format files. mpi/openmpi/3.1.4/gnu-7.3.0 Adds openmpi 3.1.4 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.1.5/gnu-9.2.0 Adds openmpi 3.1.5 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/4.0.5/gnu-10.2.0 Adds openmpi 4.0.5 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mumps/5.2.1/gnu-9.2.0 Adds mumps 5.2.1 to your environment. Sequential (threaded) version built with GNU, OpenBLAS and METIS. namd/2.13/intel-2018-update3/testing Adds NAMD 2.13 to your environment namd/2.13/plumed/intel-2018-update3/testing Adds NAMD 2.13 to your environment netcdf/4.7.4/gnu-9.2.0 adds NetCDF 4.7.4 for GCC 9.2.0 to your environment. netcdf/4.8.1/gnu-10.2.0 adds NetCDF 4.8.1 for GCC 10.2.0 to your environment. openblas/0.3.7-native-threads/gnu-9.2.0 OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version. openblas/0.3.7-openmp/gnu-9.2.0 OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version. openblas/0.3.7-serial/gnu-9.2.0 OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version. openblas/0.3.13-native-threads/gnu-10.2.0 OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version. openblas/0.3.13-openmp/gnu-10.2.0 OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version. openblas/0.3.13-serial/gnu-10.2.0 OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version. pcre2/10.35/gnu-9.2.0 Adds pcre2 10.35 to your environment. PCRE (Perl-compatible regular expressions) is a C library implementing regular expression pattern-matching using the same semantics as Perl 5. pcre2/10.37/gnu-10.2.0 Adds pcre2 10.37 to your environment. PCRE (Perl-compatible regular expressions) is a C library implementing regular expression pattern-matching using the same semantics as Perl 5. proj.4/7.0.0/gnu-9.2.0 Adds the PROJ.4 Cartographic Projections library to your environment. proj.4/8.1.1/gnu-10.2.0 Adds the PROJ.4 Cartographic Projections library to your environment. proj.4/9.2.0/gnu-10.2.0 Adds the PROJ.4 Cartographic Projections library to your environment. protobuf/3.14.0/gnu-9.2.0 adds Google Protocol Buffers for GCC 9.2.0 to your environment. protobuf/3.17.3/gnu-10.2.0 adds Google Protocol Buffers for GCC 10.2.0 to your environment. prsice/2.3.3/gnu-9.2.0 PRSice (pronounced 'precise') is a software package for calculating, applying, evaluating and plotting the results of polygenic risk scores (PRS). python/3.9.6-gnu-10.2.0 Adds Python 3.9.6 compiled with GNU 10.2.0 with pip and virtualenv to your environment variables. python3/3.9-gnu-10.2.0 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list quip/c6359e1/gnu-10.2.0 Adds QUIP to your environment built with GNU 10.2.0 for recent versions of LAMMPS r/3.6.3-openblas/gnu-9.2.0 Adds R 3.6.3 and Bioconductor 3.10 to your environment. r/4.0.2-openblas/gnu-9.2.0 Adds R 4.0.2 and Bioconductor 3.11 to your environment. r/4.1.1-openblas/gnu-10.2.0 Adds R 4.1.1 and Bioconductor 3.13 to your environment. r/r-3.6.3_bc-3.10 adds UCL recommended set of R packages to your environment variables r/r-4.0.2_bc-3.11 adds UCL recommended set of R packages for R 4.0.2 to your environment variables r/r-4.1.1_bc-3.13 adds UCL recommended set of R packages for R 4.1.1 to your environment variables r/r-4.2.0_bc-3.15 adds UCL recommended set of R packages for R 4.2.0 to your environment variables r/r-4.2.2_bc-3.16 adds UCL recommended set of R packages for R 4.2.2 to your environment variables r/r-4.2.3_bc-3.16 adds UCL recommended set of R packages for R 4.2.3 to your environment variables rcps-core-beta/gnu-7.3.0 adds a core set of applications and libraries to your environment. rcps-core-beta/gnu-8.3.0 adds a core set of applications and libraries to your environment. rcps-core-beta/gnu-9.2.0 adds a core set of applications and libraries to your environment. rcps-core-beta/gnu-10.2.0 adds a core set of applications and libraries to your environment. sac/102.0-test/gnu-10.2.0 Adds SAC 102.0 to your environment. scalapack/2.1.0/gnu-9.2.0/openblas-0.3.7 ScaLAPACK is a library of high-performance linear algebra routines for parallel distributed memory machines. ScaLAPACK solves dense and banded linear systems, least squares problems, eigenvalue problems, and singular value problems. sqlite/3.36.0/gnu-10.2.0 Adds SQLite Version 3.36.0 to your environment. sra-tools/3.0.6/gnu-10.2.0 adds SRA Tools 3.0.6 for GCC 10.2.0 to your environment. stata/16 Adds Stata/MP 16 to your environment. superlu/5.2.1/gnu-10.2.0 Adds superlu 5.2.1 to your environment. SuperLU is a general purpose library for the direct solution of large, sparse, nonsymmetric systems of linear equations. ucx/1.9.0/gnu-10.2.0 Adds ucx 1.9.0 to your environment. Unified Communication X (UCX) provides an optimized communication layer for Message Passing (MPI), PGAS/OpenSHMEM libraries and RPC/data-centric applications. udunits/2.2.26/gnu-9.2.0 adds the UDUNITS-2 package to your environment. udunits/2.2.28/gnu-10.2.0 adds the UDUNITS-2 package to your environment."},{"location":"Installed_Software_Lists/module-packages/#workaround-modules","title":"Workaround Modules","text":"

Sometimes we'll find a problem that can't be fixed properly, but can be worked-around by doing something that can be loaded as a module. That kind of module goes in this section.

Module Description bazel-compiler-helpers/intel-2018 Adds bazel compiler wrappers to your environment. getcwd-autoretry This module uses LD_PRELOAD to shadow the getcwd function with a version that retries on failure, and is intended to workaround a bug in the Lustre filesystem."},{"location":"Installed_Software_Lists/python-packages/","title":"Python Packages","text":"

We provide a collection of installed Python packages for each minor version of Python, as a bundle module. This page lists the packages for the current recommended Python 3 bundle.

This can be loaded using:

module load python3/recommended\n

The version of Python 3 provided with this bundle is currently Python 3.9.10.

Note that some packages we do not provide this way, because they have complicated non-Python dependencies. These are usually provided using the normal application modules system. This includes TensorFlow.

The following list was last updated at 15:05:04 (+0100) on 09 Sep 2023.

Module Version Description acora 2.3 Fast multi-keyword search engine for text strings appdirs 1.4.4 A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\". args 0.1.0 Command Arguments for Humans. ase 3.22.1 Atomic Simulation Environment astor 0.8.1 Read/rewrite/write Python ASTs astropy 5.0.1 Astronomy and astrophysics core library asttokens 2.0.5 Annotate AST trees with source code positions attrs 21.4.0 Classes Without Boilerplate backcall 0.2.0 Specifications for callback functions passed in to an API biopython 1.79 Freely available tools for computational molecular biology. bitarray 2.3.6 efficient arrays of booleans -- C extension black 22.1.0 The uncompromising code formatter. BoltzTraP2 20.7.1 band-structure interpolator and transport coefficient calculator certifi 2021.10.8 Python package for providing Mozilla's CA Bundle. cffi 1.15.0 Foreign Function Interface for Python calling C code. cftime 1.5.2 Time-handling functionality from netcdf4-python charset-normalizer 2.0.11 The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet. click 8.0.3 Composable command line interface toolkit clint 0.5.1 Python Command Line Interface Tools colorama 0.4.4 Cross-platform colored terminal text. coloredlogs 15.0.1 Colored terminal output for Python's logging module colormath 3.0.0 Color math and conversion library. commonmark 0.9.1 Python parser for the CommonMark Markdown spec cryptography 36.0.1 cryptography is a package which provides cryptographic recipes and primitives to Python developers. cutadapt 3.5 trim adapters from high-throughput sequencing reads cvxopt 1.2.7 Convex optimization package cycler 0.11.0 Composable style cycles Cython 0.29.27 The Cython compiler for writing C extensions for the Python language. cyvcf2 0.30.14 fast vcf parsing with cython + htslib deap 1.3.1 Distributed Evolutionary Algorithms in Python decorator 5.1.1 Decorators for Humans distlib 0.3.4 Distribution utilities dnaio 0.7.1 Read and write FASTA and FASTQ files efficiently ecdsa 0.17.0 ECDSA cryptographic signature library (pure python) emcee 3.1.1 The Python ensemble sampling toolkit for MCMC ephem 4.1.3 Compute positions of the planets and stars executing 0.8.2 Get the currently executing AST node of a frame, and other information filelock 3.4.2 A platform independent file lock. fonttools 4.29.1 Tools to manipulate font files funcparserlib 0.3.6 Recursive descent parsing library based on functional combinators future 0.18.2 Clean single-source support for Python 3 and 2 greenlet 1.1.2 Lightweight in-process concurrent programming hankel 1.1.0 Hankel Transformations using method of Ogata 2005 humanfriendly 10.0 Human friendly output for text interfaces using Python hy 0.20.0 Lisp and Python love each other. idna 3.3 Internationalized Domain Names in Applications (IDNA) imageio 2.15.0 Library for reading and writing a wide range of image, video, scientific, and volumetric data formats. importlib-metadata 4.10.1 Read metadata from Python packages iniconfig 1.1.1 iniconfig: brain-dead simple config-ini parsing ipython 8.0.1 IPython: Productive Interactive Computing isal 0.11.1 Faster zlib and gzip compatible compression and decompression by providing python bindings for the ISA-L library. jedi 0.18.1 An autocompletion tool for Python that can be used for text editors. Jinja2 3.0.3 A very fast and expressive template engine. joblib 1.1.0 Lightweight pipelining with Python functions kiwisolver 1.3.2 A fast implementation of the Cassowary constraint solver ldap3 2.9.1 A strictly RFC 4510 conforming LDAP V3 pure Python client library llvmlite 0.38.0 lightweight wrapper around basic LLVM functionality lxml 4.7.1 Powerful and Pythonic XML processing library combining libxml2/libxslt with the ElementTree API. lzstring 1.0.4 lz-string for python Mako 1.1.6 A super-fast templating language that borrows the best ideas from the existing templating languages. Markdown 3.3.6 Python implementation of Markdown. MarkupSafe 2.0.1 Safely add untrusted strings to HTML/XML markup. matplotlib 3.5.1 Python plotting package matplotlib-inline 0.1.3 Inline Matplotlib backend for Jupyter mpmath 1.2.1 Python library for arbitrary-precision floating-point arithmetic multiqc 1.12 Create aggregate bioinformatics analysis reports across many samples and tools mypy-extensions 0.4.3 Experimental type system extensions for programs checked with the mypy typechecker. netCDF4 1.5.8 Provides an object-oriented python interface to the netCDF version 4 library. networkx 2.6.3 Python package for creating and manipulating graphs and networks nibabel 3.2.2 Access a multitude of neuroimaging data formats ninja 1.10.2.3 Ninja is a small build system with a focus on speed nose 1.3.7 nose extends unittest to make testing easier numba 0.55.1 compiling Python code using LLVM numpy 1.21.5 NumPy is the fundamental package for array computing with Python. obspy 1.2.2 ObsPy - a Python framework for seismological observatories. packaging 21.3 Core utilities for Python packages pandas 1.4.0 Powerful data structures for data analysis, time series, and statistics parso 0.8.3 A Python Parser pathspec 0.9.0 Utility library for gitignore style pattern matching of file paths. pexpect 4.8.0 Pexpect allows easy control of interactive console applications. pickleshare 0.7.5 Tiny 'shelve'-like database with concurrency support Pillow 9.0.1 Python Imaging Library (Fork) pip 22.0.3 The PyPA recommended tool for installing Python packages. platformdirs 2.5.4 A small Python package for determining appropriate platform-specific dirs, e.g. a \"user data dir\". pluggy 1.0.0 plugin and hook calling mechanisms for python Pmw 2.0.1 Python Mega Widgets prompt-toolkit 3.0.27 Library for building powerful interactive command lines in Python ptyprocess 0.7.0 Run a subprocess in a pseudo terminal pure-eval 0.2.2 Safely evaluate AST nodes without side effects py 1.11.0 library with cross-python path, ini-parsing, io, code, log facilities pyasn1 0.4.8 ASN.1 types and codecs pycparser 2.21 C parser in Python pyerfa 2.0.0.1 Python bindings for ERFA pyfastaq 3.17.0 Script to manipulate FASTA and FASTQ files, plus API for developers Pygments 2.11.2 Pygments is a syntax highlighting package written in Python. pyparsing 3.0.7 Python parsing module pysam 0.18.0 pysam pyspglib 1.8.3.1 This is the pyspglib module. pytest 7.0.0 pytest: simple powerful testing with Python python-dateutil 2.8.2 Extensions to the standard Python datetime module python-Levenshtein 0.12.2 Python extension for computing string edit distances and similarities. pytz 2021.3 World timezone definitions, modern and historical PyWavelets 1.2.0 PyWavelets, wavelet transform module PyYAML 6.0 YAML parser and emitter for Python qutip 4.6.3 QuTiP: The Quantum Toolbox in Python regex 2022.1.18 Alternative regular expression module, to replace re. requests 2.27.1 Python HTTP for Humans. rich 11.2.0 Render rich text, tables, progress bars, syntax highlighting, markdown and more to the terminal rply 0.7.8 A pure Python Lex/Yacc that works with RPython scikit-image 0.19.1 Image processing in Python scikit-learn 1.0.2 A set of python modules for machine learning and data mining scipy 1.8.0 SciPy: Scientific Library for Python seaborn 0.11.2 seaborn: statistical data visualization setuptools 60.8.1 Easily download, build, install, upgrade, and uninstall Python packages simplejson 3.17.6 Simple, fast, extensible JSON encoder/decoder for Python six 1.16.0 Python 2 and 3 compatibility utilities spectra 0.0.11 Color scales and color conversion made easy for Python. spglib 1.16.3 This is the spglib module. SQLAlchemy 1.4.31 Database Abstraction Library sshpubkeys 3.3.1 SSH public key parser stack-data 0.1.4 Extract data from python stack frames and tracebacks for informative displays tabulate 0.8.9 Pretty-print tabular data threadpoolctl 3.1.0 threadpoolctl tifffile 2022.2.2 Read and write TIFF files tomli 2.0.1 A lil' TOML parser traitlets 5.1.1 Traitlets Python configuration system typing_extensions 4.0.1 Backported and Experimental Type Hints for Python 3.6+ urllib3 1.26.8 HTTP library with thread-safe connection pooling, file post, and more. virtualenv 20.13.1 Virtual Python Environment builder wcwidth 0.2.5 Measures the displayed width of unicode strings in a terminal weblogo 3.7.9 WebLogo3 : Sequence Logos Redrawn wheel 0.37.1 A built-package format for Python xlrd 2.0.1 Library for developers to extract data from Microsoft Excel (tm) .xls spreadsheet files XlsxWriter 3.0.2 A Python module for creating Excel XLSX files. xlutils 2.0.0 Utilities for working with Excel files that require both xlrd and xlwt xlwt 1.3.0 Library to create spreadsheet files compatible with MS Excel 97/2000/XP/2003 XLS files, on any platform, with Python 2.6, 2.7, 3.3+ xopen 1.4.0 Open compressed files transparently zipp 3.7.0 Backport of pathlib-compatible object wrapper for zip files"},{"location":"Installed_Software_Lists/r-packages/","title":"R Packages","text":"

We provide a collection of installed R packages for each release of R, as a bundle module. This page lists the packages for the current recommended R bundle.

This can be loaded using:

module load r/recommended\n

The version of R provided with this bundle is currently R version 4.2.0 (2022-04-22).

The following list was last updated at: 15:05:23 (+0100) on 09 Sep 2023.

Module Version Description abc 2.2.1 Tools for Approximate Bayesian Computation (ABC) abc.data 1.0 Data Only: Tools for Approximate Bayesian Computation (ABC) abind 1.4-5 Combine Multidimensional Arrays ade4 1.7-19 Analysis of Ecological Data: Exploratory and Euclidean Methods in Environmental Sciences adegenet 2.1.6 Exploratory Analysis of Genetic and Genomic Data ADGofTest 0.3 Anderson-Darling GoF test admisc 0.27 Adrian Dusa's Miscellaneous afex 1.2-1 Analysis of Factorial Experiments affxparser 1.68.1 Affymetrix File Parsing SDK affy 1.74.0 Methods for Affymetrix Oligonucleotide Arrays affydata 1.44.0 Affymetrix Data for Demonstration Purpose affyio 1.66.0 Tools for parsing Affymetrix data files affylmGUI 1.70.0 GUI for limma Package with Affymetrix Microarrays affyPLM 1.72.0 Methods for fitting probe-level models akima 0.6-3.4 Interpolation of Irregularly and Regularly Spaced Data annaffy 1.68.0 Annotation tools for Affymetrix biological metadata annmap 1.38.0 Genome annotation and visualisation package pertaining to Affymetrix arrays and NGS analysis. annotate 1.74.0 Annotation for microarrays AnnotationDbi 1.58.0 Manipulation of SQLite-based annotations in Bioconductor AnnotationFilter 1.20.0 Facilities for Filtering Bioconductor Annotation Resources AnnotationForge 1.38.0 Tools for building SQLite-based annotation data packages AnnotationHub 3.4.0 Client to access AnnotationHub resources ape 5.6-2 Analyses of Phylogenetics and Evolution arm 1.12-2 Data Analysis Using Regression and Multilevel/Hierarchical Models aroma.affymetrix 3.2.0 Analysis of Large Affymetrix Microarray Data Sets aroma.apd 0.6.0 A Probe-Level Data File Format Used by 'aroma.affymetrix' [deprecated] aroma.core 3.2.2 Core Methods and Classes Used by 'aroma.*' Packages Part of the Aroma Framework aroma.light 3.26.0 Light-Weight Methods for Normalization and Visualization of Microarray Data using Only Basic R Data Types arrangements 1.1.9 Fast Generators and Iterators for Permutations, Combinations, Integer Partitions and Compositions arsenal 3.6.3 An Arsenal of 'R' Functions for Large-Scale Statistical Summaries askpass 1.1 Safe Password Entry for R, Git, and SSH assertthat 0.2.1 Easy Pre and Post Assertions backports 1.4.1 Reimplementations of Functions Introduced Since R-3.0.0 bamsignals 1.28.0 Extract read count signals from bam files base 4.2.0 The R Base Package base64 2.0 Base64 Encoder and Decoder base64enc 0.1-3 Tools for base64 encoding BaSTA 1.9.4 Age-Specific Survival Analysis from Incomplete Capture-Recapture/Recovery Data BatchJobs 1.9 Batch Computing with R bayesplot 1.9.0 Plotting for Bayesian Models BBmisc 1.12 Miscellaneous Helper Functions for B. Bischl beachmat 2.12.0 Compiling Bioconductor to Handle Each Matrix Type beadarray 2.46.0 Quality assessment and low-level analysis for Illumina BeadArray data beadarrayExampleData 1.34.0 Example data for the beadarray package BeadDataPackR 1.48.0 Compression of Illumina BeadArray data beanplot 1.3.1 Visualization via Beanplots (Like Boxplot/Stripchart/Violin Plot) benchmarkme 1.0.7 Crowd Sourced System Benchmarks benchmarkmeData 1.0.4 Data Set for the 'benchmarkme' Package bezier 1.1.2 Toolkit for Bezier Curves and Splines BH 1.78.0-0 Boost C++ Header Files BiasedUrn 1.07 Biased Urn Model Distributions binom 1.1-1.1 Binomial Confidence Intervals for Several Parameterizations bio3d 2.4-3 Biological Structure Analysis Biobase 2.56.0 Biobase: Base functions for Bioconductor BiocFileCache 2.4.0 Manage Files Across Sessions BiocGenerics 0.42.0 S4 generic functions used in Bioconductor BiocIO 1.6.0 Standard Input and Output for Bioconductor Packages BiocManager 1.30.18 Access the Bioconductor Project Package Repository BiocParallel 1.30.2 Bioconductor facilities for parallel evaluation BiocVersion 3.15.2 Set the appropriate version of Bioconductor packages biomaRt 2.52.0 Interface to BioMart databases (i.e. Ensembl) Biostrings 2.64.0 Efficient manipulation of biological strings biovizBase 1.44.0 Basic graphic utilities for visualization of genomic data. bit 4.0.4 Classes and Methods for Fast Memory-Efficient Boolean Selections bit64 4.0.5 A S3 Class for Vectors of 64bit Integers bitops 1.0-7 Bitwise Operations blob 1.2.3 A Simple S3 Class for Representing Vectors of Binary Data ('BLOBS') blockmodeling 1.0.5 Generalized and Classical Blockmodeling of Valued Networks boot 1.3-28 Bootstrap Functions (Originally by Angelo Canty for S) BradleyTerry2 1.1-2 Bradley-Terry Models brew 1.0-7 Templating Framework for Report Generation brglm 0.7.2 Bias Reduction in Binomial-Response Generalized Linear Models bridgesampling 1.1-2 Bridge Sampling for Marginal Likelihoods and Bayes Factors brio 1.1.3 Basic R Input Output brms 2.18.0 Bayesian Regression Models using 'Stan' Brobdingnag 1.2-9 Very Large Numbers in R broom 0.8.0 Convert Statistical Objects into Tidy Tibbles BSgenome 1.64.0 Software infrastructure for efficient representation of full genomes and their SNPs BSgenome.Hsapiens.UCSC.hg19 1.4.3 Full genome sequences for Homo sapiens (UCSC version hg19, based on GRCh37.p13) bslib 0.3.1 Custom 'Bootstrap' 'Sass' Themes for 'shiny' and 'rmarkdown' bsseq 1.32.0 Analyze, manage and store bisulfite sequencing data bumphunter 1.38.0 Bump Hunter cachem 1.0.6 Cache R Objects with Automatic Pruning callr 3.7.0 Call R from R car 3.0-13 Companion to Applied Regression carData 3.0-5 Companion to Applied Regression Data Sets caret 6.0-92 Classification and Regression Training Category 2.62.0 Category Analysis caTools 1.18.2 Tools: Moving Window Statistics, GIF, Base64, ROC AUC, etc CDM 8.1-12 Cognitive Diagnosis Modeling cellranger 1.1.0 Translate Spreadsheet Cell Ranges to Rows and Columns ChAMP 2.26.0 Chip Analysis Methylation Pipeline for Illumina HumanMethylation450 and EPIC ChAMPdata 2.28.0 Data Packages for ChAMP package checkmate 2.1.0 Fast and Versatile Argument Checks class 7.3-20 Functions for Classification classInt 0.4-3 Choose Univariate Class Intervals cli 3.6.0 Helpers for Developing Command Line Interfaces clipr 0.8.0 Read and Write from the System Clipboard clue 0.3-60 Cluster Ensembles cluster 2.1.3 \"Finding Groups in Data\": Cluster Analysis Extended Rousseeuw et al. cmprsk 2.2-11 Subdistribution Analysis of Competing Risks coda 0.19-4 Output Analysis and Diagnostics for MCMC codetools 0.2-18 Code Analysis Tools for R colorRamps 2.3.1 Builds Color Tables colorspace 2.0-3 A Toolbox for Manipulating and Assessing Colors and Palettes colourpicker 1.1.1 A Colour Picker Tool for Shiny and for Selecting Colours in Plots combinat 0.0-8 combinatorics utilities commonmark 1.8.0 High Performance CommonMark and Github Markdown Rendering in R compiler 4.2.0 The R Compiler Package copula 1.0-1 Multivariate Dependence with Copulas copynumber 1.36.0 Segmentation of single- and multi-track copy number data by penalized least squares regression. corpcor 1.6.10 Efficient Estimation of Covariance and (Partial) Correlation corrplot 0.92 Visualization of a Correlation Matrix cowplot 1.1.1 Streamlined Plot Theme and Plot Annotations for 'ggplot2' cpp11 0.4.2 A C++11 Interface for R's C Interface crayon 1.5.1 Colored Terminal Output credentials 1.3.2 Tools for Managing SSH and Git Credentials crosstalk 1.2.0 Inter-Widget Interactivity for HTML Widgets crul 1.2.0 HTTP Client curl 4.3.2 A Modern and Flexible Web Client for R data.table 1.14.2 Extension of data.frame datasets 4.2.0 The R Datasets Package DBI 1.1.2 R Database Interface dbplyr 2.1.1 A 'dplyr' Back End for Databases DelayedArray 0.22.0 A unified framework for working transparently with on-disk and in-memory array-like datasets DelayedMatrixStats 1.18.0 Functions that Apply to Rows and Columns of 'DelayedMatrix' Objects deldir 1.0-6 Delaunay Triangulation and Dirichlet (Voronoi) Tessellation dendextend 1.15.2 Extending 'dendrogram' Functionality in R DEoptimR 1.0-11 Differential Evolution Optimization in Pure R Deriv 4.1.3 Symbolic Differentiation desc 1.4.1 Manipulate DESCRIPTION Files DescTools 0.99.47 Tools for Descriptive Statistics DESeq2 1.36.0 Differential gene expression analysis based on the negative binomial distribution devtools 2.4.3 Tools to Make Developing R Packages Easier DEXSeq 1.42.0 Inference of differential exon usage in RNA-Seq dichromat 2.0-0.1 Color Schemes for Dichromats diffobj 0.3.5 Diffs for R Objects digest 0.6.29 Create Compact Hash Digests of R Objects diptest 0.76-0 Hartigan's Dip Test Statistic for Unimodality - Corrected distr 2.8.0 Object Oriented Implementation of Distributions distrEx 2.8.0 Extensions of Package 'distr' distributional 0.3.0 Vectorised Probability Distributions DMRcate 2.10.0 Methylation array and sequencing spatial analysis methods DNAcopy 1.70.0 DNA copy number data analysis docopt 0.7.1 Command-Line Interface Specification Language doMC 1.3.8 Foreach Parallel Adaptor for 'parallel' doMPI 0.2.2 Foreach Parallel Adaptor for the Rmpi Package doParallel 1.0.17 Foreach Parallel Adaptor for the 'parallel' Package doRNG 1.8.2 Generic Reproducible Parallel Backend for 'foreach' Loops dotCall64 1.0-1 Enhanced Foreign Function Interface Supporting Long Vectors dparser 1.3.1-9 Port of 'Dparser' Package dplyr 1.0.9 A Grammar of Data Manipulation DSS 2.44.0 Dispersion shrinkage for sequencing data DT 0.23 A Wrapper of the JavaScript Library 'DataTables' dtplyr 1.2.1 Data Table Back-End for 'dplyr' dygraphs 1.1.1.6 Interface to 'Dygraphs' Interactive Time Series Charting Library dynamicTreeCut 1.63-1 Methods for Detection of Clusters in Hierarchical Clustering Dendrograms DynDoc 1.74.0 Dynamic document tools e1071 1.7-9 Misc Functions of the Department of Statistics, Probability Theory Group (Formerly: E1071), TU Wien easyRNASeq 2.32.0 Count summarization and normalization for RNA-Seq data EBSeq 1.36.0 An R package for gene and isoform differential expression analysis of RNA-seq data edgeR 3.38.1 Empirical Analysis of Digital Gene Expression Data in R effects 4.2-1 Effect Displays for Linear, Generalized Linear, and Other Models ellipse 0.4.2 Functions for Drawing Ellipses and Ellipse-Like Confidence Regions ellipsis 0.3.2 Tools for Working with ... ensembldb 2.20.1 Utilities to create and use Ensembl-based annotation databases Epi 2.46 Statistical Analysis in Epidemiology erer 3.1 Empirical Research in Economics with R estimability 1.3 Tools for Assessing Estimability of Linear Predictions etm 1.1.1 Empirical Transition Matrix evaluate 0.15 Parsing and Evaluation Tools that Provide More Details than the Default evd 2.3-6 Functions for Extreme Value Distributions Exact 3.2 Unconditional Exact Test ExperimentHub 2.4.0 Client to access ExperimentHub resources expm 0.999-6 Matrix Exponential, Log, 'etc' FactoMineR 2.4 Multivariate Exploratory Data Analysis and Data Mining fail 1.3 File Abstraction Interface Layer (FAIL) fansi 1.0.3 ANSI Control Sequence Aware String Functions farver 2.1.0 High Performance Colour Space Manipulation fastcluster 1.2.3 Fast Hierarchical Clustering Routines for R and 'Python' fastICA 1.2-3 FastICA Algorithms to Perform ICA and Projection Pursuit fastmap 1.1.0 Fast Data Structures fastmatch 1.1-3 Fast 'match()' Function FDb.InfiniumMethylation.hg19 2.2.0 Annotation package for Illumina Infinium DNA methylation probes fdrtool 1.2.17 Estimation of (Local) False Discovery Rates and Higher Criticism fields 13.3 Tools for Spatial Data filelock 1.0.2 Portable File Locking flashClust 1.01-2 Implementation of optimal hierarchical clustering flexmix 2.3-17 Flexible Mixture Modeling fontawesome 0.2.2 Easily Work with 'Font Awesome' Icons forcats 0.5.1 Tools for Working with Categorical Variables (Factors) foreach 1.5.2 Provides Foreach Looping Construct foreign 0.8-82 Read Data Stored by 'Minitab', 'S', 'SAS', 'SPSS', 'Stata', 'Systat', 'Weka', 'dBase', ... formatR 1.12 Format R Code Automatically Formula 1.2-4 Extended Model Formulas fpc 2.2-9 Flexible Procedures for Clustering fs 1.5.2 Cross-Platform File System Operations Based on 'libuv' futile.logger 1.4.3 A Logging Utility for R futile.options 1.0.1 Futile Options Management future 1.25.0 Unified Parallel and Distributed Processing in R for Everyone future.apply 1.9.0 Apply Function to Elements in Parallel using Futures gam 1.20.1 Generalized Additive Models gamlss 5.4-3 Generalised Additive Models for Location Scale and Shape gamlss.data 6.0-2 Data for Generalised Additive Models for Location Scale and Shape gamlss.dist 6.0-3 Distributions for Generalized Additive Models for Location Scale and Shape gamlss.mx 6.0-0 Fitting Mixture Distributions with GAMLSS gamlss.nl 4.1-0 Fitting non linear parametric GAMLSS models gargle 1.2.0 Utilities for Working with Google APIs gcrma 2.68.0 Background Adjustment Using Sequence Information gdata 2.18.0.1 Various R Programming Tools for Data Manipulation genefilter 1.78.0 genefilter: methods for filtering genes from high-throughput experiments geneLenDataBase 1.32.0 Lengths of mRNA transcripts for a number of genomes GeneNet 1.2.16 Modeling and Inferring Gene Networks geneplotter 1.74.0 Graphics related functions for Bioconductor generics 0.1.2 Common S3 Generics not Provided by Base R Methods Related to Model Fitting genetics 1.3.8.1.3 Population Genetics GenomeInfoDb 1.32.2 Utilities for manipulating chromosome names, including modifying them to follow a particular naming style GenomeInfoDbData 1.2.8 Species and taxonomy ID look up tables used by GenomeInfoDb genomeIntervals 1.52.0 Operations on genomic intervals GenomicAlignments 1.32.0 Representation and manipulation of short genomic alignments GenomicFeatures 1.48.1 Conveniently import and query gene models GenomicRanges 1.48.0 Representation and manipulation of genomic intervals geojson 0.3.4 Classes for 'GeoJSON' geojsonio 0.9.4 Convert Data from and to 'GeoJSON' or 'TopoJSON' geojsonsf 2.0.2 GeoJSON to Simple Feature Converter geometries 0.2.0 Convert Between R Objects and Geometric Structures GEOquery 2.64.2 Get data from NCBI Gene Expression Omnibus (GEO) gert 1.6.0 Simple Git Client for R getopt 1.20.3 C-Like 'getopt' Behavior GGally 2.1.2 Extension to 'ggplot2' ggforce 0.3.3 Accelerating 'ggplot2' ggplot2 3.3.6 Create Elegant Data Visualisations Using the Grammar of Graphics ggrepel 0.9.1 Automatically Position Non-Overlapping Text Labels with 'ggplot2' ggridges 0.5.3 Ridgeline Plots in 'ggplot2' ggtext 0.1.2 Improved Text Rendering Support for 'ggplot2' gh 1.3.0 'GitHub' 'API' gitcreds 0.1.1 Query 'git' Credentials from 'R' GJRM 0.2-6 Generalised Joint Regression Modelling gld 2.6.6 Estimation and Use of the Generalised (Tukey) Lambda Distribution glmnet 4.1-4 Lasso and Elastic-Net Regularized Generalized Linear Models globals 0.15.0 Identify Global Objects in R Expressions globaltest 5.50.0 Testing Groups of Covariates/Features for Association with a Response Variable, with Applications to Gene Set Testing glue 1.6.2 Interpreted String Literals gmp 0.6-5 Multiple Precision Arithmetic GO.db 3.15.0 A set of annotation maps describing the entire Gene Ontology goftest 1.2-3 Classical Goodness-of-Fit Tests for Univariate Distributions googledrive 2.0.0 An Interface to Google Drive googlesheets4 1.0.0 Access Google Sheets using the Sheets API V4 goseq 1.48.0 Gene Ontology analyser for RNA-seq and other length biased data GOstats 2.62.0 Tools for manipulating GO and microarrays gower 1.0.0 Gower's Distance gplots 3.1.3 Various R Programming Tools for Plotting Data graph 1.74.0 graph: A package to handle graph data structures graphics 4.2.0 The R Graphics Package grDevices 4.2.0 The R Graphics Devices and Support for Colours and Fonts grid 4.2.0 The Grid Graphics Package gridExtra 2.3 Miscellaneous Functions for \"Grid\" Graphics gridtext 0.1.5 Improved Text Rendering Support for 'Grid' Graphics GSEABase 1.58.0 Gene set enrichment data structures and methods gsl 2.1-7.1 Wrapper for the Gnu Scientific Library gsmoothr 0.1.7 Smoothing tools gtable 0.3.0 Arrange 'Grobs' in Tables gtools 3.9.2.1 Various R Programming Tools Gviz 1.40.1 Plotting data and annotation information along genomic coordinates HAC 1.1-0 Estimation, Simulation and Visualization of Hierarchical Archimedean Copulae (HAC) haplo.stats 1.8.7 Statistical Analysis of Haplotypes with Traits and Covariates when Linkage Phase is Ambiguous hardhat 0.2.0 Construct Modeling Packages haven 2.5.0 Import and Export 'SPSS', 'Stata' and 'SAS' Files HDF5Array 1.24.0 HDF5 backend for DelayedArray objects hexbin 1.28.2 Hexagonal Binning Routines hgu95av2.db 3.13.0 Affymetrix Affymetrix HG_U95Av2 Array annotation data (chip hgu95av2) HI 0.5 Simulation from Distributions Supported by Nested Hyperplanes highr 0.9 Syntax Highlighting for R Source Code HistData 0.8-7 Data Sets from the History of Statistics and Data Visualization Hmisc 4.7-0 Harrell Miscellaneous hms 1.1.1 Pretty Time of Day htmlTable 2.4.0 Advanced Tables for Markdown/HTML htmltools 0.5.2 Tools for HTML htmlwidgets 1.5.4 HTML Widgets for R httpcode 0.3.0 'HTTP' Status Code Helper httpuv 1.6.5 HTTP and WebSocket Server Library httr 1.4.3 Tools for Working with URLs and HTTP hugene10stprobeset.db 8.8.0 Affymetrix hugene10 annotation data (chip hugene10stprobeset) hugene10sttranscriptcluster.db 8.8.0 Affymetrix hugene10 annotation data (chip hugene10sttranscriptcluster) hwriter 1.3.2.1 HTML Writer - Outputs R Objects in HTML Format ids 1.0.1 Generate Random Identifiers igraph 1.3.1 Network Analysis and Visualization Illumina450ProbeVariants.db 1.32.0 Annotation Package combining variant data from 1000 Genomes Project for Illumina HumanMethylation450 Bead Chip probes IlluminaHumanMethylation450kanno.ilmn12.hg19 0.6.1 Annotation for Illumina's 450k methylation arrays IlluminaHumanMethylation450kmanifest 0.4.0 Annotation for Illumina's 450k methylation arrays IlluminaHumanMethylationEPICanno.ilm10b4.hg19 0.6.0 Annotation for Illumina's EPIC methylation arrays IlluminaHumanMethylationEPICmanifest 0.3.0 Manifest for Illumina's EPIC methylation arrays illuminaHumanv4.db 1.26.0 Illumina HumanHT12v4 annotation data (chip illuminaHumanv4) illuminaio 0.38.0 Parsing Illumina Microarray Output Files impute 1.70.0 impute: Imputation for microarray data ini 0.3.1 Read and Write '.ini' Files INLA 22.05.18-2 Full Bayesian Analysis of Latent Gaussian Models using Integrated Nested Laplace Approximations inline 0.3.19 Functions to Inline C, C++, Fortran Function Calls from R insight 0.17.1 Easy Access to Model Information for Various Model Objects interactiveDisplayBase 1.34.0 Base package for enabling powerful shiny web displays of Bioconductor objects intervals 0.15.2 Tools for Working with Points and Intervals ipred 0.9-12 Improved Predictors IRanges 2.30.0 Foundation of integer range manipulation in Bioconductor ismev 1.42 An Introduction to Statistical Modeling of Extreme Values isoband 0.2.5 Generate Isolines and Isobands from Regularly Spaced Elevation Grids ISOcodes 2022.01.10 Selected ISO Codes isva 1.9 Independent Surrogate Variable Analysis iterators 1.0.14 Provides Iterator Construct iterpc 0.4.2 Efficient Iterator for Permutations and Combinations JADE 2.0-3 Blind Source Separation Methods Based on Joint Diagonalization and Some BSS Performance Criteria jpeg 0.1-9 Read and write JPEG images jqr 1.2.3 Client for 'jq', a 'JSON' Processor jquerylib 0.1.4 Obtain 'jQuery' as an HTML Dependency Object jsonify 1.2.1 Convert Between 'R' Objects and Javascript Object Notation (JSON) jsonlite 1.8.0 A Simple and Robust JSON Parser and Generator for R karyoploteR 1.22.0 Plot customizable linear genomes displaying arbitrary data KEGGREST 1.36.0 Client-side REST access to the Kyoto Encyclopedia of Genes and Genomes (KEGG) kernlab 0.9-30 Kernel-Based Machine Learning Lab KernSmooth 2.23-20 Functions for Kernel Smoothing Supporting Wand & Jones (1995) knitr 1.39 A General-Purpose Package for Dynamic Report Generation in R kohonen 3.0.11 Supervised and Unsupervised Self-Organising Maps kpmt 0.1.0 Known Population Median Test labeling 0.4.2 Axis Labeling lambda.r 1.2.4 Modeling Data with Functional Programming later 1.3.0 Utilities for Scheduling Functions to Execute Later with Event Loops lattice 0.20-45 Trellis Graphics for R latticeExtra 0.6-29 Extra Graphical Utilities Based on Lattice lava 1.6.10 Latent Variable Models lazyeval 0.2.2 Lazy (Non-Standard) Evaluation lbfgsb3c 2020-3.2 Limited Memory BFGS Minimizer with Bounds on Parameters with optim() 'C' Interface leafem 0.2.0 'leaflet' Extensions for 'mapview' leaflet 2.1.1 Create Interactive Web Maps with the JavaScript 'Leaflet' Library leaflet.providers 1.9.0 Leaflet Providers leafsync 0.1.0 Small Multiples for Leaflet Web Maps leaps 3.1 Regression Subset Selection learnr 0.10.1 Interactive Tutorials for R lifecycle 1.0.1 Manage the Life Cycle of your Package Functions limma 3.52.1 Linear Models for Microarray Data listenv 0.8.0 Environments Behaving (Almost) as Lists lme4 1.1-29 Linear Mixed-Effects Models using 'Eigen' and S4 lmerTest 3.1-3 Tests in Linear Mixed Effects Models lmom 2.9 L-Moments lmtest 0.9-40 Testing Linear Regression Models locfit 1.5-9.5 Local Regression, Likelihood and Density Estimation longitudinal 1.1.13 Analysis of Multiple Time Course Data loo 2.5.1 Efficient Leave-One-Out Cross-Validation and WAIC for Bayesian Models lotri 0.4.2 A Simple Way to Specify Symmetric, Block Diagonal Matrices lpSolve 5.6.15 Interface to 'Lp_solve' v. 5.5 to Solve Linear/Integer Programs LSD 4.1-0 Lots of Superior Depictions ltm 1.2-0 Latent Trait Models under IRT lubridate 1.8.0 Make Dealing with Dates a Little Easier lumi 2.48.0 BeadArray Specific Methods for Illumina Methylation and Expression Microarrays lwgeom 0.2-8 Bindings to Selected 'liblwgeom' Functions for Simple Features made4 1.70.0 Multivariate analysis of microarray data using ADE4 magic 1.6-0 Create and Investigate Magic Squares magrittr 2.0.3 A Forward-Pipe Operator for R maps 3.4.0 Draw Geographical Maps maptools 1.1-4 Tools for Handling Spatial Objects maptree 1.4-8 Mapping, Pruning, and Graphing Tree Models markdown 1.1 Render Markdown with the C Library 'Sundown' marray 1.74.0 Exploratory analysis for two-color spotted microarray data MASS 7.3-56 Support Functions and Datasets for Venables and Ripley's MASS mathjaxr 1.6-0 Using 'Mathjax' in Rd Files Matrix 1.5-3 Sparse and Dense Matrix Classes and Methods MatrixGenerics 1.8.0 S4 Generic Summary Statistic Functions that Operate on Matrix-Like Objects MatrixModels 0.5-0 Modelling with Sparse and Dense Matrices matrixStats 0.62.0 Functions that Apply to Rows and Columns of Matrices (and to Vectors) mclust 5.4.10 Gaussian Mixture Modelling for Model-Based Clustering, Classification, and Density Estimation mcmc 0.9-7 Markov Chain Monte Carlo memoise 2.0.1 'Memoisation' of Functions MendelianRandomization 0.6.0 Mendelian Randomization Package metadat 1.2-0 Meta-Analysis Datasets metafor 3.4-0 Meta-Analysis Package for R methods 4.2.0 Formal Methods and Classes methylumi 2.42.0 Handle Illumina methylation data mgcv 1.8-40 Mixed GAM Computation Vehicle with Automatic Smoothness Estimation mime 0.12 Map Filenames to MIME Types minfi 1.42.0 Analyze Illumina Infinium DNA methylation arrays miniUI 0.1.1.1 Shiny UI Widgets for Small Screens minpack.lm 1.2-2 R Interface to the Levenberg-Marquardt Nonlinear Least-Squares Algorithm Found in MINPACK, Plus Support for Bounds minqa 1.2.4 Derivative-free optimization algorithms by quadratic approximation missMethyl 1.30.0 Analysing Illumina HumanMethylation BeadChip Data mitools 2.4 Tools for Multiple Imputation of Missing Data mlr 2.19.0 Machine Learning in R mnormt 2.0.2 The Multivariate Normal and t Distributions, and Their Truncated Versions ModelMetrics 1.2.2.2 Rapid Calculation of Model Metrics modelr 0.1.8 Modelling Functions that Work with the Pipe modeltools 0.2-23 Tools and Classes for Statistical Models msm 1.6.9 Multi-State Markov and Hidden Markov Models in Continuous Time mstate 0.3.2 Data Preparation, Estimation and Prediction in Multi-State Models multcomp 1.4-19 Simultaneous Inference in General Parametric Models multtest 2.52.0 Resampling-based multiple hypothesis testing munsell 0.5.0 Utilities for Using Munsell Colours mvtnorm 1.1-3 Multivariate Normal and t Distributions n1qn1 6.0.1-11 Port of the 'Scilab' 'n1qn1' Module for Unconstrained BFGS Optimization nleqslv 3.3.2 Solve Systems of Nonlinear Equations nlme 3.1-157 Linear and Nonlinear Mixed Effects Models nlmixr2 2.0.8 Nonlinear Mixed Effects Models in Population PK/PD nlmixr2data 2.0.7 Nonlinear Mixed Effects Models in Population PK/PD, Data nlmixr2est 2.1.3 Nonlinear Mixed Effects Models in Population PK/PD, Estimation Routines nlmixr2extra 2.0.8 Nonlinear Mixed Effects Models in Population PK/PD, Extra Support Functions nlmixr2plot 2.0.7 Nonlinear Mixed Effects Models in Population PK/PD, Plot Functions nloptr 2.0.2 R Interface to NLopt NLP 0.2-1 Natural Language Processing Infrastructure nnet 7.3-17 Feed-Forward Neural Networks and Multinomial Log-Linear Models nor1mix 1.3-0 Normal aka Gaussian (1-d) Mixture Models (S3 Classes and Methods) npde 3.2 Normalised Prediction Distribution Errors for Nonlinear Mixed-Effect Models numDeriv 2016.8-1.1 Accurate Numerical Derivatives OPE 0.7 Outer-product emulator openssl 2.0.2 Toolkit for Encryption, Signatures and Certificates Based on OpenSSL optextras 2019-12.4 Tools to Support Optimization Possibly with Bounds and Masks optparse 1.7.1 Command Line Option Parser org.Hs.eg.db 3.15.0 Genome wide annotation for Human pander 0.6.5 An R 'Pandoc' Writer parallel 4.2.0 Support for Parallel computation in R parallelly 1.31.1 Enhancing the 'parallel' Package parallelMap 1.5.1 Unified Interface to Parallelization Back-Ends ParamHelpers 1.14 Helpers for Parameters in Black-Box Optimization, Tuning and Machine Learning paran 1.5.2 Horn's Test of Principal Components/Factors pbapply 1.5-0 Adding Progress Bar to '*apply' Functions pbkrtest 0.5.1 Parametric Bootstrap, Kenward-Roger and Satterthwaite Based Methods for Test in Mixed Models pcaPP 2.0-1 Robust PCA by Projection Pursuit pegas 1.1 Population and Evolutionary Genetics Analysis System permute 0.9-7 Functions for Generating Restricted Permutations of Data phangorn 2.8.1 Phylogenetic Reconstruction and Analysis phia 0.2-1 Post-Hoc Interaction Analysis pillar 1.7.0 Coloured Formatting for Columns pixmap 0.4-12 Bitmap Images / Pixel Maps pkgbuild 1.3.1 Find Tools Needed to Build R Packages pkgconfig 2.0.3 Private Configuration for 'R' Packages pkgload 1.2.4 Simulate Package Installation and Attach pkgmaker 0.32.2 Development Utilities for R Packages plinkQC 0.3.4 Genotype Quality Control with 'PLINK' plogr 0.2.0 The 'plog' C++ Logging Library plotly 4.10.0 Create Interactive Web Graphics via 'plotly.js' pls 2.8-0 Partial Least Squares and Principal Component Regression plyr 1.8.7 Tools for Splitting, Applying and Combining Data png 0.1-7 Read and write PNG images poLCA 1.6.0.1 Polytomous Variable Latent Class Analysis polspline 1.1.20 Polynomial Spline Routines polyclip 1.10-0 Polygon Clipping polycor 0.8-1 Polychoric and Polyserial Correlations posterior 1.2.1 Tools for Working with Posterior Distributions poweRlaw 0.70.6 Analysis of Heavy Tailed Distributions prabclus 2.3-2 Functions for Clustering and Testing of Presence-Absence, Abundance and Multilocus Genetic Data pracma 2.3.8 Practical Numerical Math Functions praise 1.0.0 Praise Users PreciseSums 0.5 Accurate Floating Point Sums and Products preprocessCore 1.58.0 A collection of pre-processing functions prettydoc 0.4.1 Creating Pretty Documents from R Markdown prettyunits 1.1.1 Pretty, Human Readable Formatting of Quantities pROC 1.18.0 Display and Analyze ROC Curves processx 3.5.3 Execute and Control System Processes prodlim 2019.11.13 Product-Limit Estimation for Censored Event History Analysis profileModel 0.6.1 Profiling Inference Functions for Various Model Classes progress 1.2.2 Terminal Progress Bars progressr 0.10.0 An Inclusive, Unifying API for Progress Updates promises 1.2.0.1 Abstractions for Promise-Based Asynchronous Programming ProtGenerics 1.28.0 Generic infrastructure for Bioconductor mass spectrometry packages proto 1.0.0 Prototype Object-Based Programming protolite 2.1.1 Highly Optimized Protocol Buffer Serializers proxy 0.4-26 Distance and Similarity Measures ps 1.7.0 List, Query, Manipulate System Processes PSCBS 0.66.0 Analysis of Parent-Specific DNA Copy Numbers pspline 1.0-19 Penalized Smoothing Splines psych 2.2.5 Procedures for Psychological, Psychometric, and Personality Research purrr 0.3.4 Functional Programming Tools qs 0.25.4 Quick Serialization of R Objects quadprog 1.5-8 Functions to Solve Quadratic Programming Problems quanteda 3.2.1 Quantitative Analysis of Textual Data quanteda.corpora 0.9.2 A collection of corpora for quanteda quantmod 0.4.20 Quantitative Financial Modelling Framework quantreg 5.93 Quantile Regression qvalue 2.28.0 Q-value estimation for false discovery rate control qvcalc 1.0.2 Quasi Variances for Factor Effects in Statistical Models R.cache 0.15.0 Fast and Light-Weight Caching (Memoization) of Objects and Results to Speed Up Computations R.devices 2.17.0 Unified Handling of Graphics Devices R.filesets 2.14.0 Easy Handling of and Access to Files Organized in Structured Directories R.huge 0.9.0 Methods for Accessing Huge Amounts of Data [deprecated] R.methodsS3 1.8.1 S3 Methods Simplified R.oo 1.24.0 R Object-Oriented Programming with or without References R.rsp 0.44.0 Dynamic Generation of Scientific Reports R.utils 2.11.0 Various Programming Utilities R2HTML 2.3.3 HTML Exportation for R Objects R2WinBUGS 2.1-21 Running 'WinBUGS' and 'OpenBUGS' from 'R' / 'S-PLUS' R6 2.5.1 Encapsulated Classes with Reference Semantics randomForest 4.7-1.1 Breiman and Cutler's Random Forests for Classification and Regression rapidjsonr 1.2.0 'Rapidjson' C++ Header Files RApiSerialize 0.1.2 R API Serialization rappdirs 0.3.3 Application Directories: Determine Where to Save Data, Caches, and Logs raster 3.5-15 Geographic Data Analysis and Modeling RBGL 1.72.0 An interface to the BOOST graph library rcmdcheck 1.4.0 Run 'R CMD check' from 'R' and Capture Results RColorBrewer 1.1-3 ColorBrewer Palettes Rcpp 1.0.9 Seamless R and C++ Integration RcppArmadillo 0.11.4.2.1 'Rcpp' Integration for the 'Armadillo' Templated Linear Algebra Library RcppEigen 0.3.3.9.2 'Rcpp' Integration for the 'Eigen' Templated Linear Algebra Library RcppGSL 0.3.11 'Rcpp' Integration for 'GNU GSL' Vectors and Matrices RcppParallel 5.1.5 Parallel Programming Tools for 'Rcpp' RcppZiggurat 0.1.6 'Rcpp' Integration of Different \"Ziggurat\" Normal RNG Implementations RCurl 1.98-1.6 General Network (HTTP/FTP/...) Client Interface for R readr 2.1.2 Read Rectangular Text Data readxl 1.4.0 Read Excel Files recipes 0.2.0 Preprocessing and Feature Engineering Steps for Modeling regioneR 1.28.0 Association analysis of genomic regions based on permutation tests registry 0.5-1 Infrastructure for R Package Registries relaimpo 2.2-6 Relative Importance of Regressors in Linear Models rematch 1.0.1 Match Regular Expressions with a Nicer 'API' rematch2 2.1.2 Tidy Output from Regular Expression Matching remotes 2.4.2 R Package Installation from Remote Repositories, Including 'GitHub' renv 0.15.4 Project Environments Repitools 1.42.0 Epigenomic tools reprex 2.0.1 Prepare Reproducible Example Code via the Clipboard reshape 0.8.9 Flexibly Reshape Data reshape2 1.4.4 Flexibly Reshape Data: A Reboot of the Reshape Package restfulr 0.0.13 R Interface to RESTful Web Services rex 1.2.1 Friendly Regular Expressions Rfast 2.0.6 A Collection of Efficient and Extremely Fast R Functions rgdal 1.5-32 Bindings for the 'Geospatial' Data Abstraction Library rgeos 0.5-9 Interface to Geometry Engine - Open Source ('GEOS') rgl 0.108.3.2 3D Visualization Using OpenGL Rgraphviz 2.40.0 Provides plotting capabilities for R graph objects rhdf5 2.40.0 R Interface to HDF5 rhdf5filters 1.8.0 HDF5 Compression Filters Rhdf5lib 1.18.2 hdf5 library as an R package Rhtslib 1.28.0 HTSlib high-throughput sequencing library as an R package Ringo 1.60.0 R Investigation of ChIP-chip Oligoarrays rjson 0.2.21 JSON for R RJSONIO 1.3-1.6 Serialize R Objects to JSON, JavaScript Object Notation rlang 1.0.2 Functions for Base Types and Core R and 'Tidyverse' Features rlecuyer 0.3-5 R Interface to RNG with Multiple Streams rlist 0.4.6.2 A Toolbox for Non-Tabular Data Manipulation rmarkdown 2.14 Dynamic Documents for R Rmpfr 0.8-7 R MPFR - Multiple Precision Floating-Point Reliable Rmpi 0.6-9.2 Interface (Wrapper) to MPI (Message-Passing Interface) rms 6.3-0 Regression Modeling Strategies RMySQL 0.10.23 Database Interface and 'MySQL' Driver for R RNetCDF 2.5-2 Interface to 'NetCDF' Datasets rngtools 1.5.2 Utility Functions for Working with Random Number Generators robustbase 0.95-0 Basic Robust Statistics ROC 1.72.0 utilities for ROC, with microarray focus rootSolve 1.8.2.3 Nonlinear Root Finding, Equilibrium and Steady-State Analysis of Ordinary Differential Equations roxygen2 7.2.0 In-Line Documentation for R rpart 4.1.16 Recursive Partitioning and Regression Trees RPMM 1.25 Recursively Partitioned Mixture Model rprojroot 2.0.3 Finding Files in Project Subdirectories Rsamtools 2.12.0 Binary alignment (BAM), FASTA, variant call (BCF), and tabix file import Rsolnp 1.16 General Non-Linear Optimization RSQLite 2.2.14 SQLite Interface for R rstan 2.21.5 R Interface to Stan rstanarm 2.21.3 Bayesian Applied Regression Modeling via Stan rstantools 2.2.0 Tools for Developing R Packages Interfacing with 'Stan' rstudioapi 0.13 Safely Access the RStudio API Rsubread 2.10.1 Mapping, quantification and variant analysis of sequencing data rtracklayer 1.56.0 R interface to genome annotation files and the UCSC genome browser ruv 0.9.7.1 Detect and Remove Unwanted Variation using Negative Controls rversions 2.1.1 Query 'R' Versions, Including 'r-release' and 'r-oldrel' rvest 1.0.2 Easily Harvest (Scrape) Web Pages Rvmmin 2018-4.17.1 Variable Metric Nonlinear Function Minimization rxode2 2.0.11 Facilities for Simulating from ODE-Based Models rxode2et 2.0.9 Event Table Functions for 'rxode2' rxode2ll 2.0.9 Log-Likelihood Functions for 'rxode2' rxode2parse 2.0.13 Parsing and Code Generation Functions for 'rxode2' rxode2random 2.0.9 Random Number Generation Functions for 'rxode2' s2 1.0.7 Spherical Geometry Operators Using the S2 Geometry Library S4Vectors 0.34.0 Foundation of vector-like and list-like containers in Bioconductor saemix 3.0 Stochastic Approximation Expectation Maximization (SAEM) Algorithm sampling 2.9 Survey Sampling sandwich 3.0-1 Robust Covariance Matrix Estimators sass 0.4.1 Syntactically Awesome Style Sheets ('Sass') scales 1.2.0 Scale Functions for Visualization scam 1.2-12 Shape Constrained Additive Models scatterplot3d 0.3-41 3D Scatter Plot schoolmath 0.4.1 Functions and Datasets for Math Used in School scrime 1.3.5 Analysis of High-Dimensional Categorical Data Such as SNP Data segmented 1.5-0 Regression Models with Break-Points / Change-Points Estimation selectr 0.4-2 Translate CSS Selectors to XPath Expressions sendmailR 1.2-1 send email using R seqinr 4.2-16 Biological Sequences Retrieval and Analysis seqLogo 1.62.0 Sequence logos for DNA sequence alignments sessioninfo 1.2.2 R Session Information sf 1.0-7 Simple Features for R sfheaders 0.4.0 Converts Between R Objects and Simple Feature Objects sfsmisc 1.1-13 Utilities from 'Seminar fuer Statistik' ETH Zurich shape 1.4.6 Functions for Plotting Graphical Shapes, Colors shiny 1.7.1 Web Application Framework for R shinyjs 2.1.0 Easily Improve the User Experience of Your Shiny Apps in Seconds shinystan 2.6.0 Interactive Visual and Numerical Diagnostics and Posterior Analysis for Bayesian Models shinythemes 1.2.0 Themes for Shiny ShortRead 1.54.0 FASTQ input and manipulation siggenes 1.70.0 Multiple Testing using SAM and Efron's Empirical Bayes Approaches sitmo 2.0.2 Parallel Pseudo Random Number Generator (PPRNG) 'sitmo' Header Files slam 0.1-50 Sparse Lightweight Arrays and Matrices sn 2.0.2 The Skew-Normal and Related Distributions Such as the Skew-t and the SUN snow 0.4-4 Simple Network of Workstations SnowballC 0.7.0 Snowball Stemmers Based on the C 'libstemmer' UTF-8 Library snowfall 1.84-6.1 Easier cluster computing (based on snow). softImpute 1.4-1 Matrix Completion via Iterative Soft-Thresholded SVD sourcetools 0.1.7 Tools for Reading, Tokenizing and Parsing R Code sp 1.4-7 Classes and Methods for Spatial Data spam 2.8-0 SPArse Matrix SparseM 1.81 Sparse Linear Algebra sparseMatrixStats 1.8.0 Summary Statistics for Rows and Columns of Sparse Matrices spatial 7.3-15 Functions for Kriging and Point Pattern Analysis spatstat 2.3-4 Spatial Point Pattern Analysis, Model-Fitting, Simulation, Tests spatstat.core 2.4-4 Core Functionality of the 'spatstat' Family spatstat.data 2.2-0 Datasets for 'spatstat' Family spatstat.geom 2.4-0 Geometrical Functionality of the 'spatstat' Family spatstat.linnet 2.3-2 Linear Networks Functionality of the 'spatstat' Family spatstat.random 2.2-0 Random Generation Functionality for the 'spatstat' Family spatstat.sparse 2.1-1 Sparse Three-Dimensional Arrays and Linear Algebra Utilities spatstat.utils 2.3-1 Utility Functions for 'spatstat' spData 2.0.1 Datasets for Spatial Analysis spdep 1.2-4 Spatial Dependence: Weighting Schemes, Statistics splines 4.2.0 Regression Spline Functions and Classes SQUAREM 2021.1 Squared Extrapolation Methods for Accelerating EM-Like Monotone Algorithms stabledist 0.7-1 Stable Distribution Functions StanHeaders 2.21.0-7 C++ Header Files for Stan stars 0.5-5 Spatiotemporal Arrays, Raster and Vector Data Cubes startupmsg 0.9.6 Utilities for Start-Up Messages statmod 1.4.36 Statistical Modeling stats 4.2.0 The R Stats Package stats4 4.2.0 Statistical Functions using S4 Classes stopwords 2.3 Multilingual Stopword Lists stringdist 0.9.8 Approximate String Matching, Fuzzy Text Search, and String Distance Functions stringfish 0.15.7 Alt String Implementation stringi 1.7.6 Character String Processing Facilities stringr 1.4.0 Simple, Consistent Wrappers for Common String Operations SummarizedExperiment 1.26.1 SummarizedExperiment container survey 4.1-1 Analysis of Complex Survey Samples survival 3.3-1 Survival Analysis sva 3.44.0 Surrogate Variable Analysis symengine 0.2.2 Interface to the 'SymEngine' Library sys 3.4 Powerful and Reliable Tools for Running System Commands in R systemfit 1.1-24 Estimating Systems of Simultaneous Equations TAM 4.0-16 Test Analysis Modules tcltk 4.2.0 Tcl/Tk Interface tensor 1.5 Tensor product of arrays tensorA 0.36.2 Advanced Tensor Arithmetic with Named Indices terra 1.5-21 Spatial Data Analysis testthat 3.1.4 Unit Testing for R tgp 2.4-18 Bayesian Treed Gaussian Process Models TH.data 1.1-1 TH's Data Archive threejs 0.3.3 Interactive 3D Scatter Plots, Networks and Globes tibble 3.1.7 Simple Data Frames tictoc 1.0.1 Functions for Timing R Scripts, as Well as Implementations of Stack and List Structures tidyr 1.2.0 Tidy Messy Data tidyselect 1.1.2 Select from a Set of Strings tidyverse 1.3.1 Easily Install and Load the 'Tidyverse' timeDate 3043.102 Rmetrics - Chronological and Calendar Objects tinytex 0.39 Helper Functions to Install and Maintain TeX Live, and Compile LaTeX Documents tkrplot 0.0-26 TK Rplot tm 0.7-8 Text Mining Package tmap 3.3-3 Thematic Maps tmaptools 3.1-1 Thematic Map Tools tmvnsim 1.0-2 Truncated Multivariate Normal Simulation tools 4.2.0 Tools for Package Development topicmodels 0.2-12 Topic Models triebeard 0.3.0 'Radix' Trees in 'Rcpp' truncnorm 1.0-8 Truncated Normal Distribution trust 0.1-8 Trust Region Optimization tseries 0.10-51 Time Series Analysis and Computational Finance TTR 0.24.3 Technical Trading Rules tweenr 1.0.2 Interpolate Data for Smooth Animations TxDb.Hsapiens.UCSC.hg19.knownGene 3.2.2 Annotation package for TxDb object(s) tzdb 0.3.0 Time Zone Database Information ucminf 1.1-4.1 General-Purpose Unconstrained Non-Linear Optimization udunits2 0.13.2.1 Udunits-2 Bindings for R units 0.8-0 Measurement Units for R Vectors UpSetR 1.4.0 A More Scalable Alternative to Venn and Euler Diagrams for Visualizing Intersecting Sets urca 1.3-0 Unit Root and Cointegration Tests for Time Series Data urltools 1.7.3 Vectorised Tools for URL Handling and Parsing usethis 2.1.6 Automate Package and Project Setup utf8 1.2.2 Unicode Text Processing utils 4.2.0 The R Utils Package uuid 1.1-0 Tools for Generating and Handling of UUIDs V8 4.2.0 Embedded JavaScript and WebAssembly Engine for R VariantAnnotation 1.42.1 Annotation of Genetic Variants vctrs 0.4.1 Vector Helpers vegan 2.6-2 Community Ecology Package VGAM 1.1-6 Vector Generalized Linear and Additive Models VineCopula 2.4.4 Statistical Inference of Vine Copulas viridis 0.6.2 Colorblind-Friendly Color Maps for R viridisLite 0.4.0 Colorblind-Friendly Color Maps (Lite Version) vpc 1.2.2 Create Visual Predictive Checks vroom 1.5.7 Read and Write Rectangular Text Data Quickly vsn 3.64.0 Variance stabilization and calibration for microarray data waldo 0.4.0 Find Differences Between R Objects wateRmelon 2.2.0 Illumina 450 and EPIC methylation array normalization and metrics WGCNA 1.71 Weighted Correlation Network Analysis whisker 0.4 {{mustache}} for R, Logicless Templating widgetframe 0.3.1 'Htmlwidgets' in Responsive 'iframes' widgetTools 1.74.0 Creates an interactive tcltk widget withr 2.5.0 Run Code 'With' Temporarily Modified Global State wk 0.6.0 Lightweight Well-Known Geometry Parsing xfun 0.31 Supporting Functions for Packages Maintained by 'Yihui Xie' xgxr 1.1.1 Exploratory Graphics for Pharmacometrics XML 3.99-0.9 Tools for Parsing and Generating XML Within R and S-Plus xml2 1.3.3 Parse XML xopen 1.0.0 Open System Files, 'URLs', Anything xtable 1.8-4 Export Tables to LaTeX or HTML xts 0.12.1 eXtensible Time Series XVector 0.36.0 Foundation of external vector representation and manipulation in Bioconductor yaml 2.3.5 Methods to Convert R Data to YAML and Back zip 2.2.0 Cross-Platform 'zip' Compression zlibbioc 1.42.0 An R packaged zlib-1.2.5 zoo 1.8-10 S3 Infrastructure for Regular and Irregular Time Series (Z's Ordered Observations)"},{"location":"Other_Services/Aristotle/","title":"Aristotle","text":""},{"location":"Other_Services/Aristotle/#overview","title":"Overview","text":"

Aristotle is an interactive, Linux-based compute service for teaching, running on four nodes, each with 128 gigabytes of RAM and 16 cores. The nodes run the Red Hat Enterprise Linux operating system (RHEL 7) and have a subset of the RCPS software stack available.

The main aim of this service is to allow specific teaching courses to run that need to run Linux/UNIX applications, but it is available to all UCL users.

Warning

Aristotle is made available but is provided with minimal staff time and no budget. Any user may completely occupy the service and there is no system in place to prevent that.

"},{"location":"Other_Services/Aristotle/#access","title":"Access","text":"

Anyone with a UCL userid and within the UCL institutional firewall can access Aristotle by connecting via ssh to:

aristotle.rc.ucl.ac.uk\n

This address can point to more than one actual server (via DNS round-robin). To connect to a specific server from the set, you will need to know its number: for example, the second server has the address aristotle02.rc.ucl.ac.uk. When you connect, you should be shown which one you are connected to on your command line.

The userid and password you need to connect with are those provided to you by Information Services Division.

If you experience difficulties with your login, please make sure that you are typing your UCL user ID and your password correctly. If you still cannot get access, please contact us at rc-support@ucl.ac.uk.

If you are outside the UCL firewall, you will need to connect to the Gateway system first and then SSH in to Aristotle from there.

"},{"location":"Other_Services/Aristotle/#user-environment","title":"User Environment","text":"

Aristotle runs Red Hat Enterprise Linux 7 and NFS mounts the RCPS Software Stack. As this machine is intended for teaching, work has focused on getting specific applications required for specific courses to work and these are:

Packages are available through modules and users should consult the relevant modules documentation.

"},{"location":"Other_Services/JupyterHub/","title":"JupyterHub","text":""},{"location":"Other_Services/JupyterHub/#overview","title":"Overview","text":"

The Research Computing team provide a JupyterHub service (jupyter.org) primarily to support teaching and is part of the Data Science Platform. It uses the central UNIX filestore for user data.

"},{"location":"Other_Services/JupyterHub/#access","title":"Access","text":"

To access the service go to the following URL:

https://jupyter.data-science.rc.ucl.ac.uk/

and login with your UCL userid and password.

The Data Science Platform and all of its components are only accessible from the UCL network. When off campus, you will need to connect to the UCL VPN first or when working from China, the China Connect Service.

Please note: The main aim of this service is to support teaching and the service should not be used for computationally intensive research. If your use starts to affect the experience of other users, we reserve the right to terminate sessions without notice. For computationally intensive research you should be using the Myriad Cluster.

"},{"location":"Other_Services/RStudio/","title":"RStudio","text":""},{"location":"Other_Services/RStudio/#overview","title":"Overview","text":"

The Research Computing team currently runs 2 supported instances of RStudio:

"},{"location":"Other_Services/RStudio/#httpsrstudiodata-sciencercuclacuk","title":"https://rstudio.data-science.rc.ucl.ac.uk/","text":"

This instance is for general teaching use and is part of the Data Science Platform. It uses the central UNIX filestore for user data. Access is blocked by default due to licencing restrictions.

Staff who would like to use the service to deliver teaching should email rc-support@ucl.ac.uk to request access (please include your UCL username). To grant access to students, please pre-register the course by emailing the above address and provide the SITS code of the relevant module(s) and a pdf or link to the syllabus. Students who are registered on those SITS modules will then be added automatically.

In addition we have a smalll number of Named Researcher licenses for RStudio Pro which also run on this instance. Staff and Research Postgraduates who would like to have access to one of these licenses should email rc-support@ucl.ac.uk to request access explaining why they need to use RStudio.

The Data Science Platform and all of its components are only accessible from the UCL network. When off campus, you will need to connect to the UCL VPN first or when working from China, the China Connect Service.

Please note: The main aim of this service is to support teaching and the service should not be used for computationally intensive research. If your use starts to affect the experience of other users, we reserve the right to terminate sessions without notice. For computationally intensive research you should be using the Myriad Cluster.

"},{"location":"Other_Services/RStudio/#httpsecon-myriadrcuclacuk","title":"https://econ-myriad.rc.ucl.ac.uk/","text":"

This instance is for research use by members of the Economics department. It uses the Myriad filesystem for user data.

"},{"location":"Other_Services/RStudio/#installing-r-packages","title":"Installing R Packages","text":"

Users can install R packages in their home directories, but we recommend that if you are planning on teaching a course, you make sure the packages you want to use are pre-installed on the system. This saves time and reduces the load on the server.

The main version of R used by the RStudio server copies it's packages from the Myriad Cluster, so any package available there should be available in RStudio too. There's an automatically updated list here: R packages. Alternatively, check the list of available packages from within RStudio itself.

Requests to install packages can be sent to rc-support@ucl.ac.uk. Please give as much notice as possible when requesting packages as these requests will be handled on a best efforts basis by the research computing applications support team.

"},{"location":"Other_Services/RStudio/#troubleshooting-and-problem-pre-emption","title":"Troubleshooting and problem pre-emption","text":"

For all of the services, please take care to either run q() in the R window or press the red logout button in the top right hand corner when you are done with the window, DO NOT just close the tab. This decreases the chance of there being stale sessions and future issues with logging in.

"},{"location":"Other_Services/RStudio/#not-being-able-to-reach-the-landing-login-page","title":"Not being able to reach the landing (login) page","text":"

If you cannot reach the landing page, then please first try getting there using private browsing and if that works then clear your cookies and cache. In most browsers you can do this for a certain time range, though look at the documentation for the browser you are using.

"},{"location":"Other_Services/RStudio/#r-session-not-starting-or-rstudio-initialisation-error","title":"R session not starting or RStudio Initialisation Error","text":"

If you get an error pop-up RStudio Initialisation Error: Unable to connect to service or an ever-spinning loading screen you can try and resolve the problem using one of the methods below or get in touch with RC support.

There are 2 courses of action for the 2 supported services:

~/.local/share/rstudio/sessions/

If doing this doesn't resolve your issues, get in touch with RC support .

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/","title":"UCL UK e-Science Certificates","text":"

UCL Information Services serves as a local Registration Authority for the authentication of applications for e-Science Certificates.

A valid e-Science certificate is required to gain access to the resources of the National e-Infrastructure Service (NES) (amongst others).

Brief information to help you in applying for an e-Science Certificate is provided below. More detailed information can be found on the NGS Support website.

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#scope-of-the-ucl-registration-authority","title":"Scope of the UCL Registration Authority","text":"

In general, the UCL Registration Authority (RA) can only approve personal and server certificate requests for members of UCL and those associated with projects based at UCL. However we have approved personal certificate requests for members of other London institutions without local RAs on request.

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#before-you-apply-for-a-certificate","title":"Before you Apply for a Certificate","text":"

The recommended method for applying for a certificate is to use the Certificate Wizard. So:

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#applying-for-a-certificate","title":"Applying for a Certificate","text":"

You will be asked for a number of items when completing your request including:

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#extra-items-for-server-certificates","title":"Extra items for Server Certificates","text":"

There are two extra items for certificates for servers:

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#after-your-request-has-been-submitted","title":"After Your Request has been Submitted","text":"

After you have submitted your request, it has to be authenticated by your Registration Authority (RA) before the certificate is issued by the UK e-Science Certification Authority (CA). For authentication the UK e-Science CA requires that you present yourself in person to your RA with an appropriate form of photo-ID and your PIN. You will be asked to explain why you need a UK e-Science Certificate.

The RA for UCL is based in Informations Services Division (ISD). To arrange an appointment please email grid-ra AT ucl.ac.uk in the first instance.

Valid forms of Photo-ID are any of the following:

We are required to make and log a photocopy of your photo-ID.

If you have none of the above forms of photo-ID available, contact us for advice by e-mail at grid-ra AT ucl.ac.uk. Please don't just turn up with an alternative as we may not be able to accept it.

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#extra-requirements-for-students","title":"Extra Requirements for Students","text":"

In addition to the above, students should provide a letter (on department paper) from their project supervisor explaining why they need a certificate.

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#extra-requirements-for-servers","title":"Extra Requirements for Servers","text":"

In addition to the above, you need to provide a letter from your department explaining that you are responsible for this server. The letter should be on departmental stationary and be signed by your head of department.

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#obtaining-your-certificate","title":"Obtaining Your Certificate","text":"

After your request has been authenticated by your Registration Authority, it is forwarded to the UK e-Science Certification Authority for final creation (this stage is called signing the certificate). Signing is normally done on the same or next working day.

When your certificate is ready the CA will e-mail you using the e-mail address that you provided with details of how to download your certificate. If you used the recommend method to request it, then you can download it into the Certificate Wizard application using the Refresh button.

You should now make a backup of your certificate using the Export button in the Certificate Wizard application.

"},{"location":"Paid-For_Resources/How_to_Use/","title":"Using Paid-For Resources","text":"

Paid resources may be in the form of priority access (Gold), dedicated nodes or both.

Users with access to Gold have tools for monitoring budgets and usage.

Users with access to dedicated nodes have a tool for monitoring current use of their nodes.

"},{"location":"Paid-For_Resources/How_to_Use/#why-we-recommend-priority-access","title":"Why we recommend priority access","text":"

We recommend priority access via Gold rather than dedicated nodes in most circumstances because:

"},{"location":"Paid-For_Resources/How_to_Use/#priority-access-via-gold","title":"Priority access via Gold","text":"

If you have priority access, this is managed by a resource called Gold. Gold is divided into allocations, which have start and end dates and a given amount of Gold associated with them. Any unused Gold expires after the allocation period ends.

On Myriad, one Gold = one core hour. If you run a job that asks for a wallclock time of 3hrs and 10 cores, then the job costs 30 Gold.

Gold is reserved when you submit a job, so the entire 30 will be reserved and taken away from the available balance when you run qsub. After the job ends, how long it ran for is checked, and any unused Gold is put back for anyone to use.

For example, you asked for 3 hours and 10 cores, but your job finished in 2 hours. When you submit the job, it will reserve 30 Gold and your budget will go down by 30. When it ends, the final charge is only 20 Gold, so 10 Gold gets put back in your available budget.

"},{"location":"Paid-For_Resources/How_to_Use/#view-your-gold-budgets","title":"View your Gold budgets","text":"

To see the Gold budgets available to you, run:

budgets\n

You will see something like this:

Project    Machines  Balance\n---------  --------  -----------\nhpc.999     ANY       124560.00\nhpc.998     ANY       0.00\n

The project column shows which budgets you have access to and the balance shows how much is left unused in the current allocation.

"},{"location":"Paid-For_Resources/How_to_Use/#jobscript-additions-for-gold-jobs","title":"Jobscript additions for Gold jobs","text":"

You choose whether you want a specific job to be a Gold job or a normal priority job. For a Gold job, add these to your jobscript:

#$ -P Gold\n#$ -A hpc.xx\n

You can also pass these in on the command line to the qsub and qrsh commands: 

qsub -P Gold -A hpc.xx myscript.sh\n
"},{"location":"Paid-For_Resources/How_to_Use/#viewing-allocation-dates","title":"Viewing allocation dates","text":"

You can look at all the start and end dates for your allocations:

glsalloc -p hpc.xx\n

Output will look like this:

Id  Account Projects StartTime  EndTime    Amount    Deposited Description    \n--- ------- -------- ---------- ---------- --------- --------- -------------- \n001 01      hpc.999  -infinity  infinity        0.00      0.00 Auto-Generated \n002 01      hpc.999  2021-12-01 2022-03-01 105124.00 205124.00                \n003 01      hpc.999  2022-03-01 2022-06-01 205124.00 205124.00                \n004 01      hpc.999  2022-06-01 2022-09-01 205124.00 205124.00 \n

Allocations begin and end at approximately 00:05 on the date mentioned.

"},{"location":"Paid-For_Resources/How_to_Use/#monitoring-gold-usage","title":"Monitoring Gold usage","text":"

You can view some information about when your Gold was used, in which jobs, and by whom.

# show help\ngstatement --man\n\n# show statement between the given dates\ngstatement -p hpc.xx -s 2020-12-01 -e 2021-12-01\n\n# give a summary between the given dates\ngstatement -p hpc.xx -s 2020-12-01 -e 2021-12-01 --summarize\n
"},{"location":"Paid-For_Resources/How_to_Use/#dedicated-nodes","title":"Dedicated nodes","text":"

For dedicated nodes, only members of your project are allowed to run jobs on your node. Your project is usually set by default so you do not need to specify it in your jobscript. You can check this by looking at qstat -j $JOB_ID for an existing job ID, and looking at the project line near the bottom.

"},{"location":"Paid-For_Resources/How_to_Use/#jobscript-additions-for-dedicated-nodes","title":"Jobscript additions for dedicated nodes","text":"

If the project is not being set by default, for a job to be eligible to run on your nodes you will need to specify your project in your jobscript:

#\u00a0Specify\u00a0project\n#$\u00a0-P\u00a0<project>\n

This will allow a job to run on your nodes, but it can also be scheduled on general-use nodes if some are available first. This should be the main way you run jobs.

If you need to, you can force jobs to run on your nodes only. This is suitable when you have arranged policies on your nodes that are different from the normal policies (eg. a longer maximum wallclock time), as it means your policies will be in effect instead of the general policies.

#\u00a0Specify\u00a0paid\u00a0flag\u00a0to\u00a0force\u00a0running\u00a0on\u00a0paid\u00a0nodes\u00a0only,\u00a0with\u00a0your\u00a0policies\n#$\u00a0-l\u00a0paid=1\n
"},{"location":"Paid-For_Resources/How_to_Use/#check-what-is-running-on-your-nodes","title":"Check what is running on your nodes","text":"

We have a script named whatsonmynode, that runs qhost -j for all the nodes belonging to your project, so you can see which nodes you have, what is running on them and from which user.

module\u00a0load\u00a0userscripts\nwhatsonmynode\u00a0<project>\n
"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/","title":"Purchasing in Myriad","text":"

Sept 2023: We cannot buy any nodes or add paid time in Myriad for the time being.

This is because we have no more space in the datacentre to add new hardware, which is needed for priority access/Gold as well as dedicated node purchases.

Please raise your requests with rc-support@ucl.ac.uk but be aware that we cannot confirm our ability to fulfill them until we know what future space we will have available.

Researchers may purchase additional resources to be used as part of the Myriad High Performance Computing cluster if the free service does not meet their needs. These resources can be made available in one of two ways:

Costs will include backend infrastructure \u2013 racks, switches, cables etc \u2013 needed to integrate the compute nodes into the facility. Both options can be costed into research proposals. These costs will be stated in an agreed Statement of Work document giving details of the number of each node type to be purchased, which must be approved by ARC and the purchaser before the purchase can go ahead.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#deadline","title":"Deadline","text":"

The deadline for making requests to purchase Myriad nodes is the end of April. Before this date, researchers wishing to buy compute nodes must approve an agreed Statement of Work and transfer the funds to ARC. We can't place orders after this date because the hardware might not arrive before the end of UCL's financial year.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#faqs","title":"FAQs","text":""},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#can-i-add-the-cost-of-purchasing-nodes-to-a-grant-application","title":"Can I add the cost of purchasing nodes to a grant application?","text":"

If you are putting together a grant application and think that you may need to ask for the cost of additional computing resources to be covered, please contact us. We will be able to assess your requirements, recommend appropriate hardware, and then provide an estimate and a supporting statement.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#can-you-give-me-advice-on-what-hardware-to-buy","title":"Can you give me advice on what hardware to buy?","text":"

Yes, we\u2019d be happy to discuss this with you. Please contact rc-support@ucl.ac.uk.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#what-type-of-nodes-can-we-purchase","title":"What type of nodes can we purchase?","text":"

We currently have three types of node in Myriad, which are well tested and work reliably. These are:

If you require an alternative/custom specification, we can\u2019t guarantee that we will be able to accommodate this on the cluster, but we\u2019re happy to look into it.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#i-want-to-apply-for-more-than-50k-worth-of-equipment-will-we-have-to-go-through-a-tender-process","title":"I want to apply for more than \u00a350k worth of equipment, will we have to go through a tender process?","text":"

No. We have a framework agreement with the vendor which covers all hardware purchases.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#i-know-what-hardware-i-need-can-you-send-me-a-quote","title":"I know what hardware I need. Can you send me a quote?","text":"

Even if you know what hardware you need, before we can send you a quote, we will need to agree on a detailed specification. Please email rc-support@ucl.ac.uk with the following information:

We will then either send you a specification to approve, or ask to meet to discuss your requirements further. Once this is agreed, we aim to get back to you with a quote within two weeks.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#how-do-i-manage-who-has-permission-to-use-our-nodespriority-queue","title":"How do I manage who has permission to use our nodes/priority queue?","text":"

When you purchase nodes or priority cycles, we will ask you for a list of usernames of people who have permission to use the resource \u2014 access is managed using access control lists on Myriad. If your list of users is an entire department, we can automatically generate this list nightly. Resource owners or designated resource administrators can request a change of membership of these groups by submitting a ticket in Remedy Force or emailing rc-support@ucl.ac.uk.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#what-is-the-difference-between-paying-for-priority-cycles-and-purchasing-dedicated-nodes","title":"What is the difference between paying for priority cycles and purchasing dedicated nodes?","text":"

Priority cycles is the better option for most people as it provides greater flexibility: priority cycles can be used across many nodes at once, and there is an entire allocation period to use them. Dedicated hardware however would need to be in use 24/7 in order to get the most out of it. Researchers might want dedicated nodes if they have particular requirements which mean they can only run their work on their own nodes; e.g., they have purchased non-standard nodes, or the software they are using requires a static licence tied to a particular node.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#will-my-3-month-priority-cycle-allocation-roll-over-to-the-next-quarter-if-i-dont-use-it","title":"Will my 3-month priority cycle allocation roll over to the next quarter if I don\u2019t use it?","text":"

No.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#i-want-the-flexibility-of-priority-cycles-but-my-funder-requires-an-invoice-for-specific-hardware","title":"I want the flexibility of priority cycles, but my funder requires an invoice for specific hardware.","text":"

Even if you require an invoice for specific hardware, you can still convert the physical hardware into priority cycles. We will add the nodes you purchased to the general pool and give you the equivalent in priority cycles.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#what-happens-to-nodes-i-have-purchased-once-theyve-reached-the-end-of-warranty","title":"What happens to nodes I have purchased once they\u2019ve reached the end of warranty?","text":"

The hardware is run for the life-time of the service, however if you buy dedicated nodes and they run out of warranty, they will not be replaced.

"},{"location":"Software_Guides/ANSYS/","title":"ANSYS","text":"

License checking

Current as of Dec 2021, the -ac app=cfx license check does not work after ANSYS renamed all products and licenses. Remove this line from your jobscripts for the time being or jobs will not be able to start.

ANSYS/CFX and ANSYS/Fluent are commercial fluid dynamics packages.

The most recent version of ANSYS we have installed on the clusters is ANSYS 2021.R2 including ANSYS/CFX, ANSYS/Fluent, ANSYS Mechanical and the ANSYS Electromagnetics Suite (AnsysEM). Some other products included in the ANSYS Campus Agreement are also available including Autodyn.

Older versions of some ANSYS products including ANSYS/CFX and ANSYS/Fluent are also available - 2019.R3, 19.1 for example. However ANSYS Inc changed the way they license their software at the beginning of 2021, causing some products from versions before 2020 to have issues getting a valid license from the lciense server.

Before these applications can be run, the user needs to go though a number of set up steps. These are detailed here.

To see the versions available, type

module avail ansys\n

The desired ANSYS module needs to be loaded by issuing a command like:

module\u00a0load\u00a0ansys/2021.r2\n

This will set up various necessary config directories for you.

Module for UCL staff and Students only

these ANSYS modules are for UCL staff and Students only as they use the UCL ANSYS License Manager. Users from external Institutions on Young (the MMM Hub cluster) need to use modules specific to their institution instead.

The ANSYS applications are intended to be run primarily within batch jobs however you may run short (less than 5 minutes execution time) interactive tests on the Login Nodes and longer (up to two hours) on an interactive session on a compute node using qrsh. Interactive work can be done using the ANSYS interactive tools provided you have X-windows functionality enabled though your ssh connection. See our User Guide for more information about enabling X-windows functionality and using the qrsh command to start interactive sessions.

UCL's campus-wide license covers 125 instances with 512 HPC licenses (for parallel jobs) available for running CFX, Fluent and AnsysEM jobs and in order to make sure that jobs only run if there are licenses available, it is necessary for users to request ANSYS licenses with their jobs, by adding -ac app=cfx to their job submission.

"},{"location":"Software_Guides/ANSYS/#ansyscfx","title":"ANSYS/CFX","text":"

CFX handles its own parallelisation, so a number of complex options need to be passed in job scripts to make it run correctly.

"},{"location":"Software_Guides/ANSYS/#example-single-node-multi-threaded-ansyscfx-jobscript","title":"Example single node multi-threaded ANSYS/CFX jobscript","text":"

Here is an example runscript for running cfx5solve multi-threaded on a given .def file.

#!/bin/bash -l\n\n# ANSYS 2021.R2: Batch script to run cfx5solve on the StaticMixer.def example \n# file, single node multi-threaded (12 threads),\n\n# Force bash as the executing shell.\n\n#$ -S /bin/bash\n\n# Request 15 munutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:15:0\n\n# Request 1 gigabyte of RAM per core.\n#$ -l mem=1G\n\n# Set the name of the job.\n#$ -N StaticMixer_thread_12\n\n# Select 12 threads.\n#$ -pe smp 12\n\n# Request ANSYS licences\n# Dec 2021: comment out this check as not currently working\n###$ -ac app=cfx\n\n# Set the working directory to somewhere in your scratch space. In this\n# case the subdirectory cfxtests-2021.R2\n#$ -wd /home/<your_UCL_id>/Scratch/cfxtests-2021.R2\n\n# Load the ANSYS module to set up your environment\n\nmodule load ansys/2021.r2\n\n# Copy the .def file into the working (current) directory\n\ncp /home/<your userid>/cfx_examples/StaticMixer.def .\n\n# Run cfx5solve - Note: -max-elapsed-time needs to be set to the same\n# time as defined by 2 above.\n\ncfx5solve -max-elapsed-time \"15 [min]\" -def StaticMixer.def -par-local -partition $OMP_NUM_THREADS\n

You will need to change the -wd /home/<your_UCL_id>/Scratch/cfxtests-2021.R2 location and may need to change the memory, wallclock time, number of threads and job name directives as well. Replace the .def file with your one and modify the -max-elapsed-time value if needed. The simplest form of qsub command can be used to submit the job eg:

qsub run-StaticMixer-thr.sh\n

Output files will be saved in the job's working directory.

"},{"location":"Software_Guides/ANSYS/#example-multi-node-mpi-ansyscfx-jobscript","title":"Example multi-node MPI ANSYS/CFX jobscript","text":"

Here is an example runscript for running cfx5solve on more than one node (using MPI) on a given .def file. 

#!/bin/bash -l\n\n# ANSYS 2021.R2: Batch script to run cfx5solve on the StaticMixer.def example \n# file, distributed parallel (80 cores).\n\n# Using ANSYS 2021 licence manager running on UCL central licence server.\n\n# 1. Force bash as the executing shell.\n#$ -S /bin/bash\n\n# 2. Request one hour of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:60:0\n\n# 3. Request 2 gigabyte of RAM per core.\n#$ -l mem=2G\n\n# 4. Set the name of the job.\n#$ -N StaticMixer_P_dist_80_NLC\n\n# 5. Select the MPI parallel environment and 80 processors.\n#$ -pe mpi 80\n\n# 6. Request ANSYS licences $ not inserted so currently inactive. Job will queue until\n# suficient licenses are available when active.\n# -ac app=cfx\n\n# 7. Set the working directory to somewhere in your scratch space.  In this\n# case the subdirectory cfxtests-18.0\n#$ -wd /home/<your_UCL_id>/Scratch/cfxtests-2021.R2\n\n# 8. Load the ANSYS module to set up your environment\n\nmodule load ansys/2021.r2\n\n# 9. Copy the .def file into the working (current) directory\n\ncp /home/<your_UCL_id>/Software/ANSYS/cfx_examples/StaticMixer.def .\n\n# 10. SGE puts the machine file in $TMPDIR/machines. Use this to generate the \n# string CFX_NODES needed by cfx5solve\n\nexport CFX_NODES=`cfxnodes $TMPDIR/machines`\n\n# 11. Run cfx5solve - Note: -max-elapsed-time needs to be set to the same\n# time as defined by 2 above.\n\n# Run with default MPI.\ncfx5solve -max-elapsed-time \"60 [min]\" -def StaticMixer.def -par-dist $CFX_NODES\n

Please copy if you wish and edit it to suit your jobs. You will need to change the -wd /home/<your_UCL_userid>/Scratch/cfxtests-19.1 location and may need to change the memory, wallclock time, number of MPI processors and job name directives as well. Replace the .def file with your one and modify the -max-elapsed-time value if needed. The simplest form of qsub command can be used to submit the job eg:

qsub\u00a0run-StaticMixer-par.sh\n

Output files will be saved in the job's working directory.

"},{"location":"Software_Guides/ANSYS/#running-cfx-with-mpi-on-myriad","title":"Running CFX with MPI on Myriad","text":"

The default supplied Intel MPI doesn't work on Myriad. Instead you need to use the supplied IBM MPI. This can be done by adding: -start-method \"IBM MPI Distributed Parallel\" to the cfx5solve command. Also the maximum number of MPI processors you can request is 36.

"},{"location":"Software_Guides/ANSYS/#troubleshooting-cfx","title":"Troubleshooting CFX","text":"

If you are getting licensing errors when trying to run a parallel job and you have an older version's ~/.ansys/v161/licensing/license.preferences.xml file, delete it. It does not work with the newer license server. (This applies to all older versions, not just v161).

"},{"location":"Software_Guides/ANSYS/#ansysfluent","title":"ANSYS/Fluent","text":"

Fluent handles its own parallelisation, so a number of complex options need to be passed in job scripts to make it run correctly.

The .in file mentioned in the scripts below is a Fluent journal file, giving it the list of commands to carry out in batch mode.

"},{"location":"Software_Guides/ANSYS/#example-serial-ansysfluent-jobscript","title":"Example serial ANSYS/Fluent jobscript","text":"

Here is an example jobscript for running Fluent in serial mode (1 core).

#!/bin/bash -l\n\n# ANSYS 2021.R2: Batch script to run ANSYS/fluent in serial mode \n# (1 core). \n\n# Request 2 hours of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=2:0:0\n\n# Request 2 gigabytes of RAM.\n#$ -l mem=2G\n\n# Set the name of the job.\n#$ -N Fluent_ser1\n\n# Request ANSYS licences\n# Dec 2021: comment out this check as not currently working\n###$ -ac app=cfx\n\n# Set the working directory to somewhere in your scratch space.  In this\n# case the subdirectory fluent-tests-19.1\n#$ -wd /home/<your_UCL_userid>/Scratch/fluent-tests-2021.R2\n\n# Load the ANSYS module to set up your environment\n\nmodule load ansys/2021.r2\n\n# Copy Fluent input files into the working (current) directory\n\ncp <path to your input files>/test-1.cas .\ncp <path to your input files>/test-1.in .\n\n# Run fluent in 2D single precision (-g no GUI). For double precision use\n# 2ddp. For 3D use 3d or 3ddp. \n\nfluent 2d -g < test-1.in\n

Please copy if you wish and edit it to suit your jobs. You will need to change the -wd /home/<your_UCL_id>/Scratch/fluent-tests-2021.R2 location and may need to change the memory, wallclock time, and job name as well. Replace the .cas and .in files with your ones. The simplest form of qsub command can be used to submit the job eg:

qsub\u00a0run-ANSYS-fluent-ser.sh\n

Output files will be saved in the job's working directory.

"},{"location":"Software_Guides/ANSYS/#example-parallel-mpi-ansysfluent-jobscript","title":"Example parallel (MPI) ANSYS/Fluent jobscript","text":"

Here is an example runscript for running Fluent in parallel potentially across more than one node.

#!/bin/bash -l\n\n# ANSYS 2021.R2: Batch script to run ANSYS/fluent distributed parallel \n# (80 cores). \n\n# Request 2 hours of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=2:0:0\n\n# Request 2 gigabytes of RAM per core.\n#$ -l mem=2G\n\n# Set the name of the job.\n#$ -N Fluent_par80\n\n# Select the MPI parallel environment and 80 processors.\n#$ -pe mpi 80\n\n# Request 25 Gb TMPDIR space (if on a cluster that supports this)\n#$ -l tmpfs=25G\n\n# Request ANSYS licences\n# Dec 2021: comment out this check as not currently working\n###$ -ac app=cfx\n\n# Set the working directory to somewhere in your scratch space.  In this\n# case the subdirectory fluent-tests-19.1\n#$ -wd /home/<your_UCL_userid>/Scratch/fluent-tests-2021.R2\n\n# Load the ANSYS module to set up your environment\n\nmodule load ansys/2021.r2\n\n# Copy Fluent input files into the working (current) directory\n\ncp <path to your input files>/test-1.cas .\ncp <path to your input files>/test-1.in .\n\n# Run fluent  in 3D single precision (-g no GUI). For double precision use\n# 3ddp. For 2D use 2d or 2ddp. \n# Do not change -t, -mpi, -pinfiniband and -cnf options.\n\nfluent 3ddp -t$NSLOTS -mpi=intelmpi -cnf=$TMPDIR/machines -g < test-1.in\n

Please copy if you wish and edit it to suit your jobs. You will need to change the -wd /home/<your_UCL_id>/Scratch/fluent-tests-2021.r2 location and may need to change the memory, wallclock time, number of MPI processors and job name as well. Replace the .cas and .in files with your ones. The simplest form of qsub command can be used to submit the job eg: 

qsub\u00a0run-ANSYS-fluent-par-80.sh\n

Output files will be saved in the job's working directory.

If you want to use IBM Platform MPI instead of Intel MPI, then replace the -mpi=intelmpi with -mpi=ibmmpi.

"},{"location":"Software_Guides/ANSYS/#troubleshooting-fluent","title":"Troubleshooting Fluent","text":"

If you are getting licensing errors when trying to run a parallel job and you have an older version's ~/.ansys/v161/licensing/license.preferences.xml file, delete it. It does not work with the newer license server. (This applies to all older versions, not just v161).

Fluent 14 required -mpi=pcmpi -pinfiniband in the parallel options: if you have older scripts remember to remove this.

"},{"location":"Software_Guides/ANSYS/#ansys-mechanical","title":"ANSYS Mechanical","text":"

ANSYS Mechanical handles its own parallelisation, and needs an additional setting to work on our clusters. It also only appears to work with Intel MPI. Here is an example jobscript for running in parallel potentially across more than one node, for example on the Kathleen cluster.

#!/bin/bash -l\n\n# ANSYS 2021.R2: Batch script to run ANSYS Mechanical solver\n# file, distributed parallel (80 cores).\n\n# Using ANSYS 2021.R2 licence manager running on UCL central licence server.\n\n# Force bash as the executing shell.\n#$ -S /bin/bash\n\n# Request one hour of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=1:00:0\n\n# Request 2 gigabyte of RAM per core. (Must be an integer)\n#$ -l mem=2G\n\n# Set the name of the job.\n#$ -N Mech_P_dist_80\n\n# Select the MPI parallel environment and 80 processors.\n#$ -pe mpi 80\n\n# Request ANSYS licences $ inserted so currently active.Job will queue until\n# suficient licences are available.\n# Dec 2021: comment out this check as not currently working\n###$ -ac app=cfx\n\n# Set the working directory to somewhere in your scratch space.  In this\n# case the subdirectory ANSYS_Mech\n#$ -wd /home/<your_UCL_username>/Scratch/ANSYS_Mech\n\n# Load the ANSYS module to set up your environment\nmodule load ansys/2021.r2\n\n# Copy the .in file into the working (current) directory\ncp ~/ANSYS/steady_state_input_file.dat .\n\n# 10. SGE puts the machine file in $TMPDIR/machines. Use this to generate the \n# string CFX_NODES needed by ansys195 which requires : as the separator.\n\nexport CFX_NODES=`cfxnodes_cs $TMPDIR/machines`\necho $CFX_NODES\n\n# Need to set KMP_AFFINTY to get round error: OMP: System error #22: Invalid argument\n\nexport KMP_AFFINITY=disabled\n\n# Run ansys mechanical - Note: use ansys195 instead of ansys and -p argument\n# needed to switch to a valid UCL license.\n\nansys212 -dis -mpi intelmpi -machines $CFX_NODES -b < steady_state_input_file.dat\n

Please copy if you wish and edit it to suit your jobs. You will need to change the -wd /home/<your_UCL_id>/Scratch/ANSYS_Mech location and may need to change the memory, wallclock time, number of MPI processors and job name as well. Replace the .dat file with your one. The simplest form of qsub command can be used to submit the job eg: 

qsub\u00a0ansys-mech-2021.R2-ex.sh\n

Output files will be saved in the job's working directory.

If you have access to Kathleen, this test input and jobscript are available at /home/ccaabaa/Software/ANSYS/steady_state_input_file.dat and /home/ccaabaa/Software/ANSYS/ansys-mech-2021.R2-ex.sh

"},{"location":"Software_Guides/ANSYS/#ansys-electromagnetic-suite-ansysem","title":"ANSYS Electromagnetic Suite (AnsysEM)","text":"

The AnsysEM products handle their own parallelisation so a number of complex options need to be passed in job scripts to make it run correctly. Also additional module commands are required.

Here is an example jobscript for running in parallel potentially across more than one node, for example on the Kathleen cluster.

#!/bin/bash -l\n\n# AnsysEM 2021 R2: Batch script to run one of the Ansys Electromagnetics Products\n# example simulations on Kathleen - distributed parallel (80 cores)\n\n# 1. Force bash as the executing shell.\n#$ -S /bin/bash\n\n# 2. Request one hour of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=1:00:0\n\n# 3. Request 2 gigabyte of RAM per core.\n#$ -l mem=2G\n\n# 4. Set the name of the job.\n#$ -N DiffSL_P_dist_80\n\n# 5. Select the MPI parallel environment and 80 processors - two nodes.\n#$ -pe mpi 80\n\n# 7. Request ANSYS licences $ inserted so currently active. Job will queue until\n# suficient licenses are available. Not currently active.\n# -ac app=cfx\n\n# 8. Set the working directory to somewhere in your scratch space.  In this\n# case the directory the job was submitted from,\n#$ -cwd\n\n# 9. Load the ANSYS module to set up your environment\n\nmodule load ansys/2021.r2\nmodule load xorg-utils/X11R7.7\nmodule load giflib/5.1.1\n\n# 10. Run ansysedt\n\nansysedt -ng -Distributed -machinelist num=$NSLOTS -batchoptions 'HFSS/MPIVendor'='Intel' -batchoptions 'TempDirectory'=\"${TMPDIR}\" -batchoptions 'HFSS/HPCLicenseType'='pool'  -batchoptions 'HFSS-IE/HPCLicenseType'='pool' -BatchSolve differential_stripline.aedt\n
"},{"location":"Software_Guides/Installing_Software/","title":"Installing Software","text":""},{"location":"Software_Guides/Installing_Software/#installing-software","title":"Installing software","text":"

If you want to request that software be installed centrally, you can email us at rc-support@ucl.ac.uk. When you send in a request please address the following questions so that the install can be properly prioitised and planned,

The requests will be added to the issues in our buildscripts repository. The buildscripts themselves are there too, so you can see how we built and installed our central software stack.

You can install software yourself in your space on the cluster. Below are some tips for installing packages for languages such as Python or Perl as well as compiling software.

"},{"location":"Software_Guides/Installing_Software/#no-sudo","title":"No sudo!","text":"

You cannot install anything using sudo (and neither can we!). If the instructions tell you to do that, read further to see if they also have instructions for installing in user space, or for doing an install from source if they are RPMs.

Alternatively, just leave off the sudo from the command they tell you to run and look for an alternative way to give it an install location if it tries to install somewhere that isn't in your space (examples for some common build systems are below).

"},{"location":"Software_Guides/Installing_Software/#download-source-code","title":"Download source code","text":"

Use wget or curl to download the source code for the software you want to install to your account on the cluster. You can use tar to extract source archives named like tar.gz or .tgz or .tar.bz2 among others and unzip for .zip files. xz --decompress will expand .xz files.

wget https://www.example.com/program.tar.gz\ntar -xvf program.tar.gz\n

You will not be able to use a package manager like yum, and will need to follow the manual installation instructions for a user-space install (not using sudo).

"},{"location":"Software_Guides/Installing_Software/#set-up-modules","title":"Set up modules","text":"

Before you start compiling, you need to make sure you have the right compilers, libraries and other tools available for your software. If you haven't changed anything, you will have the default modules loaded.

Check what the instructions for your software tell you about compiling it. If the website doesn't say much, the source code will hopefully have a README or INSTALL file.

You may want to use a different compiler - the default is the Intel compiler.

module avail compilers will show you all the compiler modules available. Most Open Source software tends to assume you're using GCC and OpenMPI (if it uses MPI) and is most tested with that combination, so if it doesn't tell you otherwise you may want to begin there (do check what the newest modules available are - the below is correct at time of writing):

# unload your current compiler and mpi modules\nmodule unload -f compilers mpi\n# load the GNU compiler\nmodule load compilers/gnu/4.9.2\n\n# these three modules are only needed on Myriad\nmodule load numactl/2.0.12\nmodule load binutils/2.29.1/gnu-4.9.2\nmodule load ucx/1.8.0/gnu-4.9.2\n\n# load OpenMPI\nmodule load mpi/openmpi/4.0.3/gnu-4.9.2\n
"},{"location":"Software_Guides/Installing_Software/#newer-versions-of-gcc-and-glibcxx","title":"Newer versions of GCC and GLIBCXX","text":"

The software you want to run may require newer compilers or a precompiled binary may say that it needs a newer GLIBCXX to be able to run. You can access these as follows:

# make all the newer versions visible\nmodule load beta-modules\n# unload current compiler, mpi and gcc-libs modules\nmodule unload -f compilers mpi gcc-libs\n# load GCC 10.2.0\nmodule load gcc-libs/10.2.0\nmodule load compilers/gnu/10.2.0\n

The gcc-libs module contains the actual compiler and libraries, while the compilers/gnu module sets environment variables that are likely to be picked up by build systems, telling them what the C, C++ and Fortran compilers are called.

"},{"location":"Software_Guides/Installing_Software/#glibc-version-error","title":"GLIBC version error","text":"

If you get an error saying that a precompiled binary that you are installing needs a newer GLIBC (not GLIBCXX) then this has been compiled on a newer operating system and will not work on our clusters. Look for a binary that was created for CentOS 7 (we have RHEL 7) or build the program from source if possible.

"},{"location":"Software_Guides/Installing_Software/#build-systems","title":"Build systems","text":"

Most software will use some kind of build system to manage how files are compiled and linked and in what order. Here are a few common ones.

"},{"location":"Software_Guides/Installing_Software/#automake-configure","title":"Automake configure","text":"

Automake will generate the Makefile for you and hopefully pick up sensible options through configuration. You can give it an install prefix to tell it where to install (or you can build it in place and not use make install at all).

./configure\u00a0--prefix=/home/username/place/you/want/to/install\nmake\n#\u00a0if\u00a0it\u00a0has\u00a0a\u00a0test\u00a0suite,\u00a0good\u00a0idea\u00a0to\u00a0use\u00a0it\nmake\u00a0test\u00a0\nmake\u00a0install\n

If it has more configuration flags, you can use ./configure --help to view them.

Usually configure will create a config.log: you can look in there to find if any tests have failed or things you think should have been picked up haven't.

"},{"location":"Software_Guides/Installing_Software/#cmake","title":"CMake","text":"

CMake is another build system. It will have a CMakeFile or the instructions will ask you to use cmake or ccmake rather than make. It also generates Makefiles for you. ccmake is a terminal-based interactive interface where you can see what variables are set to and change them, then repeatedly configure until everything is correct, generate the Makefile and quit. cmake is the commandline version. The interactive process tends to go like this:

ccmake\u00a0CMakeLists.txt\n#\u00a0press\u00a0c\u00a0to\u00a0configure\u00a0-\u00a0will\u00a0pick\u00a0up\u00a0some\u00a0options\n#\u00a0press\u00a0t\u00a0to\u00a0toggle\u00a0advanced\u00a0options\n#\u00a0keep\u00a0making\u00a0changes\u00a0and\u00a0configuring\u00a0until\u00a0no\u00a0more\u00a0errors\u00a0or\u00a0changes\n#\u00a0press\u00a0g\u00a0to\u00a0generate\u00a0and\u00a0exit\nmake\n#\u00a0if\u00a0it\u00a0has\u00a0a\u00a0test\u00a0suite,\u00a0good\u00a0idea\u00a0to\u00a0use\u00a0it\nmake\u00a0test\u00a0\nmake\u00a0install\n

The options that you set using ccmake can also be passed on the commandline to cmake with -D. This allows you to script an install and run it again later. CMAKE_INSTALL_PREFIX is how you tell it where to install.

# making a build directory allows you to clean it up more easily\nmkdir build\ncd build\ncmake .. -DCMAKE_INSTALL_PREFIX=/home/username/place/you/want/to/install\n

If you need to rerun cmake/ccmake and reconfigure, remember to delete the CMakeCache.txt file first or it will still use your old options. Turning on verbose Makefiles in cmake is also useful if your code didn't compile first time - you'll be able to see what flags the compiler or linker is actually being given when it fails.

"},{"location":"Software_Guides/Installing_Software/#make","title":"Make","text":"

Your code may come with a Makefile and have no configure, in which case the generic way to compile it is as follows:

make\u00a0targetname\n

There's usually a default target, which make on its own will use. make all is also frequently used. If you need to change any configuration options, you'll need to edit those sections of the Makefile (usually near the top, where the variables/flags are defined).

Here are some typical variables you may want to change in a Makefile.

These are what compilers/mpi wrappers to use - these are also defined by the compiler modules, so you can see what they should be. Intel would be icc, icpc, ifort, while the GNU compiler would be gcc, g++, gfortran. If this is a program that can be compiled using MPI and only has a variable for CC, then set that to mpicc.

CC=gcc\nCXX=g++\nFC=gfortran\nMPICC=mpicc\nMPICXX=mpicxx\nMPIF90=mpif90\n

CFLAGS and LDFLAGS are flags for the compiler and linker respectively, and there might be LIBS or INCLUDE in the Makefile as well. When linking a library with the name libfoo, use -lfoo.

CFLAGS=\"-I/path/to/include\"\nLDFLAGS=\"-L/path/to/foo/lib\u00a0-L/path/to/bar/lib\"\nLDLIBS=\"-lfoo\u00a0-lbar\"\n

Remember to make clean first if you are recompiling with new options. This will delete object files from previous attempts.

"},{"location":"Software_Guides/Installing_Software/#blas-and-lapack","title":"BLAS and LAPACK","text":"

BLAS and LAPACK are linear algebra libraries that are provided as part of MKL, OpenBLAS or ATLAS. There are several different OpenBLAS and ATLAS modules for different compilers. MKL is available as part of each Intel compiler module.

Your code may try to link -lblas -llapack: this isn't the right way to use BLAS and LAPACK with MKL or ATLAS (though our OpenBLAS now has symlinks that mean this will work).

"},{"location":"Software_Guides/Installing_Software/#mkl","title":"MKL","text":"

When you have an Intel compiler module loaded, typing 

echo $MKLROOT\n

will show you that MKL is available.

"},{"location":"Software_Guides/Installing_Software/#easy-linking-of-mkl","title":"Easy linking of MKL","text":"

If you can, try to use -mkl as a compiler flag - if that works, it should get all the correct libraries linked in the right order. Some build systems do not work with this however and need explicit linking.

"},{"location":"Software_Guides/Installing_Software/#intel-mkl-link-line-advisor","title":"Intel MKL link line advisor","text":"

It can be complicated to get the correct link line for MKL, so Intel has provided a tool which will give you the link line with the libraries in the right order.

Pick the version of MKL you are using (for the Intel 2018 compiler it should be Intel(R) MKL 2018.0), and these options:

You'll get something like this:

${MKLROOT}/lib/intel64/libmkl_blas95_ilp64.a ${MKLROOT}/lib/intel64/libmkl_lapack95_ilp64.a -L${MKLROOT}/lib/intel64 -lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lmkl_blacs_intelmpi_ilp64 -lpthread -lm -ldl\n

and compiler options:

-i8 -I${MKLROOT}/include/intel64/ilp64 -I${MKLROOT}/include\n

It is a good idea to double check the library locations given by the tool are correct: do an ls ${MKLROOT}/lib/intel64 and make sure the directory exists and contains the libraries. In the past there have been slight path differences between tool and install for some versions.

"},{"location":"Software_Guides/Installing_Software/#openblas","title":"OpenBLAS","text":"

We have native threads, OpenMP and serial versions of OpenBLAS. Type module avail openblas to see the available versions.

"},{"location":"Software_Guides/Installing_Software/#linking-openblas","title":"Linking OpenBLAS","text":"

Our OpenBLAS modules now contain symlinks for libblas and liblapack that both point to libopenblas. This means that the default -lblas -llapack will in fact work.

This is how you would normally link OpenBLAS:

-L${OPENBLASROOT}/lib -lopenblas\n

If code you are compiling requires separate entries for BLAS and LAPACK, set them both to -lopenblas.

"},{"location":"Software_Guides/Installing_Software/#troubleshooting-openmp-loop-warning","title":"Troubleshooting: OpenMP loop warning","text":"

If you are running a threaded program and get this warning:

OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.\n

Then tell OpenBLAS to use only one thread by adding the below to your jobscript (this overrides $OMP_NUM_THREADS for OpenBLAS only):

export OPENBLAS_NUM_THREADS=1\n

If it is your own code, you can also set it in the code with the function

void openblas_set_num_threads(int num_threads);\n

You can avoid this error by compiling with one of the native-threads or serial OpenBLAS modules instead of the openmp one.

"},{"location":"Software_Guides/Installing_Software/#atlas","title":"ATLAS","text":"

We would generally recommend using OpenBLAS instead at present, but we do have ATLAS modules.

"},{"location":"Software_Guides/Installing_Software/#dynamic-linking-atlas","title":"Dynamic linking ATLAS","text":"

There is one combined library each for serial and threaded ATLAS (in most circumstances you probably want the serial version).

Serial:

-L${ATLASROOT}/lib -lsatlas\n

Threaded:

-L${ATLASROOT}/lib -ltatlas\n
"},{"location":"Software_Guides/Installing_Software/#static-linking-atlas","title":"Static linking ATLAS","text":"

There are multiple libraries to link.

Serial:

-L${ATLASROOT}/lib -llapack -lf77blas -lcblas -latlas\n

Threaded:

-L${ATLASROOT}/lib -llapack -lptf77blas -lptcblas -latlas\n
"},{"location":"Software_Guides/Installing_Software/#troubleshooting-libgfortran-or-lifcore-cannot-be-found","title":"Troubleshooting: libgfortran or lifcore cannot be found","text":"

If you get a runtime error saying that libgfortran.so cannot be found, you need to add -lgfortran to your link line.

The Intel equivalent is -lifcore.

You can do a module show on the compiler module you are using to see where the Fortran libraries are located if you need to give a full path to them.

"},{"location":"Software_Guides/Installing_Software/#installing-additional-packages-for-an-existing-scripting-language","title":"Installing additional packages for an existing scripting language","text":""},{"location":"Software_Guides/Installing_Software/#python","title":"Python","text":"

There are python2/recommended and python3/recommended module bundles you will see if you type module avail python. These use a virtualenv, have a lot of Python packages installed already, like numpy and scipy (see the Python package list) and have pip set up for you.

"},{"location":"Software_Guides/Installing_Software/#load-the-gnu-compiler","title":"Load the GNU compiler","text":"

Our Python installs were built with GCC. You can run them without problems with the default Intel compilers loaded because it also depends on the gcc-libs/4.9.2 module. However, when you are installing your own Python packages you should make sure you have the GNU compiler module loaded. This is to avoid the situation where you build your package with the Intel compiler and then try to run it with our GNU-based Python. If it compiled any C code, it will be unable to find Intel-specific instructions and give you errors.

Change your compiler module:

module unload compilers\nmodule load compilers/gnu/4.9.2\n

If you get an error like this when trying to run something, you built a package with the Intel compiler.

undefined symbol: __intel_sse2_strrchr\n
"},{"location":"Software_Guides/Installing_Software/#install-your-own-packages-in-the-same-virtualenv","title":"Install your own packages in the same virtualenv","text":"

This will use our central virtualenv and the packages we have already installed.

# for Python 2\npip install --user <python2pkg>\n# for Python 3\npip3 install --user <python3pkg>\n

These will install into .python2local or .python3local in your home directory.

If your own installed Python packages get into a mess, you can delete (or rename) the whole .python3local and start again.

"},{"location":"Software_Guides/Installing_Software/#using-your-own-virtualenv","title":"Using your own virtualenv","text":"

If you need different packages that are not compatible with the centrally installed versions (eg. what you are trying to install depends on a different version of something we have already installed) then you can create a new virtualenv and only packages you are installing yourself will be in it.

In this case, you do not want our virtualenv with our packages to also be active. We have two types of Python modules. If you type module avail python there are \"bundles\" which are named like python3/3.7 - these include our virtualenv and packages. Then there are the base modules for just python itself, like python/3.7.4.

When using your own virtualenv, you want to load one of the base python modules.

# load a base python module (you will always need to do this)\nmodule load python/3.7.4\n# create the new virtualenv, with any name you want\nvirtualenv <DIR>\n# activate it\nsource <DIR>/bin/activate\n

Your bash prompt will change to show you that a different virtualenv is active. (This one is called venv).

(venv) [uccacxx@login03 ~]$ \n

deactivate will deactivate your virtualenv and your prompt will return to normal.

You only need to create the virtualenv the first time.

"},{"location":"Software_Guides/Installing_Software/#error-while-loading-shared-libraries","title":"Error while loading shared libraries","text":"

You will always need to load the base python module before activating your virtualenv or you will get an error like this:

python3: error while loading shared libraries: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory\n
"},{"location":"Software_Guides/Installing_Software/#installing-via-setuppy","title":"Installing via setup.py","text":"

If you need to install by downloading a package and using setup.py, you can use the --user flag and as long as one of our python module bundles are loaded, it will install into the same .python2local or .python3local as pip does and your packages will be found automatically.

python setup.py install --user\n

If you want to install to a different directory in your space to keep this package separate, you can use --prefix instead. You'll need to add that location to your $PYTHONPATH and $PATH as well so it can be found. Some install methods won't create the prefix directory you requested for you automatically, so you would need to create it yourself first.

This type of install makes it easier for you to only have this package in your paths when you want to use it, which is helpful if it conflicts with something else.

# add location to PYTHONPATH so Python can find it\nexport PYTHONPATH=/home/username/your/path/lib/python3.7/site-packages:$PYTHONPATH\n# if necessary, create lib/pythonx.x/site-packages in your desired install location\nmkdir -p /home/username/your/path/lib/python3.7/site-packages\n# do the install\npython setup.py install --prefix=/home/username/your/path\n

It will tend to tell you at install time if you need to change or create the $PYTHONPATH directory.

To use this package, you'll need to add it to your paths in your jobscript or .bashrc. Check that the PATH is where your Python executables were installed.

export PYTHONPATH=/home/username/your/path/lib/python3.7/site-packages:$PYTHONPATH\nexport PATH=/home/username/your/path/bin:$PATH\n

It is very important that you keep the :$PYTHONPATH or :$PATH at the end of these - you are putting your location at the front of the existing contents of the path. If you leave them out, then only your package location will be found and nothing else.

"},{"location":"Software_Guides/Installing_Software/#troubleshooting-remove-your-pip-cache","title":"Troubleshooting: remove your pip cache","text":"

If you built something and it went wrong, and are trying to reinstall it with pip and keep getting errors that you think you should have fixed, you may still be using a previous cached version. The cache is in .cache/pip in your home directory, and you can delete it.

You can prevent caching entirely by installing using pip3 install --user --no-cache-dir <python3pkg>

"},{"location":"Software_Guides/Installing_Software/#troubleshooting-python-script-executable-paths","title":"Troubleshooting: Python script executable paths","text":"

If you have an executable Python script (eg. something you run using pyutility and not python pyutility.py) that begins like this: 

#!/usr/bin/python2.6\n

and fails because that Python doesn't exist in that location or isn't the one that has the additional packages installed, then you should change it so it uses the first Python found in your environment instead, which will be the one from the Python module you've loaded.

#!/usr/bin/env python\n
"},{"location":"Software_Guides/Matlab/","title":"MATLAB","text":"

MATLAB is a numerical computing environment and proprietary programming language developed by MathWorks.

Our MATLAB installs include all the toolboxes included in UCL's Total Academic Headcount-Campus licence plus the MATLAB Parallel Server. We also have the NAG toolbox for Matlab available.

You can submit single node multi-threaded MATLAB jobs, single node jobs which use the Parallel Computing Toolbox and the MATLAB Parallel Server (MPS) and MATLAB GPU jobs. Currently MATLAB jobs can only be run on Myriad however we are working with MathWorks to allow the submission of multi-node MPS jobs on Kathleen.

You can also submit jobs to Myriad from MATLAB running on your own desktop or laptop.

"},{"location":"Software_Guides/Matlab/#setup","title":"Setup","text":"

You need to load MATLAB once from a login node before you can submit any jobs. This allows it to set up your ~/.matlab directory as a symbolic link to ~/Scratch/.matlab so that the compute nodes can write to it.

# on a login node\nmodule load xorg-utils/X11R7.7\nmodule load matlab/full/r2021a/9.10\n

We have other versions of MATLAB installed. You can run module avail matlab to see all the available installed versions.

"},{"location":"Software_Guides/Matlab/#single-node-multi-threaded-batch-jobs","title":"Single node multi-threaded batch jobs","text":"

This is the simplest way to start using MATLAB on the cluster.

You will need a .m file containing the MATLAB commands you want to carry out.

Here is an example jobscript which you would submit using the qsub command, after you have loaded the MATLAB module once on a login node as mentioned in Setup:

#!/bin/bash -l\n\n# Batch script to run a multi-threaded MATLAB job under SGE.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM per core. \n#$ -l mem=1G\n\n# Request 15 gigabytes of TMPDIR space (default is 10 GB)\n#$ -l tmpfs=15G\n\n# Request a number of threads (which will use that number of cores). \n# On Myriad you can set the number of threads to a maximum of 36. \n#$ -pe smp 36\n\n# Request one MATLAB licence - makes sure your job doesn't start \n# running until sufficient licenses are free.\n#$ -l matlab=1\n\n# Set the name of the job.\n#$ -N Matlab_multiThreadedJob1\n\n# Set the working directory to somewhere in your scratch space.\n# This is a necessary step as compute nodes cannot write to $HOME.\n# Replace \"<your_UCL_id>\" with your UCL user ID.\n# This directory must already exist.\n#$ -wd /home/<your_UCL_id>/Scratch/Matlab_examples\n\n# Your work should be done in $TMPDIR\ncd $TMPDIR\n\nmodule load xorg-utils/X11R7.7\nmodule load matlab/full/r2021a/9.10\n# outputs the modules you have loaded\nmodule list\n\n# Optional: copy your script and any other files into $TMPDIR.\n# This is only practical if you have a small number of files.\n# If you do not copy them in, you must always refer to them using a\n# full path so they can be found, eg. ~/Scratch/Matlab_examples/analyse.m\n\ncp ~/Scratch/Matlab_examples/myMatlabJob.m $TMPDIR\ncp ~/Scratch/Matlab_examples/initialise.m $TMPDIR\ncp ~/Scratch/Matlab_examples/analyse.m $TMPDIR\n\n# These echoes output what you are about to run\necho \"\"\necho \"Running matlab -nosplash -nodisplay < myMatlabJob.m ...\"\necho \"\"\n\nmatlab -nosplash -nodesktop -nodisplay < myMatlabJob.m\n# Or if you did not copy your files:\n# matlab -nosplash -nodesktop -nodisplay < ~/Scratch/Matlab_examples/myMatlabJob.m\n\n# tar up all contents of $TMPDIR back into your space\ntar zcvf $HOME/Scratch/Matlab_examples/files_from_job_${JOB_ID}.tgz $TMPDIR\n\n# Make sure you have given enough time for the copy to complete!\n

Alternative syntax: Instead of using Unix input redirection like this:

matlab -nosplash -nodesktop -nodisplay < $Matlab_infile\n

you can also do:

matlab -nosplash -nodesktop -nodisplay -r $Matlab_infile\n

NOTE: You should use the -r syntax if your input file contains function definitions as using < treats the script as keyboard input and MATLAB does not allow you to define functions directly on the command line.

"},{"location":"Software_Guides/Matlab/#run-without-the-jvm-to-reduce-overhead","title":"Run without the JVM to reduce overhead","text":"

You can give the -nojvm option to tell MATLAB to run without the Java Virtual Machine. This will speed up startup time, possibly execution time, and remove some memory overhead, but will prevent you using any tools that require Java (eg, tools that use the Java API for I/O and networking like URLREAD, or call Java object methods).

"},{"location":"Software_Guides/Matlab/#run-single-threaded","title":"Run single-threaded","text":"

Most of the time, MATLAB will create many threads and use them as it wishes. If you know your job is entirely single-threaded, you can force MATLAB to run with only one thread on one core, which will allow you to have more jobs running at once.

To request one core only, set #$ -pe smp 1 in your jobscript.

Run MATLAB like this:

matlab -nosplash -nodesktop -nodisplay -nojvm -singleCompThread < $Matlab_infile\n

The -singleCompThread forces MATLAB to run single-threaded, and the -nojvm tells it to run without the Java Virtual Machine, as above.

"},{"location":"Software_Guides/Matlab/#using-the-matlab-gui-interactively","title":"Using the MATLAB GUI interactively","text":"

You can run MATLAB interactively for short amounts of time on the login nodes (please do not do this if your work will be resource-intensive). You can also run it interactively in a qrsh session on the compute nodes.

Launching with matlab will give you the full graphical user interface - you will need to have logged in to the cluster with X-forwarding on for this to work.

Launching with matlab -nodesktop -nodisplay will give you the MATLAB terminal.

[//]: # Up to this point on revision.

"},{"location":"Software_Guides/Matlab/#submitting-jobs-using-the-matlab-parallel-server-mps","title":"Submitting jobs using the MATLAB Parallel Server (MPS)","text":"

You must have loaded the MATLAB module once from a login node as described in Setup before you can submit any MATLAB MPS jobs.

MATLAB PCS jobs must be submitted from an interactive or scripted Matlab session which can be running on the cluster login nodes, from another MATLAB job or on your own machine.

MATLAB MPS jobs will currently only work inside a single node on our clusters. On Myriad this means a maximum of 36 workers can be used per job.

"},{"location":"Software_Guides/Matlab/#importing-the-cluster-profile","title":"Importing the cluster profile","text":"

You need to import the cluster profile into your MATLAB environment and set it as the default before you can submit DCS jobs. This only needs doing once. The imported profile will be saved in your MATLAB settings directory.

Importing the profile can be done either by calling MATLAB functions or via the graphical interface. The profile is stored here (for R2021a): 

/shared/ucl/apps/Matlab/toolbox_local/R2021a/myriad_R2021a.mlsettings\n
"},{"location":"Software_Guides/Matlab/#import-using-matlab-functions","title":"Import using MATLAB functions","text":"

Run these functions from a MATLAB session:

profile_Myriad = parallel.importProfile ('/shared/ucl/apps/Matlab/toolbox_local/R2021a/myriad_R2021a.mlsettings');\nparallel.defaultClusterProfile ('myriad_R2021a');\n
"},{"location":"Software_Guides/Matlab/#import-from-matlab-gui","title":"Import from MATLAB GUI","text":"

To import using the graphical interface instead, do this.

  1. From the Home tab select the Parallel menu and click Create and Manage Clusters....
  2. The Cluster Profile Manager window will open.
  3. Select Import and in the Import Profiles from file window navigate to the myriad_R2021a.mlsettings file shown above and select Open.
  4. Select the resulting myriad_R2021a profile and click Set as Default. The Cluster Profile Manager window should now look like this:

In both cases after you exit MATLAB your cluster profile is saved for future use.

"},{"location":"Software_Guides/Matlab/#environment-variables-needed-for-job-submission","title":"Environment variables needed for job submission","text":"

We have set up four Grid Engine environment variables to assist with job submission from within MATLAB. These are needed to pass in job resource parameters that aren't supported by the internal MATLAB job submission mechanism.

and two project and Gold related variables. Most users will not need to use either of these:

-ac exclusive prevents anything else running on the same node as your job, even if you aren't using all the cores. This is no longer a necessary option for MATLAB jobs.

There are two ways to set these:

1) Before starting your MATLAB session, using the usual Bash method of exporting environment variables:

export SGE_CONTEXT=exclusive\nexport SGE_OPT=h_rt=0:15:0,mem=2G,tmpfs=15G\nexport SGE_PROJECT=<your project ID>\nexport SGE_ACCOUNT=<your Gold project>\n

2) Inside your MATLAB session, using MATLAB's setenv function:

setenv ('SGE_CONTEXT', 'exclusive');\nsetenv ('SGE_OPT', 'h_rt=0:15:0,mem=2G,tmpfs=15G'); \nsetenv ('SGE_PROJECT', '<your project ID>');\nsetenv ('SGE_ACCOUNT', '<your Gold project>');\n
"},{"location":"Software_Guides/Matlab/#example-a-simple-mps-job","title":"Example: a simple MPS job","text":"

This submits a job from inside a MATLAB session running on a login node. You need to start MATLAB from a directory in Scratch - jobs will inherit this as their working directory.

This is an example where you have only one MATLAB source file.

1) Change to the directory in Scratch you want the job to run from and set the SGE environment variables.

cd ~/Scratch/Matlab_examples\nexport SGE_OPT=h_rt=0:10:0,mem=2G,tmpfs=15G\n

2) Either start the MATLAB GUI:

matlab\n

or start a MATLAB terminal session:

matlab -nodesktop -nodisplay\n

3) Inside MATLAB, create a cluster object using the cluster profile:

c = parcluster ('myriad_R2021a');\n

4) Use your cluster object to create a job object of the type you need. For this example the job is a parallel job with communication between MATLAB workers of type \"Single Program Multiple Data\":

myJob = createCommunicatingJob (c, 'Type', 'SPMD');\n

5) Set the number of workers:

num_workers = 8;\n

The maximum value you can set here on Myriad is 36.

6) Tell the job the files needed to be made available to each worker - in this example there is only one file:

myJob.AttachedFiles = {'colsum.m'};\n

colsum.m contains the simple magic square example from the MATLAB manual \"Parallel Computing Toolbox User's Guide\".

7) Set the minimum and maximum number of workers for the job (we are asking for an exact number here by setting them the same):

myJob.NumWorkersRange = [num_workers, num_workers];\n

8) Create a MATLAB task to be executed as part of the job. Here it consists of executing the MATLAB function colsum. The other arguments say that the task returns one parameter and there are no input arguments to the colsum function:

task = createTask (myJob, @colsum, 1, {});\n

9) Submit the job:

submit (myJob);\n

Your job is now submitted to the scheduler and you can see its queue status in qstat as normal. If you were using the MATLAB GUI you can also monitor jobs by selecting Monitor Jobs from the Parallel menu on the Home tab.

10) When the job has completed get the results using:

results = fetchOutputs(myJob)\n

You can access the job log from MATLAB using:

logMess = getDebugLog (c, myJob);\n
"},{"location":"Software_Guides/Matlab/#example-a-mps-job-with-more-than-one-input-file","title":"Example: a MPS job with more than one input file","text":"

This example has several input files. The job type is \"MATLAB Pool\". A \"Pool\" job runs the specified task function with a MATLAB pool available to run the body of parfor loops or spmd blocks and is the default job type. This example was kindly supplied to assist in testing MATLAB by colleagues from CoMPLEX.

The first part of creating the job is the same as the above example apart from the longer runtime and larger amount of memory per core:

1) Change into a directory in Scratch, set the SGE variables and launch MATLAB:

cd ~/Scratch/Matlab_examples\nexport SGE_OPT=h_rt=1:0:0,mem=4G,tmpfs=15G\nmatlab\n

to launch the GUI or:

matlab -nodesktop -nodisplay\n

to start a terminal session. 

c = parcluster ('myriad_R2021a');\n

2) Using our cluster object create a job object of type \"Pool\":

myJob2 = createCommunicatingJob (c, 'Type', 'Pool');\n

3) Set the number of workers and another variable used by the example:

num_workers = 8;\nsimulation_duration_ms = 1000;\n

4) Tell the job all the input files needed to be made available to each worker as a cell array:

myJob2.AttachedFiles = {\n'AssemblyFiniteDifferencesMatrix.m'\n'AssemblyFiniteDifferencesRightHandSide.m'\n'CellModelsComputeIonicCurrents.m'\n'CellModelsGetVoltage.m'\n'CellModelsInitialise.m'\n'CellModelsSetVoltage.m'\n'GetStimuliForTimeStep.m'\n'SubmitMonodomainJob.m'\n'RegressionTest.m'\n'RunAndVisualiseMonodomainSimulation.m'\n'SolveLinearSystem.m'\n'luo_rudy_1991_iionic.m'\n'luo_rudy_1991_time_deriv.m'};\n

5) Set the minimum and maximum number of workers for the job:

myJob2.NumWorkersRange = [num_workers, num_workers];\n

6) Create a MATLAB task to be executed as part of the job. For this example it will consist of executing the MATLAB function RunAndVisualiseMonodomainSimulation. The rest of the arguments indicate that the task returns three parameters and there are five input arguments to the function. These are passed as a cell array:

task = createTask (myJob2, @RunAndVisualiseMonodomainSimulation, 3, {5000, simulation_duration_ms, 1.4, 1.4, false});\n

7) Submit the job:

submit (myJob2);\n

As before use fetchOutputs to collect the results.

If you closed your session, you can get your results by:

c = parcluster ('myriad_R2021a');                  # get a cluster object\njobs = findJob(c)                                  # get a list of jobs submitted to that cluster\njob = jobs(3);                                     # pick a particular job\nresults = fetchOutputs(job)\n

You can get other information: diary(job) will give you the job diary, and load(job) will load the workspace.

"},{"location":"Software_Guides/Matlab/#further-reading","title":"Further reading","text":"

There is a lot more information about using the MATLAB Distributed Computing Server in the MATLAB manual: Parallel Computing Toolbox User\u2019s Guide.

"},{"location":"Software_Guides/Matlab/#submitting-matlab-jobs-from-your-workstationlaptop","title":"Submitting MATLAB jobs from your workstation/laptop","text":"

You can submit MATLAB jobs to Myriad from MATLAB sessions running on your own desktop workstation or laptop systems provided they are running the same version of MATLAB and your computer is within the UCL firewall.

With MATLAB R2021a you can currently submit jobs to Myriad. Support for R2018b is also still available.

"},{"location":"Software_Guides/Matlab/#prerequisites","title":"Prerequisites","text":"
  1. You must already have an account on the clusters!
  2. Have MATLAB R2021a (or R2018b) installed on your local workstation/laptop. The local version must match the version running jobs. MATLAB R2021a can be downloaded from the UCL Software Database.
  3. Your local workstation/laptop installation of MATLAB must include the Parallel Computing toolbox. This is included in the UCL TAH MATLAB license and may be installed automatically. Home tab > Add-Ons > Get Add-Ons You will find the Parallel Computing Toolbox in the Workflows category.
  4. If your local workstation/laptop is not directly connected to the UCL network (at home for example), you need to have the UCL VPN client installed and running on it.
"},{"location":"Software_Guides/Matlab/#remote-setup","title":"Remote setup","text":"

1) On the cluster you are using (Myriad in this case) create a directory to hold remotely submitted job details. For example:

mkdir ~/Scratch/Matlab_remote_jobs\n

This directory needs to be in your Scratch directory as compute nodes need to be able to write to it. You should not use this directory for anything else.

2) On your local workstation/laptop create a directory to hold information about jobs that have been submitted to the cluster. Again you should not use this directory for anything else.

3) Download either the the support files for remote submission to Myriad for R2021a or support files for remote submission to Myriad for R2018b. Make sure you download the correct one for your version of MATLAB!

4) This step MUST be done while Matlab is shut down. Unzip the archive into MATLAB's local toolbox directory. Default locations for the local toolbox directory are:

5) Download the parallelProfileMyriad function to your local workstation/laptop. It will need to be unzipped. This function create a cluster profile for Myriad for R2021a or R2018b.

6) Start MATLAB, navigate to where you saved the parallelProfileMyriad.m file and run the function by typing:

parallelProfileMyriad\n

at your MATLAB prompt (in your MATLAB Command Window if running the MATLAB GUI) and answer the questions.

"},{"location":"Software_Guides/Matlab/#submitting-a-job-to-the-cluster","title":"Submitting a job to the cluster","text":"

1) You need to set the Grid Engine support environment variables on your local computer. Eg. in your MATLAB session set:

setenv ('SGE_CONTEXT', 'exclusive');                # optional\nsetenv ('SGE_OPT', 'h_rt=0:15:0,mem=2G,tmpfs=15G'); \nsetenv ('SGE_PROJECT', '<your project ID>');        # optional\nsetenv ('SGE_ACCOUNT', '<your Gold project>');      # optional\n

2) In your MATLAB session create a cluster object using the cluster profile created by the parallelProfile... functions. For Myriad:

c = parcluster ('myriad_R2021a');\n

3) You can now create and submit jobs in a similar way to that shown in the MPS examples above starting from step 4 in the simple MPS job example or step 2 in the MPS job with multiple input files example.

"},{"location":"Software_Guides/Matlab/#viewing-your-results","title":"Viewing your results","text":"

After submitting your job remotely from your desktop, you can close MATLAB and come back later. To see your jobs:

Click \"Parallel > Monitor jobs\"

This will bring up the job monitor where you can see the status of your jobs and whether they are finished. MATLAB numbers the jobs sequentially.

Right-click on a job and choose \"fetch outputs\".

This is what will be executed (for job4 on Myriad):

myCluster = parcluster('myriad_R2021a');\njob4 = myCluster.findJob('ID',4);\njob4_output = fetchOutputs(job4);\n

The Workspace will show the available data and you can view your results. The data is fetched from the Matlab_remote_jobs directory you created on Myriad (or Legion) in Remote setup step 1, so that will also have files and directories in it called job1, job2 and so on.

If you have already fetched the data, you can view the results straight away by selecting that job. If you need to reload everything, you can right-click on the job and the option will be to load variables instead.

"},{"location":"Software_Guides/Matlab/#writing-intermediate-results","title":"Writing intermediate results","text":"

If you want to explicitly write out intermediate results, you need to provide a full path to somewhere in Scratch otherwise MATLAB will try to write them in your home, which isn't writable by the compute nodes.

"},{"location":"Software_Guides/Matlab/#troubleshooting-remote-jobs","title":"Troubleshooting remote jobs","text":"

If you get a message like this when retrieving your outputs then something has gone wrong in your job:

Task with ID xxx returned 0 outputs but 1 were expected\n

You need to retrieve the debug log to find out what happened. Example:

myCluster = parcluster('myriad_R2021a');\njob4 = myCluster.findJob('ID',4);\n\njobLog = getDebugLog (myCluster, job4);\njobLog\n

There will be a lot of output. Look for lines related to errors happening in your own code.

"},{"location":"Software_Guides/Matlab/#running-matlab-on-gpus","title":"Running MATLAB on GPUs","text":"

This uses MATLAB's Mandelbrot Set GPU example.

#!/bin/bash -l\n\n# Batch script to run a GPU MATLAB job on Myriad.\n\n# Request 15 minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:15:0\n\n# Request 2 gigabytes of RAM per core.\n#$ -l mem=2G\n\n# Request 15 gigabytes of TMPDIR space (default is 10 GB)\n#$ -l tmpfs=15G\n\n# Request 1 GPU\n#$ -l gpu=1\n\n# Request 1 CPU core. (Max on Myriad is 36)\n#$ -pe smp 1\n\n# Request one MATLAB licence - makes sure your job doesn't start \n# running until sufficient licenses are free.\n#$ -l matlab=1\n\n# Set the name of the job.\n#$ -N Matlab_GPU_Job1\n\n# Set the working directory to somewhere in your scratch space.\n# This is a necessary step as compute nodes cannot write to $HOME.\n# Replace \"<your_UCL_id>\" with your UCL user ID.\n# This directory must already exist.\n#$ -wd /home/<your_UCL_id>/Scratch/Matlab_examples\n\n# Your work should be done in $TMPDIR\ncd $TMPDIR\n\n# Optional: Copy your script and any other files into $TMPDIR.\n# If not, you must always refer to them using a full path.\ncp /home/ccaabaa/Software/Matlab/Mandelbrot_GPU.m $TMPDIR\n\nmodule unload compilers mpi\nmodule load compilers/gnu/4.9.2\nmodule load xorg-utils/X11R7.7\nmodule load matlab/full/r2021a/9.10\nmodule list\n\n# These echoes output what you are about to run\necho \"\"\necho \"Running matlab -nosplash -nodisplay < Mandelbrot_GPU.m ...\"\necho \"\"\nmatlab -nosplash -nodesktop -nodisplay < Mandelbrot_GPU.m\n\n# tar up all contents of $TMPDIR back into your space\ntar zcvf $HOME/Scratch/Matlab_examples/files_from_job_${JOB_ID}.tgz $TMPDIR\n\n# Make sure you have given enough time for the copy to complete!\n
"},{"location":"Software_Guides/Other_Software/","title":"Other Software","text":"

We maintain a large software stack that is available across all our clusters (licenses permitting).

We use environment modules to let you manage which specific versions of software packages you are using.

"},{"location":"Software_Guides/Other_Software/#general-use-of-environment-modules","title":"General use of environment modules","text":"

We have a default set of modules that everyone has loaded when they log in: these include the current default compiler and MPI, some utili ties to make your life easier and some text editors.

"},{"location":"Software_Guides/Other_Software/#summary-of-module-commands","title":"Summary of module commands","text":"
module avail            # shows available modules\nmodule whatis           # shows available modules with brief explanations\nmodule list             # shows your currently loaded modules\n\nmodule load <module>    # load this module\nmodule unload <module>  # unload this module\nmodule purge            # unload all modules\n\nmodule show <module>    # Shows what the module requires and what it sets up\nmodule help <module>    # Shows a longer text description for the software\n
"},{"location":"Software_Guides/Other_Software/#find-out-if-a-software-package-is-installed-and-load-it","title":"Find out if a software package is installed and load it","text":"

Generically, the way you find out if a piece of software is installed is to run

module load beta-modules\nmodule avail packagename\n

By loading beta-modules you will also be able to see newer versions of GCC and the software that has been built using them.

Then module avail gives you a list of all the modules we have that match the name you searched for. You can then type 

module show packagename\n

and it will show you the other software dependencies this module has: these have to be loaded first. It also shows where the software is installed and what environment variables it sets up.

Once you have found the modules you want to load, it is good practice to refer to them using their full name, including the version. If you use the short form (package rather than package/5.1.2/gnu-4.9.2) then a matching module will be loaded, but if we install a different version, your jobs may begin using the new one and you would not know which version created your results. Different software versions may not be compatible or may have different default settings, so this is undesirable.

You may need to unload current modules in order to load some requirements (eg different compiler, different MPI).

This example switches from Intel compiler and MPI modules to GNU ones.

module unload -f compilers mpi\nmodule load compilers/gnu/4.9.2\nmodule load mpi/openmpi/3.1.4/gnu-4.9.2\n

You can use the short name when unloading things because there is usually only one match in your current modules.

The last part of a module name usually tells you what compiler it was built with and which version of that compiler. There may be GNU compiler versions and Intel compiler versions of the same software available.

Once the module is loaded, you should have all the usual executables in your path, and can use its commands. You load modules in exactly the same way inside a jobscript.

"},{"location":"Software_Guides/Other_Software/#notes-on-how-to-run-specific-packages","title":"Notes on how to run specific packages","text":"

The packages below have slightly complex commands needed to run them, or different settings needed on our clusters. These are examples of what should be added to your jobscripts. Change the module load command to the version you want to load and check that the dependencies are the same.

The top of a jobscript should contain your resource requests. See also examples of full jobscripts .

"},{"location":"Software_Guides/Other_Software/#abaqus","title":"ABAQUS","text":"

ABAQUS is a commercial software suite for finite element analysis and computer-aided engineering.

You must be authorised by the Mech Eng Department before you can be added to the group controlling access to ABAQUS (legabq).

A serial interactive analysis can be run on the compute nodes (via a qrsh session) like this:

abaqus interactive job=myJobSerial input=myInputFile.inp\n

A parallel job can be run like this (fill in your own username):

module load abaqus/2017\n\nINPUT_FILE=/home/<username>/ABAQUS/heattransfermanifold_cavity_parallel.inp\nABAQUS_ARGS=\nABAQUS_PARALLELSCRATCH=/home/<username>/Scratch/Abaqus/parallelscratch\n# creates a parallel scratch dir and a new working dir for this job\nmkdir -p $ABAQUS_PARALLELSCRATCH\nmkdir -p $JOB_NAME.$JOB_ID\ncd $JOB_NAME.$JOB_ID\ncp $INPUT_FILE .\n\nINPUT=$(basename $INPUT_FILE)\nabaqus interactive cpus=$NSLOTS mp_mode=mpi job=$INPUT.$JOB_ID input=$INPUT \\\n       scratch=$ABAQUS_PARALLELSCRATCH $ABAQUS_ARGS\n
"},{"location":"Software_Guides/Other_Software/#beast","title":"BEAST","text":"

BEAST is an application for Bayesian MCMC analysis of molecular sequences orientated towards rooted, time-measured phylogenies inferred using strict or relaxed molecular clock models.

Note that FigTree and Tracer are available as standalone modules. The addons DISSECT, MODEL_SELECTION, and SNAPP are installed for BEAST. 

cd $TMPDIR\n\nmodule load java/1.8.0_45\nmodule load beast/2.3.0\n\nbeast -threads $OMP_NUM_THREADS ~/Scratch/BEAST/gopher.xml\n\n# tar up all contents of $TMPDIR back into your space\ntar zcvf $HOME/Scratch/BEAST/files_from_job_$JOB_ID.tar.gz $TMPDIR\n
"},{"location":"Software_Guides/Other_Software/#bowtie","title":"Bowtie","text":"

Bowtie 1 and 2 are tools for aligning sequencing reads to their reference sequences.

Bowtie 1 and 2 are available. For reads longer than about 50 bp Bowtie 2 is generally faster, more sensitive, and uses less memory than Bowtie 1. For relatively short reads (e.g. less than 50 bp) Bowtie 1 is sometimes faster and/or more sensitive. For further differences, see How is Bowtie 2 different from Bowtie 1?.

Bowtie sets $BT1_HOME and Bowtie2 sets $BT2_HOME. You can have both modules loaded at once. 

cd $TMPDIR\nmodule load bowtie2/2.2.5\n\n# Run Bowtie2 example from getting started guide:\n# http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml#getting-started-with-bowtie-2-lambda-phage-example\nbowtie2-build $BT2_HOME/example/reference/lambda_virus.fa lambda_virus\nbowtie2 -x lambda_virus -U $BT2_HOME/example/reads/reads_1.fq -S eg1.sam\n\n# tar up all contents of $TMPDIR back into your space\ntar zcvf $HOME/Scratch/Bowtie2_output/files_from_job_$JOB_ID.tgz $TMPDIR\n
"},{"location":"Software_Guides/Other_Software/#castep","title":"CASTEP","text":"

CASTEP is a full-featured materials modelling code based on a first-principles quantum mechanical description of electrons and nuclei.

module load castep/17.21/intel-2017\n# Gerun is our mpirun wrapper which sets the machinefile and number of \n# processes to the amount you requested with -pe mpi.\ngerun castep.mpi input\n

CASTEP 19 has different pre-reqs:

module unload -f compilers mpi\nmodule load compilers/intel/2019/update4\nmodule load mpi/intel/2019/update4/intel\nmodule load castep/19.1.1/intel-2019\n# Gerun is our mpirun wrapper which sets the machinefile and number of \n# processes to the amount you requested with -pe mpi.\ngerun castep.mpi input\n

If you have access to the source code and wish to build your own copy, it has been suggested that compiling with these options (on Grace) gave a build that ran about 10% faster than the default compilation options:

make COMMS_ARCH=mpi SUBARCH=mpi FFT=mkl MATHLIBS=mkl10 BUILD=fast\n

Do check for numerical accuracy with any optimisations you carry out.

"},{"location":"Software_Guides/Other_Software/#cctools","title":"Cctools","text":"

Provides the Parrot connector to CVMFS, the CernVM File System.

By default, the cctools module sets the following:

export PARROT_CVMFS_REPO=<default-repositories> \nexport PARROT_ALLOW_SWITCHING_CVMFS_REPOSITORIES=yes \nexport HTTP_PROXY=DIRECT;\nexport PARROT_HTTP_PROXY=DIRECT;\n

Example usage - will list the contents of the repository then exit:

module load cctools/7.0.11/gnu-4.9.2\nparrot_run bash\nls /cvmfs/alice.cern.ch\nexit\n

That will create the cache in /tmp/parrot.xxxxx on the login nodes when run interactively. To use in a job, you will want to put the cache somewhere in your space that the compute nodes can access. You can set the cache to be in your Scratch, or to $TMPDIR on the nodes if it just needs to exist for the duration of that job.

export PARROT_CVMFS_ALIEN_CACHE=</path/to/cache>\n
"},{"location":"Software_Guides/Other_Software/#cfd-ace","title":"CFD-ACE","text":"

CFD-ACE+ is a commercial computational fluid dynamics solver developed by ESI Group. It solves the conservation equations of mass, momentum, energy, chemical species and other scalar transport equations using the finite volume method. These equations enable coupled simulations of fluid, thermal, chemical, biological, electrical and mechanical phenomena.

The license is owned by the Department of Mechanical Engineering who must give permission for users to be added to the group lgcfdace.

module load cfd-ace/2018.0\n\nCFD-SOLVER -model 3Dstepchannel_060414.DTF -num $NSLOTS -wd `pwd` \\ \n   -hosts $TMPDIR/machines -rsh=ssh -decomp -metis -sim 1 -platformmpi -job\n
"},{"location":"Software_Guides/Other_Software/#comsol","title":"COMSOL","text":"

COMSOL Multiphysics is a cross-platform finite element analysis, solver and multiphysics simulation software.

Electrical Engineering have a group license for version 52 and must give permission for users to be added to the group legcomsl. Chemical Engineering have a Departmental License for version 53 and members of that department may be added to the group lgcomsol. 

# Run a parallel COMSOL job\n\n# Versions 52 and 52a have this additional module prerequisite\nmodule load xulrunner/3.6.28/gnu-4.9.2\n\n# pick the version to load\nmodule load comsol/53a\n\n# Parallel multinode options:\n# $NHOSTS gets the number of nodes the job is running on and\n# $TMPDIR/machines is the machinefile that tells it which nodes.\n# These are automatically set up in a \"-pe mpi\" job environment.\ncomsol -nn $NHOSTS -clustersimple batch -f $TMPDIR/machines -inputfile micromixer_batch.mph \\ \n       -outputfile micromixer_batch_output_${JOB_ID}.mph\n\n# On Myriad you need to specify the fabric:\ncomsol batch -f $TMPDIR/machines -np $NSLOTS -mpifabrics shm:tcp \\ \n    -inputfile micromixer_batch.mph -outputfile micromixer_batch_output_${JOB_ID}.mph\n
"},{"location":"Software_Guides/Other_Software/#cp2k","title":"CP2K","text":"

CP2K performs atomistic and molecular simulations.

To see all available versions type

module load beta-modules\nmodule avail cp2k\n

To load CP2K 8.2:

module unload -f compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load compilers/gnu/10.2.0\n\n# These three modules only needed on Myriad\nmodule load numactl/2.0.12\nmodule load binutils/2.36.1/gnu-10.2.0\nmodule load ucx/1.9.0/gnu-10.2.0\n\nmodule load mpi/openmpi/4.0.5/gnu-10.2.0\nmodule load openblas/0.3.13-openmp/gnu-10.2.0\nmodule load cp2k/8.2/ompi/gnu-10.2.0\n\n# Gerun is our mpirun wrapper which sets the machinefile and number of \n# processes to the amount you requested with -pe mpi.\ngerun cp2k.popt < input.in > output.out\n

For CP2K 4.1 there is also a Chemistry department version with submission script generator. To access it:

module load chemistry-modules\nmodule load submission-scripts\n

The command submitters will then list the submitters available. You can then run cp2k.submit which will ask you questions in order to create a suitable jobscript.

The cp2k.submit submitter takes up to 6 arguments, and any omitted will be asked for interactively:

cp2k.submit \u00abinput_file\u00bb \u00abcores\u00bb \u00abversion\u00bb \u00abmaximum_run_time\u00bb \u00abmemory_per_core\u00bb \u00abjob_name\u00bb\n

So, for example:

cp2k.submit water.inp 8 4.1 2:00:00 4G mywatermolecule\n

would request a job running CP2K 4.1 with the input file water.inp, on 8 cores, with a maximum runtime of 2 hours, with 4 gigabytes of memory per core, and a job name of mywatermolecule.

"},{"location":"Software_Guides/Other_Software/#crystal","title":"CRYSTAL","text":"

CRYSTAL is a general-purpose program for the study of crystalline solids. The CRYSTAL program computes the electronic structure of periodic systems within Hartree Fock, density functional or various hybrid approximations.

CRYSTAL is commercial software which is available free of charge to UK academics. You must obtain a license from Crystal Solutions: How to get CRYSTAL - Academic UK license. You need to create an account and then request to be upgraded to Academic UK. Access to CRYSTAL is enabled by being a member of the reserved application group legcryst. For proof of access we accept emails from CRYSTAL saying your account has been upgraded to \"Academic UK\", or a screenshot of your account page showing you have the full download available rather than just the demo version. 

module unload -f mpi\nmodule load mpi/openmpi/2.1.2/intel-2017\nmodule load crystal17/v1.0.1\n\n# 9. Create a directory for this job and copy the input file into it. \nmkdir test00\ncd test00\ncp ~/Scratch/Crystal17/test_cases/inputs/test00.d12 INPUT\n\n# Gerun is our mpirun wrapper which sets the machinefile and number of\n# processes to the amount you requested with -pe mpi.\n# The CRYSTAL module sets $CRYxx_EXEDIR and $VERSION environment variables.\ngerun $CRY17_EXEDIR/$VERSION/Pcrystal\n
# Similarly, for Pproperties the command would be\ngerun $CRY17_EXEDIR/$VERSION/Pproperties\n

For CRYSTAL 17 v1.0.2, the modules and path are slightly different and you would do this instead:

module unload -f compilers mpi\nmodule load compilers/intel/2017/update4\nmodule load mpi/intel/2017/update3/intel\nmodule load crystal17/v1.0.2/intel-2017\n\n# Create a directory for this job and copy the input file into it.\nmkdir test00\ncd test00\ncp ~/Scratch/Crystal17/test_cases/inputs/test00.d12 INPUT\n\n# Gerun is our mpirun wrapper which sets the machinefile and number of\n# processes to the amount you requested with -pe mpi.\n# The CRYSTAL module sets $CRYxx_EXEDIR and $VERSION environment variables.\ngerun $CRY17_EXEDIR/Pcrystal\n
"},{"location":"Software_Guides/Other_Software/#freesurfer","title":"FreeSurfer","text":"

FreeSurfer is a set of tools for analysis and visualization of structural and functional brain imaging data.

Freesurfer can use threads to run on multiple cores in one node: request the number with -pe smp in the resource-request part of your jobscript.

#$ -pe smp 4\n\nmodule load xorg-utils/X11R7.7\nmodule load freesurfer/6.0.0\nexport SUBJECTS_DIR=~/Scratch/FreeSurfer_examples/subjects\n\n# -openmp $NSLOTS runs with the number of threads you requested\nrecon-all -openmp $NSLOTS -i sample-001.nii.gz -s bert -all\n
"},{"location":"Software_Guides/Other_Software/#gamess","title":"GAMESS","text":"

The General Atomic and Molecular Electronic Structure System (GAMESS) is a general ab initio quantum chemistry package.

The GAMESS module should be loaded once from a login node before submitting a job - this creates the ~/Scratch/gamess directory for you which is used as USERSCR to write some scratch files during the job. If you don't want to keep these files and would prefer them to be written to $TMPDIR instead, you can put export $GAMESS_USERSCR=$TMPDIR in your jobscript after the module load command.

module unload compilers mpi\nmodule load compilers/intel/2015/update2\nmodule load mpi/intel/2015/update3/intel\nmodule load gamess/5Dec2014_R1/intel-2015-update2\n\n# Optional: set where the USERSCR files go. \n# By default, the module sets it to ~/Scratch/gamess\nexport $GAMESS_USERSCR=$TMPDIR\n\nrungms exam01.inp 00 $NSLOTS $(ppn)\n
"},{"location":"Software_Guides/Other_Software/#gatk","title":"GATK","text":"

The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyze high-throughput sequencing data.

Version 4 of GATK is BSD-licensed so does not require a group to control access to the software.

Version 3 of GATK requires you to agree to the GATK license before we can add you to the leggatk group which gives you access: you can do this by downloading GATK 3 from The Broad Institute GATK download page, reading the license, and telling us you agree to it. You may need to create a gatkforums account before you can download.

GATK 3 uses Java 1.7 (the system Java) so you do not need to load a Java module. GATK 4 uses 1.8 so you need to load java/1.8.0_92 first.

GATK 4.2.5.0 or newer uses the newest version of Java 8, so you need to load java/temurin-8.

Load the version you want, then to run GATK you should either prefix the .jar you want to run with $GATKPATH:

java -Xmx2g -jar $GATKPATH/GenomeAnalysisTK.jar OPTION1=value1 OPTION2=value2...\n

Or we provide wrappers, so you can run it one of these ways instead:

GenomeAnalysisTK OPTION1=value1 OPTION2=value2...\n
gatk OPTION1=value1 OPTION2=value2...\n

If you want to use some of the newer tools in GATK 4 which rely on Python/Conda, you must use GATK >= 4.2.5.0 and additionally set up your miniconda environment. With 4.2.5.0 this means:

module load java/temurin-8\nmodule load gatk/4.2.5.0\nmodule load python/miniconda3/4.10.3\nsource $UCL_CONDA_PATH/etc/profile.d/conda.sh \nconda activate $GATK_CONDA\n

(For newer versions of GATK it will tell you which version of miniconda to load)

"},{"location":"Software_Guides/Other_Software/#gaussian","title":"Gaussian","text":"

Access to Gaussian 09 or Gaussian 16 are controlled by membership of separate groups. UCL has a site license so UCL users can be added on request.

Gaussian is too resource-intensive to ever be run on the login nodes.

"},{"location":"Software_Guides/Other_Software/#multithreaded-shared-memory-gaussian-jobs","title":"Multithreaded shared memory Gaussian jobs","text":"

The main Gaussian executable lets you run jobs that use from 1 core up to a full node. When using more than one core, make sure your input file contains %NProcShared= with the number of cores your job is requesting.

$GAUSS_SCRDIR is where Gaussian puts temporary files which can use a lot of space. On Myriad in a job this is created inside $TMPDIR by default. On diskless clusters, this is set this to a directory in your Scratch instead: loading one of the Gaussian modules will handle this automatically and show where it has created the directory.

# Example for Gaussian 16\n\n# Set up runtime environment\nmodule load gaussian/g16-a03/pgi-2016.5\nsource $g16root/g16/bsd/g16.profile\n\n# Run g16 job\ng16 input.com\n
# Example for Gaussian 09\n\n# Setup runtime environment\nmodule load gaussian/g09-d01/pgi-2015.7\nsource $g09root/g09/bsd/g09.profile\n\n# Run g09 job\ng09 input.com\n
"},{"location":"Software_Guides/Other_Software/#linda-parallel-gaussian-jobs","title":"Linda parallel Gaussian jobs","text":"

Only currently working for Gaussian 09.

Gaussian Linda jobs can run across multiple nodes.

# Select the MPI parallel environment and 80 cores total\n#$ -pe mpi 80\n\n# 8. Select number of threads per Linda worker (value of NProcShared in your\n#     Gaussian input file. This will give 80/40 = 2 Linda workers.\nexport OMP_NUM_THREADS=40\n\n# Setup g09 runtime environment\nmodule load gaussian/g09-d01/pgi-2015.7\nsource $g09root/g09/bsd/g09.profile\n\n# Pre-process g09 input file to include nodes allocated to job\necho \"Running: lindaConv testdata.com $JOB_ID $TMPDIR/machines\"\necho ''\n$lindaConv testdata.com $JOB_ID $TMPDIR/machines\n\n# Run g09 job\n\necho \"Running: g09 \\\"job$JOB_ID.com\\\"\"\n\n# communication needs to be via ssh not the Linda default\nexport GAUSS_LFLAGS='-v -opt \"Tsnet.Node.lindarsharg: ssh\"'\n\ng09 \"job$JOB_ID.com\"\n
"},{"location":"Software_Guides/Other_Software/#troubleshooting-memory-errors","title":"Troubleshooting: Memory errors","text":"

If you encounter errors like:

Out-of-memory error in routine ShPair-LoodLd2 (IEnd= 257724 MxCore= 242934)\n\nUse %mem=48MW to provide the minimum amount of memory required to complete this step.\n

Try adding this to your jobscript:

export GAUSS_MEMDEF=48000000\n

You may need to increase this value even more to allow it to run.

"},{"location":"Software_Guides/Other_Software/#troubleshooting-no-space-left-on-device","title":"Troubleshooting: No space left on device","text":"

If you get this error

  g_write: No space left on device\n

The $GAUSS_SCRDIR is probably full - if it was on a cluster that has local disks and is using $TMPDIR you should increase the amount of tmpfs you are requesting in your jobscript. Otherwise check lquota for your data usage and potentially request a larger Scratch.

"},{"location":"Software_Guides/Other_Software/#gromacs","title":"GROMACS","text":"

We have many versions of GROMACS installed, some built with Plumed. The module name will indicate this.

Which executable you should run depends on the problem you wish to solve. For both single and double precision version builds, serial binaries and an MPI binary for mdrun (mdrun_mpi for newer versions, gmx_mpi for Plumed and some older versions) are provided. Double precision binaries have a _d suffix (so gmx_d, mdrun_mpi_d, gmx_mpi_d etc).

You can see what the executable names are by running module show gromacs/2021.2/gnu-7.3.0 for example and then running the ls command on the bin directory that the module tells you that version is installed in.

# Example for GPU gromacs/2021.5/cuda-11.3\nmodule load beta-modules\nmodule unload -f compilers mpi gcc-libs\nmodule load gcc-libs/10.2.0\nmodule load compilers/gnu/10.2.0\nmodule load\u00a0python3/3.9-gnu-10.2.0\u00a0\nmodule load cuda/11.3.1/gnu-10.2.0\nmodule load mpi/openmpi/4.0.5/gnu-10.2.0\nmodule load\u00a0gromacs/2021.5/cuda-11.3\n\n# Run GROMACS - the executables are gmx_cuda, gmx_mpi_cuda and mdrun_mpi_cuda\n
# Example for gromacs/2021.2/gnu-7.3.0\nmodule load beta-modules\nmodule unload -f compilers mpi gcc-libs\nmodule load gcc-libs/7.3.0\nmodule load compilers/gnu/7.3.0\nmodule load mpi/openmpi/3.1.4/gnu-7.3.0\nmodule load python3\nmodule load gromacs/2021.2/gnu-7.3.0\n\n# Run GROMACS - replace with mdrun command line suitable for your job!\n\ngerun mdrun_mpi -v -stepout 10000\n
# Example for gromacs/2019.3/intel-2018\nmodule unload -f compilers mpi\nmodule load compilers/intel/2018/update3\nmodule load mpi/intel/2018/update3/intel\nmodule load gromacs/2019.3/intel-2018\n\n# Run GROMACS - replace with mdrun command line suitable for your job!\n\ngerun mdrun_mpi -v -stepout 10000\n
# Plumed example for gromacs/2019.3/plumed/intel-2018\nmodule unload -f compilers mpi\nmodule load compilers/intel/2018/update3 \nmodule load mpi/intel/2018/update3/intel \nmodule load libmatheval \nmodule load flex \nmodule load plumed/2.5.2/intel-2018\nmodule load gromacs/2019.3/plumed/intel-2018\n\n# Run GROMACS - replace with mdrun command line suitable for your job!\n\ngerun gmx_mpi -v -stepout 10000\n
"},{"location":"Software_Guides/Other_Software/#passing-in-options-to-gromacs-non-interactively","title":"Passing in options to GROMACS non-interactively","text":"

Some GROMACS executables like trjconv normally take interactive input. You can't do this in a jobscript, so you need to pass in the input you would normally type in. There are several ways of doing this, mentioned at GROMACS Documentation - Using Commands in Scripts. The simplest is to echo the input in and keep your gmx options as they would normally be. If the inputs you would normally type were 3 and 3, then you can do this:

echo 3 3 | gmx whatevercommand -options\n
"},{"location":"Software_Guides/Other_Software/#checkpoint-and-restart","title":"Checkpoint and restart","text":"

GROMACS has built-in checkpoint and restart ability, so you can use this if your runs will not complete in the maximum 48hr wallclock time.

Have a look at the GROMACS manual for full details, as there are more options than mentioned here.

You can tell GROMACS to write a checkpoint file when it is approaching the maximum wallclock time available, and then exit.

In this case, we had asked for 48hrs wallclock. This tells GROMACS to start from the last checkpoint if there is one, and write a new checkpoint just before it reaches 47 hrs runtime.

gerun mdrun_mpi -cpi -maxh 47 <options>\n

The next job you submit with the same script will carry on from the checkpoint the last job wrote. You could use job dependencies to submit two identical jobs at the same time and have one dependent on the other, so it won't start until the first finishes - have a look at man qsub for the -hold_jid option.

You can also write checkpoints at given intervals:

# Write checkpoints every 120 mins, start from checkpoint if there is one.\ngerun mdrun_mpi -cpi -cpt 120 <options>\n
"},{"location":"Software_Guides/Other_Software/#hammock","title":"Hammock","text":"

Hammock is a tool for peptide sequence clustering. It is able to cluster extremely large amounts of short peptide sequences into groups sharing sequence motifs. Typical Hammock applications are NGS-based experiments using large combinatorial peptide libraries, e.g. Phage display.

Hammock has to be installed in your own space to function, so we provide a hammock module that contains the main dependencies and creates a quick-install alias:

# on the login nodes\nmodule unload compilers\nmodule load hammock/1.0.5\ndo-hammock-install\n

This will install Hammock 1.0.5 in your home, edit settings.prop to use clustal-omega and hmmer from our modules and tell it to write temporary files in your Scratch directory (in the form Hammock_temp_time). 

# in your jobscript\nmodule unload compilers\nmodule load hammock/1.0.5\n\n# This copies the MUSI example that comes with Hammock into your working\n# directory and runs it. The module sets $HAMMOCKPATH for you. \n# You must set the output directory to somewhere in Scratch with -d. \n# Below makes a different outputdir per job so multiple runs don't overwrite files.\n\ncp $HAMMOCKPATH/../examples/MUSI/musi.fa .\noutputdir=~/Scratch/hammock-examples/musi_$JOB_ID\nmkdir -p $outputdir\necho \"Running java -jar $HAMMOCKPATH/Hammock.jar full -i musi.fa -d $outputdir\"\n\njava -jar $HAMMOCKPATH/Hammock.jar full -i musi.fa -d $outputdir\n
"},{"location":"Software_Guides/Other_Software/#hopspack","title":"HOPSPACK","text":"

HOPSPACK (Hybrid Optimization Parallel Search PACKage) solves derivative-free optimization problems using an open source, C++ software framework.

We have versions of HOPSPACK built using the GNU compiler and OpenMPI, and the Intel compiler and MPI. This example shows the GNU version. Serial and parallel versions are available, HOPSPACK_main_mpi and HOPSPACK_main_serial.

module unload compilers\nmodule unload mpi\nmodule load compilers/gnu/4.9.2\nmodule load mpi/openmpi/1.8.4/gnu-4.9.2\nmodule load atlas/3.10.2/gnu-4.9.2\nmodule load hopspack/2.0.2/gnu-4.9.2\n\n# Add the examples directory we are using to our path. \n# Replace this with the path to your own executables.\nexport PATH=$PATH:~/Scratch/examples/1-var-bnds-only/\n\n# Run parallel HOPSPACK.\n# Gerun is our mpirun wrapper which sets the machinefile and number of\n# processes to the amount you requested with -pe mpi.\ngerun HOPSPACK_main_mpi ~/Scratch/examples/1-var-bnds-only/example1_params.txt > example1_output.txt\n
"},{"location":"Software_Guides/Other_Software/#idl","title":"IDL","text":"

IDL is a complete environment and language for the analysis and visualisation of scientific and other technical data. It can be used for everything from quick interactive data exploration to building complex applications.

Single-threaded jobscript:

cd $TMPDIR\n\nmodule load idl/8.4.1\n\n# Copy IDL source files to $TMPDIR\ncp ~/Scratch/IDL/fib.pro $TMPDIR\ncp ~/Scratch/IDL/run1.pro $TMPDIR\n\nidl -queue -e @run1.pro\n\n# tar up all contents of $TMPDIR back into your space\ntar zcvf $HOME/Scratch/IDL_output/files_from_job_$JOB_ID.tgz $TMPDIR\n

Parallel jobscript:

cd $TMPDIR\n\nmodule load idl/8.1\n\n# this sets the IDL thread pool: do not change this\nexport IDL_CPU_TPOOL_NTHREADS=$OMP_NUM_THREADS\n\n# Copy IDL source files to $TMPDIR\ncp ~/Scratch/IDL/fib.pro $TMPDIR\ncp ~/Scratch/IDL/run2mp.pro $TMPDIR\n\nidl -queue -e @run2mp.pro\n\n# tar up all contents of $TMPDIR back into your space\ntar zcvf $HOME/Scratch/IDL_output/files_from_job_$JOB_ID.tgz $TMPDIR\n
"},{"location":"Software_Guides/Other_Software/#jags","title":"JAGS","text":"

JAGS (Just Another Gibbs Sampler) is a program for analysis of Bayesian hierarchical models using Markov Chain Monte Carlo (MCMC) simulation not wholly unlike BUGS.

Use this to use JAGS in standalone command line mode:

module unload compilers mpi\nmodule load compilers/gnu/4.9.2\nmodule load openblas/0.2.14/gnu-4.9.2\nmodule load jags/4.2.0/gnu.4.9.2-openblas\n

We have also added JAGS support to r/recommended using the rjags and R2jags R packages.

"},{"location":"Software_Guides/Other_Software/#lammps","title":"LAMMPS","text":"

LAMMPS is an open source parallel molecular dynamics code which exhibits good scaling in a wide range of environments.

The LAMMPS binaries are called lmp_$cluster and all have an lmp_default symlink which can be used.

LAMMPS-8Dec15 and later were built with additional packages kspace, replica, rigid, and class2.

The versions from lammps-16Mar18-basic_install onwards (not lammps/16Mar18/intel-2017) have most of the included packages built. There are also userintel and gpu versions from this point.

We do not install the LAMMPS user packages as part of our central install, but you can build your own version with the ones that you want in your space.

module -f unload compilers mpi\nmodule load compilers/intel/2018\nmodule load mpi/intel/2018\nmodule load lammps/16Mar18/basic/intel-2018\n\n# Gerun is our mpirun wrapper which sets the machinefile and number of\n# processes to the amount you requested with -pe mpi.\ngerun $(which lmp_default) -in inputfile\n

For the latest version of LAMMPS we have installed which is 29th September 2021 Update 2 where the binaries are called lmp_mpi for the MPI version and lmp_gpu for the GPU version:

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load compilers/gnu/10.2.0\nmodule load mpi/openmpi/4.0.5/gnu-10.2.0\nmodule load python3/3.9-gnu-10.2.0\nmodule load lammps/29sep21up2/basic/gnu-10.2.0\n\ngerun lmp_mpi -in inputfile\n

for the basic MPI version and:

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load compilers/gnu/10.2.0\n\n# these three modules for Myriad only\nmodule load numactl/2.0.12\nmodule load binutils/2.36.1/gnu-10.2.0\nmodule load ucx/1.9.0/gnu-10.2.0\n\nmodule load mpi/openmpi/4.0.5/gnu-10.2.0\nmodule load cuda/11.3.1/gnu-10.2.0\nmodule load python3/3.9-gnu-10.2.0\nmodule load lammps/29sep21up2/gpu/gnu-10.2.0\n\ngerun lmp_gpu -sf gpu -pk gpu 1 -in inputfile\n

for the version with GPU support which is only available on clusters with GPUs. The MPI version is available on all clusters. On Myriad the numactl, binutils and ucx modules are additionally needed by OpenMPI.

LAMMPS 29th September 2021 Update 2 has been built with the GNU compilers, OpenMPI and CUDA for the GPU version.

We also have Intel installs:

# LAMMPS 29 Sep 2021 Update 2 with Intel compilers and INTEL package\nmodule unload -f compilers mpi\nmodule load compilers/intel/2020/release\nmodule load mpi/intel/2019/update6/intel\nmodule load python/3.9.10\nmodule load lammps/29sep21up2/userintel/intel-2020\n\ngerun lmp_mpi -in inputfile\n
# LAMMPS 29 Sep 2021 Update 2 for GPU with Intel compilers\nmodule unload -f compilers mpi\nmodule load beta-modules\nmodule load compilers/intel/2020/release\nmodule load mpi/intel/2019/update6/intel\nmodule load python/3.9.10\nmodule load cuda/11.3.1/gnu-10.2.0\nmodule load lammps/29sep21up2/gpu/intel-2020\n\ngerun lmp_gpu -sf gpu -pk gpu 1 -in inputfile\n
"},{"location":"Software_Guides/Other_Software/#meme-suite","title":"MEME Suite","text":"

MEME Suite: Motif-based sequence analysis tools. This install is for the command-line tools and connects to their website for further analysis. 

module unload compilers\nmodule unload mpi\nmodule load compilers/gnu/4.9.2\nmodule load mpi/openmpi/1.8.4/gnu-4.9.2\nmodule load perl/5.22.0\nmodule load python2/recommended\nmodule load ghostscript/9.16/gnu-4.9.2\nmodule load meme/4.10.1_4\n
"},{"location":"Software_Guides/Other_Software/#mirdeep2","title":"miRDeep2","text":"

Discovering known and novel miRNAs from deep sequencing data, miRDeep2 is a completely overhauled tool which discovers microRNA genes by analyzing sequenced RNAs. The tool reports known and hundreds of novel microRNAs with high accuracy in seven species representing the major animal clades. 

module load squid/1.9g\nmodule load randfold/2.0\nmodule load perl/5.22.0\nmodule load bowtie/1.1.2\nmodule load python/2.7.9\nmodule load viennarna/2.1.9\nmodule load mirdeep/2.0.0.7\n
"},{"location":"Software_Guides/Other_Software/#misomisopy","title":"MISO/misopy","text":"

MISO (Mixture of Isoforms) is a probabilistic framework that quantitates the expression level of alternatively spliced genes from RNA-Seq data, and identifies differentially regulated isoforms or exons across samples.

misopy is available as part of the python2/recommended bundle.

MISO can run multithreaded on one node, or can submit multiple independent single-core jobs at once using the --use-cluster option.

If you want to use MISO's ability to create and submit jobs itself, you need a MISO settings file like the one shown below. You give your job options as arguments to the qsub command in the cluster_command line.

Settings files can be used with the --settings-filename=SETTINGS_FILENAME option. You will also need to put your module unload and load commands in your .bashrc if using MISO's own job submission, because you are no longer including them in a jobscript.

Example miso_settings.txt. Multithreaded jobs will use num_processors. num_processors is ignored if --use-cluster is specified:

[data]\nfilter_results = True\nmin_event_reads = 20\n\n[cluster]\ncluster_command = \"qsub -l h_rt=00:10:00 -l mem=1GB -wd ~/Scratch\"\n\n[sampler]\nburn_in = 500\nlag = 10\nnum_iters = 5000\nnum_chains = 6\nnum_processors = 4\n
"},{"location":"Software_Guides/Other_Software/#molpro","title":"MOLPRO","text":"

Molpro is a complete system of ab initio programs for molecular electronic structure calculations.

Molpro 2015.1.3 was provided as binary only and supports communication over Ethernet and not Infiniband - use this one on single-node jobs primarily.

Molpro 2015.1.5 was built from source with the Intel compilers and Intel MPI, so can be run multi-node.

Molpro 2020.1 is a more recent binary install and supports both.

module load molpro/2015.1.5/intel-2015-update2\n\n# Example files available in /shared/ucl/apps/molpro/2015.1.5/intel-2015-update2/molprop_2015_1_linux_x86_64_i8/examples/\n# If this is a multi-node job you need to set the wavefunction directory to \n# somewhere in Scratch with -W. For a single-node job -W should be in $TMPDIR.\n# You can use $SGE_O_WORKDIR to refer to the directory you set with -wd in your jobscript.\n# $NSLOTS will use the number of cores you requested with -pe mpi.\n\necho \"Running molpro -n $NSLOTS -W $TMPDIR h2o_scf.com\"\n\nmolpro -n $NSLOTS -W $TMPDIR h2o_scf.com\n

On Myriad, if you get this error with the MPI 2015 install, please use the binary 2015.1.3 install.

libi40iw-i40iw_ucreate_qp: failed to create QP, unsupported QP type: 0x4\n

Output: MOLPRO can end up writing very many small output files, and this is detrimental to the performance of a parallel filesystem like Lustre. If you are running jobs on Myriad then you should set your -I -d and (especially) -W directories to be in $TMPDIR so they can be accessed quickly and not slow down other jobs. At the end of the job, copy back the data you want to keep into your Scratch.

If you are running parallel multi-node jobs and the directories need to be readable by all the nodes, then you need to write to Scratch.

"},{"location":"Software_Guides/Other_Software/#mrtrix","title":"MRtrix","text":"

MRtrix provides a set of tools to perform diffusion-weighted MRI white matter tractography in the presence of crossing fibres. 

module load python3/recommended\nmodule load qt/4.8.6/gnu-4.9.2\nmodule load eigen/3.2.5/gnu-4.9.2\nmodule load fftw/3.3.6-pl2/gnu-4.9.2\nmodule load mrtrix/3.0rc3/gnu-4.9.2/nogui\n

You must load these modules once from a login node before submitting a job. It copies a .mrtrix.conf to your home directory the first time you run this module from a login node, which sets:

  Analyse.LeftToRight: false\n  NumberOfThreads: 4\n

You need to alter NumberOfThreads to what you are using in your job script before you submit a job.

The MRtrix GUI tools are unavailable: mrview and shview in MRtrix 3 cannot be run over a remote X11 connection so are not usable on our clusters. To use these tools you will need a local install on your own computer.

"},{"location":"Software_Guides/Other_Software/#mutect","title":"MuTect","text":"

MuTect is a tool developed at the Broad Institute for the reliable and accurate identification of somatic point mutations in next generation sequencing data of cancer genomes. It is built on top of the GenomeAnalysisToolkit (GATK), which is also developed at the Broad Institute, so it uses the same command-line conventions and (almost all) the same input and output file formats.

MuTect requires you to agree to the GATK license before we can add you to the lgmutect group which gives you access: you can do this by downloading MuTect from The Broad Institute CGA page. You may need to create a gatkforums account before you can download.

MuTect is currently not compatible with Java 1.8, so you need to use the system Java 1.7. Set up your modules as follows: 

module load mutect/1.1.7\n

Then to run MuTect, you should either prefix the .jar you want to run with $MUTECTPATH:

java -Xmx2g -jar $MUTECTPATH/mutect-1.1.7.jar OPTION1=value1 OPTION2=value2...\n

Or we provide wrappers, so you can run it this way instead: 

mutect OPTION1=value1 OPTION2=value2...\n
"},{"location":"Software_Guides/Other_Software/#namd","title":"NAMD","text":"

NAMD is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems.

We have several different types of install, some of them suited to particular clusters only. To see all the versions, type module avail namd.

These examples are running the apoa1 benchmark, available from the NAMD website.

"},{"location":"Software_Guides/Other_Software/#multicore-gpu","title":"Multicore GPU","text":"

This version of NAMD runs within one GPU node. It can run on multiple GPUs on that node, but not across multiple different nodes. NAMD uses the CPUs and GPUs together so it is recommended you request all the cores on the node if you are requesting all the GPUs.

For best performance of simulations it is recommended that you use an entire node, all the CPUs and all the available GPUs.

# request a number of CPU cores and GPUs\n#$ -pe smp 10\n#$ -l gpu=1\n\nmodule load namd/2.14/multicore-gpu\n\n# ${NSLOTS} will get the number of cores you asked for with -pe smp.\n# +setcpuaffinity is recommended to make sure threads are assigned to specific CPUs.\n\nnamd2 +p${NSLOTS} +setcpuaffinity apoa1_nve_cuda.namd\n
"},{"location":"Software_Guides/Other_Software/#ofi","title":"OFI","text":"

This version of NAMD is for clusters with OmniPath interconnects (not Myriad). It can run across multiple nodes. The OFI versions should use significantly less memory than the older MPI-based installs.

module unload -f compilers mpi\nmodule load compilers/intel/2019/update5\nmodule load mpi/intel/2019/update5/intel\nmodule load namd/2.14/ofi/intel-2019\n\n# ${NSLOTS} will get the number of cores you asked for with -pe.\n\ncharmrun +p${NSLOTS} namd2 apoa1.namd\n
"},{"location":"Software_Guides/Other_Software/#ofi-smp","title":"OFI-SMP","text":"

This version of NAMD runs with threads (smp) and processes and is for clusters with OmniPath interconnects (not Myriad). It can run across multiple nodes. The OFI versions should use significantly less memory than the older MPI-based installs.

module unload -f compilers mpi\nmodule load compilers/intel/2019/update5\nmodule load mpi/intel/2019/update5/intel\nmodule load namd/2.14/ofi-smp/intel-2019\n\n# ${NSLOTS} will get the number of cores you asked for with -pe.\n# +setcpuaffinity is recommended to make sure threads are assigned to specific CPUs.\n# ++ppn is the number of PEs (or worker threads) to create for each process.\n\ncharmrun +p${NSLOTS} namd2 apoa1.namd ++ppn2 +setcpuaffinity\n
"},{"location":"Software_Guides/Other_Software/#ofi-smp-gpu","title":"OFI-SMP-GPU","text":"

This version of NAMD runs with threads (smp) and processes and is for clusters with OmniPath interconnects as well as GPUs (not Myriad). It can run across multiple nodes.

# request a number of CPU cores and GPUs\n#$ -pe smp 24\n#$ -l gpu=2\n\nmodule unload -f compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/7.3.0\nmodule load compilers/intel/2019/update5\nmodule load mpi/intel/2019/update5/intel\nmodule load cuda/11.3.1/gnu-10.2.0\nmodule load namd/2.14/ofi-smp-gpu/intel-2019\n\n# ${NSLOTS} will get the number of cores you asked for with -pe.\n# +setcpuaffinity is recommended to make sure threads are assigned to specific CPUs.\n# ++ppn is the number of PEs (or worker threads) to create for each process.\n\n# The number of GPU devices must be a multiple of the number of NAMD processes\n# since processes cannot share GPUs.\n# Here we have ++ppn12 for 12 threads, and charmrun works out we have 2 NAMD processes \n# available for the 2 GPUs.\n\ncharmrun +p${NSLOTS} namd2 apoa1_nve_cuda.namd ++ppn12 +setcpuaffinity \n
"},{"location":"Software_Guides/Other_Software/#mpi","title":"MPI","text":"

These are older versions. It is recommended to run the OFI versions above instead if possible.

module load fftw/2.1.5/intel-2015-update2\nmodule load namd/2.13/intel-2018-update3\n\n# GErun is our mpirun wrapper that gets $NSLOTS and the machinefile for you.\n\ngerun namd2 apoa1.namd \n
"},{"location":"Software_Guides/Other_Software/#nextflow","title":"Nextflow","text":"

We do not currently have central installs of Nextflow, but a group of UCL researchers have contributed a config file and instructions for Myriad at the nf-core/configs repository

Nextflow containers can be run using Singularity.

"},{"location":"Software_Guides/Other_Software/#nonmem","title":"NONMEM","text":"

NONMEM\u00ae is a nonlinear mixed effects modelling tool used in population pharmacokinetic / pharmacodynamic analysis.

We have one build that uses the GNU compiler and ATLAS and an Intel build using MKL. Both use Intel MPI.

This example uses the Intel build.

jobDir=example1_parallel_$JOB_ID\nmkdir $jobDir\n\n# Copy control and datafiles to jobDir\ncp /shared/ucl/apps/NONMEM/examples/foce_parallel.ctl $jobDir\ncp /shared/ucl/apps/NONMEM/examples/example1b.csv $jobDir\ncd $jobDir\n\nmodule unload compilers mpi\nmodule load compilers/intel/2015/update2\nmodule load mpi/intel/2015/update3/intel\nmodule load nonmem/7.3.0/intel-2015-update2\n\n# Create parafile for job using $TMPDIR/machines\nparafile.sh $TMPDIR/machines > example1.pnm\n\nnmfe73 foce_parallel.ctl example1.res -parafile=example1.pnm -background -maxlim=1 > example1.log\n
"},{"location":"Software_Guides/Other_Software/#nwchem","title":"NWChem","text":"

NWChem applies theoretical techniques to predict the structure, properties, and reactivity of chemical and biological species ranging in size from tens to millions of atoms.

You should load the NWChem module you wish to use once from a login node, as it will create a symlinked .nwchemrc in your home. 

module unload compilers mpi\nmodule load compilers/intel/2017/update4\nmodule load mpi/intel/2017/update3/intel\nmodule load python/2.7.12\nmodule load nwchem/6.8-47-gdf6c956/intel-2017\n\n# $NSLOTS will get the number of processes you asked for with -pe mpi.\nmpirun -np $NSLOTS -machinefile $TMPDIR/machines nwchem hpcvl_sample.nw\n
"},{"location":"Software_Guides/Other_Software/#nwchem-troubleshooting","title":"NWChem troubleshooting","text":"

If you get errors like this

{    0,    3}:  On entry to PDSTEDC parameter number   10 had an illegal value\n

then you are coming across an error in Intel MKL 2018, and should make sure you change to the Intel 2017 compiler module as shown above. (MKL versions are bundled with the corresponding Intel compiler modules).

If your run terminates with an error saying

ARMCI supports block process mapping only\n

then you are probably trying to use round-robin MPI process placement, which ARMCI does not like. gerun uses round-robin for Intel MPI by default as it works better in most cases. Use mpirun instead of gerun:

mpirun -np $NSLOTS -machinefile $TMPDIR/machines nwchem input.nw\n

If you get an error complaining about $NWCHEM_NWPW_LIBRARY similar to this:

warning:::::::::::::: from_compile\nNWCHEM_NWPW_LIBRARY is: <\n/dev/shm/tmp.VB3DpmjULc/nwchem-6.6/src/nwpw/libraryps/>\nbut file does not exist or you do not have access to it !\n------------------------------------------------------------------------\nnwpwlibfile: no nwpw library found 0\n

then your ~/.nwchemrc symlink is likely pointing to a different version that you used previously. Deleting the symlink and loading the module you want to use will recreate it correctly.

"},{"location":"Software_Guides/Other_Software/#orca","title":"ORCA","text":"

ORCA is an ab initio, DFT, and semi-empirical SCF-MO package.

module unload compilers \nmodule unload mpi\nmodule load compilers/gnu/4.9.2\nmodule load mpi/openmpi/3.1.4/gnu-4.9.2\nmodule load orca/4.2.1-bindist/gnu-4.9.2\n\norca input.inp > output.out\n

If you want to run ORCA in parallel using MPI, the jobscript will be the same but you will need to add the !PAL keyword to your input file to tell it how many processes to use. (You do not use mpirun or gerun with ORCA).

"},{"location":"Software_Guides/Other_Software/#picard","title":"Picard","text":"

Picard comprises Java-based command-line utilities that manipulate SAM files, and a Java API (SAM-JDK) for creating new programs that read and write SAM files. Both SAM text format and SAM binary (BAM) format are supported.

Picard requires a Java 1.8 module to be loaded.

module load java/1.8.0_92\nmodule load picard-tools/2.18.9\n

To run Picard you can prefix the .jar you want to run with $PICARDPATH and give the full command, or we have wrappers:

java -Xmx2g -jar $PICARDPATH/picard.jar PicardCommand TMP_DIR=$TMPDIR OPTION1=value1 OPTION2=value2...\n

The wrappers allow you to run commands like this - in this case our wrapper sets TMP_DIR for you as well: 

PicardCommand OPTION1=value1 OPTION2=value2...\n

Temporary files: by default, Picard writes temporary files into /tmp rather than into $TMPDIR. These are not cleaned up after your job ends, and means future runs can fail because /tmp is full (and requesting more tmpfs in your job doesn't make it larger). If you run Picard with the full java -jar command then give Picard the TMP_DIR=$TMPDIR option as our example above does to get it to write in the correct place.

"},{"location":"Software_Guides/Other_Software/#quantum-espresso","title":"Quantum Espresso","text":"

Quantum Espresso is an integrated suite of Open-Source computer codes for electronic-structure calculations and materials modelling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials. 

module load xorg-utils\nmodule load quantum-espresso/6.1-impi/intel2017\n\n# Set the path here to where ever you keep your pseudopotentials.\nexport ESPRESSO_PSEUDO=$HOME/qe-psp\n\n# Gerun is our mpirun wrapper which sets the machinefile and number of\n# processes to the amount you requested with -pe mpi.\ngerun pw.x -in input.in >output.out\n
"},{"location":"Software_Guides/Other_Software/#repast-hpc","title":"Repast HPC","text":"

Repast for High Performance Computing (Repast HPC) is a next generation agent-based modelling system intended for large-scale distributed computing platforms. It implements the core Repast Simphony concepts (e.g. contexts and projections), modifying them to work in a parallel distributed environment. 

module unload compilers\nmodule unload mpi\nmodule load compilers/gnu/4.9.2\nmodule load hdf/5-1.8.15/gnu-4.9.2\nmodule load netcdf/4.3.3.1/gnu-4.9.2\nmodule load netcdf-fortran/4.4.1/gnu-4.9.2\nmodule load mpi/openmpi/1.8.4/gnu-4.9.2\nmodule load python/2.7.9\nmodule load boost/1_54_0/mpi/gnu-4.9.2\nmodule load netcdf-c++/4.2/gnu-4.9.2\nmodule load repast-hpc/2.1/gnu-4.9.2\n

The module sets the environment variables $REPAST_HPC_INCLUDE, $REPAST_HPC_LIB_DIR and $REPAST_HPC_LIB.

"},{"location":"Software_Guides/Other_Software/#root","title":"ROOT","text":"

ROOT provides a set of OO frameworks for handling, analysing, and visualising large amounts of data. Included are specialised storage methods, methods for histograming, curve fitting, function evaluation, minimization etc. ROOT includes a built-in CINT C++ interpreter. 

module unload compilers mpi\nmodule load compilers/gnu/4.9.2\nmodule load fftw/3.3.4/gnu-4.9.2\nmodule load gsl/1.16/gnu-4.9.2\nmodule load root/6.04.00/gnu-4.9.2\n\n# run root in batch mode\nroot -b -q myMacro.C > myMacro.out\n
"},{"location":"Software_Guides/Other_Software/#sas","title":"SAS","text":"

SAS is a statistics package providing a wide range of tools for data management, analysis and presentation. 

cd $TMPDIR\n\nmodule load sas/9.4/64\n\n# copy all your input files into $TMPDIR\ncp ~/Scratch/sas_input/example1/* $TMPDIR\n\nsas example1.in\n\n# tar up all contents of $TMPDIR back into your space\ntar cvzf $HOME/Scratch/SAS_output/files_from_job_$JOB_ID.tgz $TMPDIR\n
"},{"location":"Software_Guides/Other_Software/#starccm","title":"StarCCM+","text":"

StarCCM+ is a commercial CFD package that handles fluid flows, heat transfer, stress simulations, and other common applications of such.

Before running any StarCCM+ jobs on the clusters you must load the StarCCM+ module on a login node. This is so the module can set up two symbolic links in your home directory to directories created in your Scratch area so that user settings etc can be written by running jobs.

module load star-ccm+/13.06.012\n

Here is the jobscript example.

# Request one license per core - makes sure your job doesn't start \n# running until sufficient licenses are free.\n#$ -l ccmpsuite=1\n\nmodule load star-ccm+/13.06.012\n\nstarccm+ -np $NSLOTS -machinefile $TMPDIR/machines -rsh ssh -batch my_input.sim\n
"},{"location":"Software_Guides/Other_Software/#hfi-error","title":"hfi error","text":"

If you get an error like this:

hfi_wait_for_device: The /dev/hfi1_0 device failed to appear after 15.0 seconds: Connection timed out\n

then you need to add -fabric ibv to your options as shown in the example script.

It is trying to use an OmniPath device on a cluster that has InfiniBand, so the fabric needs to be changed. If you have this left over in jobscripts from Grace, you need to remove it on Kathleen.

"},{"location":"Software_Guides/Other_Software/#starcd","title":"StarCD","text":"

StarCD is a commercial package for modelling and simulating combustion and engine dynamics.

You must request access to the group controlling StarCD access (legstarc) to use it. The license is owned by the Department of Mechanical Engineering who will need to approve your access request. 

# Request one license per core - makes sure your job doesn't start \n# running until sufficient licenses are free.\n#$ -l starsuite=1\n\nmodule load star-cd/4.28.050\n\n# run star without its tracker process as this causes multicore jobs to die early\nstar -notracker\n

StarCD uses IBM Platform MPI by default. You can also run StarCD simulations using Intel MPI by changing the command line to:

star -notracker -mpi=intel\n

Simulations run using Intel MPI may run faster than they do when using IBM Platform MPI.

If being run on a diskless cluster without available $TMPDIR like Kathleen, then StarCD will create a $HPC_SCRATCH location to store its temporary files when the module is loaded. In a job this is set to $HOME/Scratch/STAR_ScrDirs/[randomLabel] and it will make this directory and notify that it did this in your .e file. You can delete the randomly-named directory after the job ends. To set the location yourself, after you load the module you can set it to any other existing directory instead:

export HPC_SCRATCH=/path/to/desired/location\n
"},{"location":"Software_Guides/Other_Software/#statamp","title":"Stata/MP","text":"

Stata is a statistics, data management, and graphics system. Stata/MP is the version of the package that runs on multiple cores.

We have a sixteen user license of Stata/MP. Our license supports Stata running on up to four cores per job. 

# Select 4 OpenMP threads (the most possible)\n#$ -pe smp 4\n\ncd $TMPDIR\nmodule load stata/15\n\n# copy files to $TMPDIR\ncp myfile.do $TMPDIR\n\nstata-mp -b do myfile.do\n\n# tar up all contents of $TMPDIR back into your space\ntar zcvf $HOME/Scratch/Stata_output/files_from_job_$JOB_ID.tar.gz $TMPDIR\n
"},{"location":"Software_Guides/Other_Software/#torch","title":"Torch","text":"

Torch is a scientific computing framework with wide support for machine learning algorithms that puts GPUs first.

We provide a torch-deps module that contains the main Torch dependencies and creates a quick-install alias, do-torch-install. This uses Torch's installation script to git clone the current distribution and install LuaJIT, LuaRocks and Torch in ~/torch. 

module unload compilers mpi\nmodule load torch-deps\n\ndo-torch-install\n

You should load these same modules in your jobscript when using the version of torch this installs.

"},{"location":"Software_Guides/Other_Software/#turbomole","title":"Turbomole","text":"

Turbomole is an ab initio computational chemistry program that implements various quantum chemistry methods. Turbomole has a Chemistry-wide license. Reserved application group legtmole for Chemistry users only.

There are scripts you can use to generate Turbomole jobs for you:

/shared/ucl/apps/turbomole/turbomole-mpi.submit\n/shared/ucl/apps/turbomole/turbomole-smp.submit\n

They will ask you which version you want to use, how much memory, how many cores etc and set up and submit the job for you.

Use the first for MPI jobs and the second for single-node shared memory threaded jobs.

"},{"location":"Software_Guides/Other_Software/#varscan","title":"VarScan","text":"

VarScan is a platform-independent mutation caller for targeted, exome, and whole-genome resequencing data generated on Illumina, SOLiD, Life/PGM, Roche/454, and similar instruments. 

module load java/1.8.0_45\nmodule load varscan/2.3.9\n

Then to run VarScan, you should either prefix the .jar you want to run with $VARSCANPATH:

java -Xmx2g -jar $VARSCANPATH/VarScan.v2.3.9.jar OPTION1=value1 OPTION2=value2...\n

Or we provide wrappers, so you can run it this way instead:

varscan OPTION1=value1 OPTION2=value2...\n
"},{"location":"Software_Guides/Other_Software/#vasp","title":"VASP","text":"

The Vienna Ab initio Simulation Package (VASP) is a computer program for atomic scale materials modelling, e.g. electronic structure calculations and quantum-mechanical molecular dynamics, from first principles.

VASP is licensed software. To gain access, you need to email us letting us know what email address you are named on a VASP license using. You can also mention the name and email of the main VASP license holder and the license number if you have it, though this is not necessary. We will then check in the VASP portal if we can give you access. We will add you to the legvasp5 or legvasp6 reserved application groups depending on which versions you are licensed for, and remove you when VASP tell us you no longer have access.

The VASP executables for current versions are named like this:

"},{"location":"Software_Guides/Other_Software/#vasp-5","title":"VASP 5","text":"
# vasp 5\nmodule unload -f compilers mpi\nmodule load compilers/intel/2017/update1\nmodule load mpi/intel/2017/update1/intel\nmodule load vasp/5.4.4-18apr2017/intel-2017-update1\n\n# Gerun is our mpirun wrapper which sets the machinefile and number of\n# processes to the amount you requested with -pe mpi.\ngerun vasp_std > vasp_output.$JOB_ID\n

Note: although you can run VASP using the default Intel 2018 compiler this can lead to numerical errors in some types of simulation. In those cases we recommend switching to the specific compiler and MPI version used to build that install (mentioned at the end of the module name). We do this in the example above.

Building your own VASP: You may also install your own copy of VASP in your home if you have access to the source, and we provide a simple VASP individual install script (tested with VASP 5.4.4, no patches). You need to download the VASP source code into your home directory and then you can run the script following the instructions at the top.

"},{"location":"Software_Guides/Other_Software/#vasp-6","title":"VASP 6","text":"
# vasp 6\nmodule unload -f compilers mpi\nmodule load compilers/intel/2019/update5\nmodule load mpi/intel/2019/update5/intel\nmodule load vasp/6.3.0-24Jan2022/intel-2019-update5\n\ngerun vasp_std > vasp_output.$JOB_ID\n
"},{"location":"Software_Guides/Other_Software/#vasp-6-gpu","title":"VASP 6 GPU","text":"

This is the OpenACC GPU port of VASP. The VASP documentation has some information about suitable numbers of MPI processes vs GPUs.

# vasp 6 GPU\n\n# request a gpu\n#$ -l gpu=1\n\nmodule unload -f compilers mpi\nmodule load compilers/nvidia/hpc-sdk/22.1\nmodule load fftw/3.3.10/nvidia-22.1\nmodule load vasp/6.3.0-24Jan2022/nvidia-22.1-gpu\n\ngerun vasp_std > vasp_output.$JOB_ID\n
"},{"location":"Software_Guides/Other_Software/#xmds","title":"XMDS","text":"

XMDS allows the fast and easy solution of sets of ordinary, partial and stochastic differential equations, using a variety of efficient numerical algorithms.

We have XMDS 3 and XMDS 2 installed.

For XMDS 3.0.0 you will need to load the modules on a login node and run xmds3-setup to set up XMDS.

module unload compilers\nmodule unload mpi\nmodule load compilers/gnu/4.9.2\nmodule load mpi/intel/2015/update3/gnu-4.9.2\nmodule load python3/3.7\nmodule load fftw/3.3.4-impi/gnu-4.9.2\nmodule load hdf/5-1.8.15/gnu-4.9.2\nmodule load xmds/3.0.0\n\n# run this on a login node to set up XMDS\nxmds3-setup\n

You can also build the current developmental version from SVN in your space by running create-svn-xmds3-inst.

For XMDS 2.2.2 you will need to load the modules on a login node and run xmds2-setup to set up XMDS. 

module unload compilers\nmodule unload mpi\nmodule load compilers/gnu/4.9.2\nmodule load mpi/intel/2015/update3/gnu-4.9.2\nmodule load python2/recommended\nmodule load fftw/3.3.4-impi/gnu-4.9.2\nmodule load hdf/5-1.8.15/gnu-4.9.2\nmodule load xmds/2.2.2\n\n# run this on a login node to set up XMDS\nxmds2-setup\n

Note that the create-svn-xmds-inst SVN install using the 2.2.2 modules will no longer work since the release of XMDS 3.0.0 (see above to use that).

"},{"location":"Software_Guides/R/","title":"R","text":"

Type module avail r to see the currently available versions of R.

The current version will always also exist as r/recommended - this is a module bundle and loading it will also load its many dependencies.

module show r/recommended shows you exactly which versions loading this module will give you.

R can be run on a single core or multithreaded using many cores (some commands can run threaded automatically, otherwise you may wish to look at R's parallel package).

doMPI, Rmpi and snow allow multi-node parallel jobs using MPI to be run.

List of additional R packages shows you what packages are installed and available for the current R version.

"},{"location":"Software_Guides/R/#setup","title":"Setup","text":"

Before you can use R interactively, you need to load the R module using: 

module -f unload compilers mpi gcc-libs\nmodule load r/recommended\n
"},{"location":"Software_Guides/R/#example-serial-jobscript","title":"Example serial jobscript","text":"

This script runs R using only one core.

#!/bin/bash -l\n\n# Example jobscript to run a single core R job\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n# Change this to suit your requirements.\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM. Change this to suit your requirements.\n#$ -l mem=1G\n\n# Set the name of the job. You can change this if you wish.\n#$ -N R_job_1\n\n# Set the working directory to somewhere in your scratch space.  This is\n# necessary because the compute nodes cannot write to your $HOME\n# NOTE: this directory must exist.\n# Replace \"<your_UCL_id>\" with your UCL user ID\n#$ -wd /home/<your_UCL_id>/Scratch/R_output\n\n# Your work must be done in $TMPDIR (serial jobs particularly) \ncd $TMPDIR\n\n# Load the R module and run your R program\nmodule -f unload compilers mpi gcc-libs\nmodule load r/recommended\n\nR --no-save < /home/username/Scratch/myR_job.R > myR_job.out\n\n# Preferably, tar-up (archive) all output files to transfer them back \n# to your space. This will include the R_output file above.\ntar zcvf $HOME/Scratch/R_output/files_from_job_$JOB_ID.tgz $TMPDIR\n\n# Make sure you have given enough time for the copy to complete!\n

You will need to change the -wd /home/<your_UCL_id>/Scratch/R_output location and the location of your R input file, called myR_job.R here. myR_job.out is the file we are redirecting the output into. The output file is saved in the tar archive produced by the last command in the runscript and will be in $HOME/Scratch/R_output.

If your jobscript is called run-R.sh then your job submission command would be:

qsub run-R.sh\n
"},{"location":"Software_Guides/R/#example-shared-memory-threaded-parallel-job","title":"Example shared memory threaded parallel job","text":"

This script uses multiple cores on the same node. It cannot run across multiple nodes.

#!/bin/bash -l\n\n# Example jobscript to run an OpenMP threaded R job across multiple cores on one node.\n# This may be using the foreach packages foreach(...) %dopar% for example.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n# Change this to suit your requirements.\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM per core. Change this to suit your requirements.\n#$ -l mem=1G\n\n# Set the name of the job. You can change this if you wish.\n#$ -N R_jobMC_2\n\n# Select 12 threads. The number of threads here must equal the number of worker \n# processes in the registerDoMC call in your R program.\n#$ -pe smp 12\n\n# Set the working directory to somewhere in your scratch space.  This is\n# necessary because the compute nodes cannot write to your $HOME\n# NOTE: this directory must exist.\n# Replace \"<your_UCL_id>\" with your UCL user ID\n#$ -wd /home/<your_UCL_id>/Scratch/R_output\n\n# Your work must be done in $TMPDIR\ncd $TMPDIR\n\n# Load the R module and run your R program\nmodule -f unload compilers mpi gcc-libs\nmodule load r/recommended\n\nR --no-save < /home/username/Scratch/myR_job.R > myR_job.out\n\n# Preferably, tar-up (archive) all output files to transfer them back \n# to your space. This will include the R_output file above.\ntar zcvf $HOME/Scratch/R_output/files_from_job_$JOB_ID.tgz $TMPDIR\n\n# Make sure you have given enough time for the copy to complete!\n

You will need to change the -wd /home/<your_UCL_id>/Scratch/R_output location and the location of your R input file, called myR_job.R here. myR_job.out is the file we are redirecting the output into. The output file is saved in the tar archive produced by the last command in the runscript and will be in $HOME/Scratch/R_output.

If your jobscript is called run-R.sh then your job submission command would be:

qsub run-R.sh\n
"},{"location":"Software_Guides/R/#example-multi-node-parallel-job-using-rmpi-and-snow","title":"Example multi-node parallel job using Rmpi and snow","text":"

This script uses Rmpi and snow to allow it to run across multiple nodes using MPI.

#!/bin/bash -l\n\n# Example jobscript to run an R MPI parallel job\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM per process.\n#$ -l mem=1G\n\n# Request 15 gigabytes of TMPDIR space per node (default is 10 GB)\n#$ -l tmpfs=15G\n\n# Set the name of the job.\n#$ -N snow_monte_carlo\n\n# Select the MPI parallel environment with 32 processes\n#$ -pe mpi 32\n\n# Set the working directory to somewhere in your scratch space.  This is\n# necessary because the compute nodes cannot write to your $HOME\n# NOTE: this directory must exist.\n# Replace \"<your_UCL_id>\" with your UCL user ID\n#$ -wd /home/<your_UCL_id>/Scratch/R_output\n\n# Load the R module\nmodule -f unload compilers mpi gcc-libs\nmodule load r/recommended\n\n# Copy example files in to the working directory (not necessary if already there)\ncp ~/R/Examples/snow_example.R .\ncp ~/R/Examples/monte_carlo.R .\n\n# Run our MPI job. GERun is our wrapper for mpirun, which launches MPI jobs  \ngerun RMPISNOW < snow_example.R > snow.out.${JOB_ID}\n

The output file is saved in $HOME/Scratch/R_examples/snow/snow.out.${JOB_ID}.

If your jobscript is called run-R-snow.sh then your job submission command would be:

qsub run-R-snow.sh\n
"},{"location":"Software_Guides/R/#example-r-script-using-rmpi-and-snow","title":"Example R script using Rmpi and snow","text":"

This R script has been written to use Rmpi and snow and can be used with the above jobscript. It is snow_example.R above.

#Load the snow and random number package.\nlibrary(snow)\nlibrary(Rmpi)\n\n# This example uses the already installed LEcuyers RNG library(rlecuyer)\nlibrary(rlecuyer)\n\n# Set up our input/output\nsource('./monte_carlo.R')\nsink('./monte_carlo_output.txt')\n\n# Get a reference to our snow cluster that has been set up by the RMPISNOW\n# script.\ncl <- getMPIcluster ()\n\n# Display info about each process in the cluster\nprint(clusterCall(cl, function() Sys.info()))\n\n# Load the random number package on each R process\nclusterEvalQ (cl, library (rlecuyer))\n\n# Generate a seed for the pseudorandom number generator, unique to each\n# processor in the cluster.\n\n#Uncomment below line for default (unchanging) random number seed.\n#clusterSetupRNG(cl, type = 'RNGstream')\n\n#The lines below set up a time-based random number seed.  Note that \n#this only demonstrates the virtues of changing the seed; no guarantee\n#is made that this seed is at all useful.  Comment out if you uncomment\n#the above line.\ns <- sum(strtoi(charToRaw(date()), base = 32))\nclusterSetupRNGstream(cl, seed=rep(s,6))\n\n#Choose which of the following blocks best fit your own needs.\n\n# BLOCK 1\n# Set up the input to our Monte Carlo function.\n# Input is identical across the batch, only RNG seed has changed. \n# For this example, both clusters will roll one die. \n\nnrolls <- 2\nprint(\"Roll the dice once...\")\noutput <- clusterCall(cl, monte_carlo, nrolls)\noutput\nprint(\"Roll the dice again...\")\noutput <- clusterCall(cl, monte_carlo, nrolls)\noutput\n\n# Output should show the results of two rolls of a six-sided die.\n\n#BLOCK 2\n# Input is different for each processor\nprint(\"Second example: coin flip plus 3 dice\")\ninput <- array(1:2)  # Set up array of inputs, with each entry\ninput[1] <- 1        #   corresponding to one processor.\ninput[2] <- 3\nparameters <- array(1:2)  # Set up inputs that will be used by each cluster.\nparameters[1] <- 2        #   These will be passed to monte_carlo as its\nparameters[2] <- 6        #   second argument.\noutput <- clusterApply(cl, input, monte_carlo, parameters)\n\n# Output should show the results of a coin flip and the roll of three \n# six-sided die.\n\n# Output the output.\noutput\n\ninputStrings <- array(1:2)\ninputStrings[1] <- 'abc'\ninputStrings[2] <- 'def'\noutput <- clusterApply(cl, inputStrings, paste, 'foo')\noutput\n\n#clusterEvalQ(cl, sinkWorkerOutput(\"snow_monte_carlo.out\"))\n\n# Clean up the cluster and release the relevant resources.\nstopCluster(cl)\nsink()\n\nmpi.quit()\n

This is monte_carlo.R which is called by snow_example.R:

monte_carlo <- function(x, numsides=6){\n  streamname <- .lec.GetStreams ()\n  dice <- .lec.uniform.int(streamname[1], n = 1, a=1, b=numsides)\n  outp <- sum(dice)\n  return(outp)\n}\n

This example is based on SHARCNET's Using R and MPI.

"},{"location":"Software_Guides/R/#using-your-own-r-packages","title":"Using your own R packages","text":"

If we do not have R packages installed centrally that you wish to use, you can install them in your space on the cluster and tell R where to find them.

First you need to tell R where to install your package to and where to look for user-installed packages, using the R library path.

"},{"location":"Software_Guides/R/#set-your-r-library-path","title":"Set your R library path","text":"

There are several ways to modify your R library path so you can pick up packages that you have installed in your own space.

The easiest way is to add them to the R_LIBS environment variable (insert the correct path):

export R_LIBS=/your/local/R/library/path:$R_LIBS\n

This is a colon-separated list of directories that R will search through.

Setting that in your terminal will let you install to that path from inside R and should also be put in your jobscript (or your .bashrc) when you submit a job using those libraries. This appends your directory to the existing value of $R_LIBS rather than overwriting it so the centrally-installed libraries can still be found.

You can also change the library path for a session from within R:

.libPaths(c('~/MyRlibs',.libPaths()))\n

This puts your directory at the beginning of R's search path, and means that install.packages() will automatically put packages there and the library() function will find libraries in your local directory.

"},{"location":"Software_Guides/R/#install-an-r-package","title":"Install an R package","text":"

To install, after setting your library path:

From inside R, you can do

install.packages('package_name', repos=\"http://cran.r-project.org\")\n

Or if you have downloaded the tar file, you can do

R CMD INSTALL -l /home/username/your_R_libs_directory package.tar.gz\n

If you want to keep some libraries separate, you can have multiple colon-separated paths in your $R_LIBS and specify which one you want to install into with R CMD INSTALL.

"},{"location":"Software_Guides/R/#bioconductor","title":"BioConductor","text":"

If you are installing extra packages for BioConductor, check that you are using the same version that the R module you have loaded is using.

Eg. you can find the BioConductor 3.15 package downloads here.

"},{"location":"Software_Guides/Singularity/","title":"Using Singularity on Our Clusters","text":"

Singularity is installed on all our clusters. You can use containers you have downloaded in your space.

Run singularity --version to see which version we currently have installed.

"},{"location":"Software_Guides/Singularity/#set-up-cache-locations-and-bind-directories","title":"Set up cache locations and bind directories","text":"

The cache directories should be set to somewhere in your space so they don't fill up /tmp on the login nodes.

The bindpath mentioned below specifies what directories are made available inside the container - only your home is bound by default so you need to add Scratch.

You can either use the singularity-env environment module for this, or run the commands manually.

module load singularity-env\n

or:

# Create a .singularity directory in your Scratch\nmkdir $HOME/Scratch/.singularity\n\n# Create cache subdirectories we will use / export\nmkdir $HOME/Scratch/.singularity/tmp\nmkdir $HOME/Scratch/.singularity/localcache\nmkdir $HOME/Scratch/.singularity/pull\n\n# Set all the Singularity cache dirs to Scratch\nexport SINGULARITY_CACHEDIR=$HOME/Scratch/.singularity\nexport SINGULARITY_TMPDIR=$SINGULARITY_CACHEDIR/tmp\nexport SINGULARITY_LOCALCACHEDIR=$SINGULARITY_CACHEDIR/localcache\nexport SINGULARITY_PULLFOLDER=$SINGULARITY_CACHEDIR/pull\n\n# Bind your Scratch directory so it is accessible from inside the container\n#      and the temporary storage jobs are allocated\nexport SINGULARITY_BINDPATH=/scratch/scratch/$USER,/tmpdir\n

Different subdirectories are being set for each cache so you can tell which files came from where.

You probably want to add those export statements to your .bashrc under # User specific aliases and functions so those environment variables are always set when you log in.

For more information on these options, have a look at the Singularity documentation:

"},{"location":"Software_Guides/Singularity/#downloading-and-running-a-container","title":"Downloading and running a container","text":"

Assuming you want to run an existing container, first you need to pull it from somewhere online that provides it:

# make sure we set up singularity as above\nmodule load singularity-env\n\n# get image from location and call it hello-world.sif in our current directory\nsingularity pull hello-world.sif shub://vsoch/hello-world\n

Run the container.

singularity run hello-world.sif\n

Run a specific command within our container.

singularity exec hello-world.sif /bin/echo Hello World!\n

You can run containers inside jobscripts in the same way.

Useful links:

"},{"location":"Software_Guides/Singularity/#docker-containers","title":"Docker containers","text":"

You can use Singularity to run Docker containers. Docker itself is not suitable for use on a multi-user HPC system, but Singularity can convert and run Docker containers for you.

singularity pull python-3.9.6.sif docker://python:3.9.6-slim-buster\n

In this case, singularity pull is downloading a Docker image, and also converting it into a format that Singularity uses. You then use singularity run or singularity exec on the .sif image as above.

"},{"location":"Software_Guides/Singularity/#graphical-containers-in-interactive-jobs","title":"Graphical containers in interactive jobs","text":"

If you are trying to run a graphical application from inside a container in an interactive job and it is failing with errors about not being able to open a display, you will need to:

To do the bind mount, you could add it to your $SINGULARITY_BINDPATH

export SINGULARITY_BINDPATH=/scratch/scratch/$USER,/tmpdir,$HOME/.Xauthority\n

or you can pass it in with the --bind option to singularity shell or singularity exec.

"},{"location":"Supplementary/Compiling_Your_Code/","title":"Compiling Your Code","text":""},{"location":"Supplementary/Compiling_Your_Code/#download-your-code","title":"Download your code","text":"

Use wget or curl to download the source code for the software you want to install to your account. There might be binaries available, but they often won't work on our clusters because they were compiled for other machines with other library versions available. Use tar or unzip or similar depending on archive type to uncompress your source code.

wget\u00a0https://www.example.com/program.tar.gz\ntar\u00a0-xvf\u00a0program.tar.gz\n

You won't be able to use a package manager like yum, you'll need to follow the manual installation instructions for a user-space install (not using sudo).

"},{"location":"Supplementary/Compiling_Your_Code/#set-up-your-modules","title":"Set up your modules","text":"

Before you start compiling, you need to make sure you have the right compilers, libraries and other tools available for your software. If you haven't changed anything, you will have the default modules loaded.

Check what the instructions for your software tell you about compiling it. If the website doesn't say much, the source code will hopefully have a README or INSTALL file.

You may want to use a different compiler - the default is the Intel compiler.

module avail compilers will show you all the compiler modules available. Most Open Source software tends to assume you're using GCC and OpenMPI (if it uses MPI) and is most tested with that combination, so if it doesn't specify you may want to begin there (do check what the newest modules available are):

module\u00a0unload\u00a0compilers\u00a0mpi\u00a0mkl\nmodule\u00a0load\u00a0compilers/gnu/4.9.2  \nmodule\u00a0load\u00a0mpi/openmpi/1.10.1/gnu-4.9.2\n
"},{"location":"Supplementary/Compiling_Your_Code/#available-compilers","title":"Available compilers","text":"

The following compilers are available and supported on Legion:

We currently have a limited number of licenses for the Intel compilers so only a certain number of users can use them simultaneously. This means that your compilation may fail with an error complaining about not being able to obtain a valid license. If this happens, simply wait for a few minutes and try again.

In addition to the supported tools, there are a number of tools installed on Legion which are not supported (for example the PGI compilers) which were installed to build certain supported packages. Users who use the unsupported packages do so at their own risk.

"},{"location":"Supplementary/Compiling_Your_Code/#build-systems","title":"Build systems","text":"

Most software will use some kind of build system to manage how files are compiled and linked and in what order. Here are a few common ones.

"},{"location":"Supplementary/Compiling_Your_Code/#automake-configure","title":"Automake configure","text":"

Automake will generate the Makefile for you and hopefully pick up sensible options through configuration. You can give it an install prefix to tell it where to install (or you can build it in place and not use make install at all).

./configure\u00a0--prefix=/home/username/place/you/want/to/install  \nmake  \n#\u00a0if\u00a0it\u00a0has\u00a0a\u00a0test\u00a0suite,\u00a0good\u00a0idea\u00a0to\u00a0use\u00a0it  \nmake\u00a0test\u00a0  \nmake\u00a0install\n

If it has more configuration flags, you can use ./configure --help to view them.

Usually configure will create a config.log: you can look in there to find if any tests have failed or things you think should have been picked up haven't.

"},{"location":"Supplementary/Compiling_Your_Code/#cmake","title":"CMake","text":"

CMake is another build system. It will have a CMakeFile or the instructions will ask you to use cmake or ccmake rather than make. It also generates Makefiles for you. ccmake is a terminal-based interactive interface where you can see what variables are set to and change them, then repeatedly configure until everything is correct, generate the Makefile and quit. cmake is the commandline version. The process tends to go like this:

ccmake\u00a0CMakeLists.txt  \n#\u00a0press\u00a0c\u00a0to\u00a0configure\u00a0-\u00a0will\u00a0pick\u00a0up\u00a0some\u00a0options  \n#\u00a0press\u00a0t\u00a0to\u00a0toggle\u00a0advanced\u00a0options  \n#\u00a0keep\u00a0making\u00a0changes\u00a0and\u00a0configuring\u00a0until\u00a0no\u00a0more\u00a0errors\u00a0or\u00a0changes  \n#\u00a0press\u00a0g\u00a0to\u00a0generate\u00a0and\u00a0exit  \nmake  \n#\u00a0if\u00a0it\u00a0has\u00a0a\u00a0test\u00a0suite,\u00a0good\u00a0idea\u00a0to\u00a0use\u00a0it  \nmake\u00a0test\u00a0  \nmake\u00a0install\n

If you need to rerun ccmake and reconfigure, remember to delete the CMakeCache.txt file, or you'll be wondering why your changes haven't taken. Turning on verbose Makefiles in ccmake is also useful if your code didn't compile first time - you'll be able to see what flags the compiler or linker is actually being given when it fails.

"},{"location":"Supplementary/Compiling_Your_Code/#make","title":"Make","text":"

Your code may just come with a Makefile and have no configure, in which case the generic way to compile it is as follows:

make\u00a0targetname\n

There's usually a default target, which make on its own will use. If you need to change any configuration options, you'll need to edit those sections of the Makefile (at the top, where the variables/flags are defined).

Here are some typical variables you may want to change in a Makefile.

These are what compilers/mpi wrappers to use - these are also defined by the compiler modules, so you can see what they should be. Intel would be icc, icpc, ifort, for example. If it's a program that can be compiled using MPI and only has a variable for CC, then set that to mpicc.

CC=gcc  \nCXX=g++  \nFC=gfortran  \nMPICC=mpicc  \nMPICXX=mpicxx  \nMPIF90=mpif90\n

CFLAGS and LDFLAGS are flags for the compiler and linker respectively, and there might be LIBS or INCLUDE as well. When linking a library with the name libfoo, use -lfoo.

CFLAGS=\"-I/path/to/include\"  \nLDFLAGS=\"-L/path/to/foo/lib\u00a0-L/path/to/bar/lib\"  \nLDLIBS=\"-lfoo\u00a0-lbar\"\n

Remember to make clean first if you are recompiling with new options!

"},{"location":"Supplementary/Compiling_Your_Code/#blas-and-lapack","title":"BLAS and LAPACK","text":"

BLAS and LAPACK are provided as part of MKL, OpenBLAS or ATLAS. There are several different OpenBLAS and ATLAS modules on Legion for different compilers. MKL is available in the Intel compiler module.

Your code may try to link -lblas -llapack: this isn't the right way to use BLAS and LAPACK with MKL or ATLAS (our OpenBLAS now has symlinks that allow you to do this).

"},{"location":"Supplementary/Compiling_Your_Code/#set-your-path-and-other-environment-variables","title":"Set your PATH and other environment variables","text":"

After you have installed your software, you'll need to add it to your PATH environment variable so you can run it without having to give the full path to its location.

Put this in your ~/.bashrc file so it will set this with every new session you create. Replace username with your username and point to the directory your binary was built in (frequently program/bin). This adds it to the front of your PATH, so if you install a newer version of something, it will be found before the system one.

export\u00a0PATH=/home/username/location/of/software/binary:$PATH\n

If you built a library that you'll go on to compile other software with, you probably want to also add the lib directory to your LD_LIBRARY_PATH and LIBRARY_PATH, and the include directory to CPATH (add export statements as above). This may mean your configure step will pick your library up correctly without any further effort on your part.

To make these changes to your .bashrc take effect in your current session:

source\u00a0~/.bashrc\n
"},{"location":"Supplementary/Compiling_Your_Code/#python","title":"Python","text":"

There are python2/recommended and python3/recommended bundles. These use a virtualenv and have pip set up for you. They both have numpy and scipy available.

"},{"location":"Supplementary/Compiling_Your_Code/#set-compiler-module","title":"Set compiler module","text":"

The Python versions on Myriad were built with GCC. You can run them with the default Intel compilers loaded because everything depends on the gcc-libs/4.9.2 module. When you are building your own Python packages you should have the GCC compiler module loaded however, to avoid the situation where you build a package with the Intel compiler and then try to run it with GCC, in which case it will be unable to find Intel-specific instructions.

module\u00a0unload\u00a0compilers\nmodule\u00a0load\u00a0compilers/gnu/4.9.2\n

If you get an error like this when trying to run something, recheck what compiler you used.

undefined\u00a0symbol:\u00a0__intel_sse2_strrchr\n
"},{"location":"Supplementary/Compiling_Your_Code/#install-your-own-packages-in-the-same-virtualenv","title":"Install your own packages in the same virtualenv","text":"

This will use our central virtualenv, which contains a number of packages already installed.

pip\u00a0install\u00a0--user\u00a0<python2pkg>  \npip3\u00a0install\u00a0--user\u00a0<python3pkg>\n

These will install into .python2local or .python3local directories in your home directory, respectively.

To see what is already installed, the Python-shared list shows what is installed for both Python2 and 3, while the Python2 list and Python3 list show what is only installed for one or the other. (There may also be prereqs that aren't listed explicitly - pip will tell you if something is already installed as long as you have the recommended module bundle loaded).

"},{"location":"Supplementary/Compiling_Your_Code/#use-your-own-virtualenvs","title":"Use your own virtualenvs","text":"

If you need different packages that are not compatible with the central installs, you can create a new virtualenv and only yours will be available.

virtualenv\u00a0<DIR> \nsource\u00a0<DIR>/bin/activate\n

Your bash prompt will show you that a different virtualenv is active.

"},{"location":"Supplementary/Compiling_Your_Code/#installing-via-setuppy","title":"Installing via setup.py","text":"

If you need to install using setup.py, you can use the --user flag and as long as one of the python bundles is loaded, it will install into the same .python2local or .python3local as pip and you won't need to add any new paths to your environment.

python\u00a0setup.py\u00a0install\u00a0--user\n

You can alternatively use --prefix in which case you will have to set the install prefix to somewhere in your space, and also set PYTHONPATH and PATH to include your install location. Some installs won't create the prefix directory for you, in which case create it first. This is useful if you want to keep this package entirely separate and only in your paths on demand.

export\u00a0PYTHONPATH=/home/username/your/path/lib/python2.7/site-packages:$PYTHONPATH  \n#\u00a0if\u00a0necessary,\u00a0create\u00a0install\u00a0path  \nmkdir\u00a0-p\u00a0home/username/your/path/lib/python2.7/site-packages  \npython\u00a0setup.py\u00a0install\u00a0--prefix=/home/username/your/path\n\n#\u00a0add\u00a0these\u00a0to\u00a0your\u00a0.bashrc\u00a0or\u00a0jobscript  \nexport\u00a0PYTHONPATH=/home/username/your/path/lib/python2.7/site-packages:$PYTHONPATH  \nexport\u00a0PATH=/home/username/your/path/bin:$PATH\n

Check that the PATH is where your Python executables were installed, and the PYTHONPATH is correct. It will tend to tell you at install time if you need to change or create the PYTHONPATH directory.

"},{"location":"Supplementary/Compiling_Your_Code/#python-script-executable-paths","title":"Python script executable paths","text":"

If you have an executable python script giving the location of python like this, and it fails because that python doesn't exist in that location or isn't the one that has the additional packages installed:

#!/usr/bin/python2.7\n

You should change it so it uses the first python found in your environment. 

#!/usr/bin/env\u00a0python \n
"},{"location":"Supplementary/Compiling_Your_Code/#perl","title":"Perl","text":"

Perl modules will freqently have a Makefile.PL (especially if you download the tar files from CPAN.org yourself). You can install manually as:

perl\u00a0Makefile.PL\u00a0PREFIX=/home/username/your/perl/location\nmake\nmake\u00a0install\n
"},{"location":"Supplementary/Compiling_Your_Code/#cpan","title":"CPAN","text":"

You can use CPAN to download and install modules locally for you. The first time you run the cpan command, it will create a .cpan directory for you and ask you to give it configuration settings or allow it to set them automatically.

You need to tell it where you want your install prefix to be.

If it is automatically configured, you need to edit these lines in your .cpan/CPAN/MyConfig.pm, for example if you want it to be in a lib directory in your home (change username to your own username):

'make_install_arg'\u00a0=>\u00a0q[PREFIX=/home/username/lib],\n#\u00a0other\u00a0lines\u00a0in\u00a0here  \n'makepl_arg'\u00a0=>\u00a0q[PREFIX=/home/username/lib],\n'mbuild_install_arg'\u00a0=>\u00a0q[PREFIX=/home/username/lib],\n'mbuildpl_arg'\u00a0=>\u00a0q[--install_base\u00a0/home/username/lib],\n

It will download and build modules inside .cpan and install them where you specified.

"},{"location":"Supplementary/Compiling_Your_Code/#set-perl5lib-paths","title":"Set PERL5LIB paths","text":"

If you install your own Perl or Perl modules, you will need to append them to your PERL5LIB:

export\u00a0PERL5LIB=/home/username/your/perl/location:$PERL5LIB\n

If you installed with CPAN, you may need to add several paths to this based on the layout it creates inside your nominated Perl directory.

"},{"location":"Supplementary/Compiling_Your_Code/#errors-when-using-non-default-perl-versions","title":"Errors when using non-default Perl versions","text":""},{"location":"Supplementary/Compiling_Your_Code/#warningspm","title":"warnings.pm","text":"

If you are using a version of Perl that is not the default system Perl and get strange errors when trying to run a Perl script, particularly ones about warnings.pm:

Search\u00a0pattern\u00a0not\u00a0terminated\u00a0at\u00a0/shared/ucl/apps/perl/5.20.0/lib/5.20.0/warnings.pm\u00a0line\u00a01099\n

then you need to edit the script so that instead of beginning with #!/usr/bin/perl, it begins with #!/usr/bin/env perl. Otherwise it will try to use the old system Perl libraries with your newer Perl executable, which won't work.

"},{"location":"Supplementary/Compiling_Your_Code/#libperlso-not-found","title":"libperl.so not found","text":"

You probably built perl without telling it to build the shared library too. Add -Duseshrplib to your build flags.

"},{"location":"Supplementary/Compiling_Your_Code/#r","title":"R","text":"

There are instructions on installing and using local R packages in Using your own R packages.

"},{"location":"Supplementary/Compiling_Your_Code/#compiling-with-mpi","title":"Compiling with MPI","text":"

OpenMPI and Intel MPI are available. Certain programs do not work well with one or the other, so if you are having problems try the other one. Intel MPI is based on MPICH, so if the program you are compiling mentions that, try Intel MPI first.

The Intel MPI is threadsafe; some versions of OpenMPI aren't.

Note that OpenMPI 1.8.4 had a segv bug in non-blocking collectives that is fixed in OpenMPI 1.10.1.

"},{"location":"Supplementary/Compiling_Your_Code/#enabling-openmp","title":"Enabling OpenMP","text":"

To enable OpenMP with the Intel compilers, you simply need to add -openmp to your compile line. With the GNU compilers you need to add -fopenmp.

"},{"location":"Supplementary/Compiling_Your_Code/#problems","title":"Problems","text":"

If you experience problems building your applications, please contact your local IT support in the first instance. We are available at rc-support AT ucl.ac.uk to help you if you still cannot build your app or if you need to report a problem with our software stack.

"},{"location":"Supplementary/Compiling_and_Running_Matlab_Programs/","title":"Compiling and Running Matlab Programs","text":"

Although full Matlab is available on Myriad, you can also compile Matlab programs on an external machine and then run them on Myriad using the Matlab runtime.

There are some caveats, however:

"},{"location":"Supplementary/Compiling_and_Running_Matlab_Programs/#compiling-your-program","title":"Compiling your program:","text":"

The Matlab code is must be compiled using the mcc tool; this must be initially run as mcc -setup before anything is built. The mcc tool can actually be invoked from the interpreter command prompt and executing help mcc will give you quite a lot of information about how to use the tool, along with examples.

All .m files must be built into the compiled code with the first .m referenced in the build line acting as the main entry point for the built code. It may be useful to include data files in the built code which are handled in the build line using the -a <datafile> option. Please remember to make the .m file an actual function and all other dependencies sub-functions, otherwise the compiled code will not execute.

"},{"location":"Supplementary/Compiling_and_Running_Matlab_Programs/#some-important-mcc-options","title":"Some important mcc options:","text":""},{"location":"Supplementary/Compiling_and_Running_Matlab_Programs/#some-important-runtime-options","title":"Some important runtime options:","text":"

Once the application has been built, there should be an executable named after the prefix of the .m file, generally <app name>.m, and a shell script with the name run\\_<app name>.sh - both these files need to be transferred to Myriad.

If you have been given pre-compiled code by someone else, the application may not work as the Matlab runtime version must reasonably match that of the Matlab compiler that was used to build the application. The runtime is freely distributable and can be found in the installation directory of Matlab. The runtime has a GUI install interface and it can be installed at any location in your home directory.

For more information, please read the Matlab documentation.

"},{"location":"Supplementary/Compiling_and_Running_Matlab_Programs/#job-submission-scripts","title":"Job submission scripts:","text":"

There are three things that you must take into account:

  1. The location of the Matlab compiler runtime needs to be passed to the script used to run the compiled Matlab code as the first argument.
  2. The compiler runtime needs a directory (cache) to unpack files to when it is running. By default this directory is in the home folder. Since the Matlab runs will be single node jobs, the cache location should be in the storage on the compute nodes which is stored in TMPDIR.

For example, a multi-threaded serial script should look something like:

#!/bin/bash\u00a0-l\n#\u00a0Batch\u00a0script\u00a0to\u00a0run\u00a0a\u00a0serial\u00a0job\u00a0on\u00a0Legion\u00a0under\u00a0SGE.\n\n#\u00a0Force\u00a0bash\u00a0as\u00a0the\u00a0executing\u00a0shell.\n#$\u00a0-S\u00a0/bin/bash\n\n#\u00a0Request\u00a0ten\u00a0minutes\u00a0of\u00a0wallclock\u00a0time\u00a0(format\u00a0hours:minutes:seconds).\n#$\u00a0-l\u00a0h_rt=0:10:0\n\n#\u00a0Request\u00a01\u00a0gigabyte\u00a0of\u00a0RAM\u00a0\n#$\u00a0-l\u00a0mem=1G\n\n#\u00a0Request 36 cores\n#$\u00a0-pe smp 36\n\n#\u00a0Set\u00a0the\u00a0name\u00a0of\u00a0the\u00a0job.\n#$\u00a0-N\u00a0Matlab_Job_1\n\n#\u00a0Set\u00a0the\u00a0working\u00a0directory\u00a0to\u00a0somewhere\u00a0in\u00a0your\u00a0scratch\u00a0space.\n#\u00a0For\u00a0example:\n##$\u00a0-wd\u00a0/home//Scratch\n#\u00a0Alternatively,\u00a0you can automatically use the current working directory \n#  if you launch\u00a0your\u00a0job\u00a0from\u00a0anywhere\u00a0*within\u00a0~/Scratch*\n#$\u00a0-cwd\n\n#\u00a0store\u00a0the\u00a0MATLAB\u00a0runtime\u00a0path\u00a0in\u00a0a\u00a0global\u00a0environment\u00a0variable\u00a0(MCR_HOME)\nexport\u00a0MCR_HOME=/shared/ucl/apps/Matlab/R2011a/Runtime7.15/v715/\n\n#\u00a0the\u00a0path\u00a0to\u00a0the\u00a0Matlab\u00a0cache\u00a0is\u00a0stored\u00a0in\u00a0the\u00a0global\u00a0variable\u00a0MCR_CACHE_ROOT\u00a0\nexport\u00a0MCR_CACHE_ROOT=$TMPDIR/mcr_cache\n\n#\u00a0make\u00a0sure\u00a0the\u00a0directory\u00a0in\u00a0MCR_CACHE_ROOT\u00a0exists\nmkdir\u00a0-p\u00a0$MCR_CACHE_ROOT\n\n#\u00a0Run\u00a0the\u00a0executable,\u00a0passing\u00a0the\u00a0path\u00a0stored\u00a0in\u00a0MCR_HOME\u00a0as\u00a0the\u00a0first\u00a0argument.\n#\u00a0There\u00a0is\u00a0no\u00a0need\u00a0to\u00a0pass\u00a0the\u00a0content\u00a0of\u00a0MCR_CACHE_ROOT\u00a0as\u00a0an\u00a0argument\u00a0to\u00a0the\n#\u00a0to\u00a0the\u00a0run_appname.sh\u00a0script\u00a0since\u00a0it\u00a0is\u00a0a\u00a0variable\u00a0that\u00a0the\u00a0Matlab\u00a0runtime\u00a0is\u00a0aware\u00a0of.\n./run_appname.sh\u00a0$MCR_HOME\u00a0[arguments\u00a0list]\n\n#\u00a0Preferably,\u00a0tar-up\u00a0(archive)\u00a0all\u00a0output\u00a0files\u00a0onto\u00a0the\u00a0shared\u00a0scratch\u00a0area\ntar\u00a0zcvf\u00a0$HOME/Scratch/files_from_job_${JOB_ID}.tgz\u00a0$TMPDIR\n\n#\u00a0Make\u00a0sure\u00a0you\u00a0have\u00a0given\u00a0enough\u00a0time\u00a0for\u00a0the\u00a0copy\u00a0to\u00a0complete!\n

For any queries and problem reports, please contact rc-support@ucl.ac.uk.

"},{"location":"Supplementary/Connecting_to_RDSS/","title":"Connecting to the Research Data Storage Service","text":"

The Research Data Storage Service (RDSS) is a system run by the Research Data group in the Advanced Research Computing department, and is designed to help with data storage during and after a project. Several solutions for copying data between RDSS and the central UCL research computing platforms are presented below. Sections of the example code surrounded by angle brackets (\\<>) should be replaced by the information indicated (do not keep the angle brackets in).

"},{"location":"Supplementary/Connecting_to_RDSS/#between-myriad-and-rdss","title":"Between Myriad and RDSS","text":"

If you already have an account with the Research Data Storage Service, you can transfer data directly between Legion and Research Data Storage using the Secure Copy (scp) command.

"},{"location":"Supplementary/Connecting_to_RDSS/#from-rds-to-myriad","title":"From RDS to Myriad","text":"

If you are on an RDSS login node, you can transfer data to Myriad\u2019s Scratch area at the highest rate currently possible by running the command: 

scp\u00a0data_file.tgz\u00a0myriad.rc.ucl.ac.uk:~/Scratch/\n

Or from somewhere within Myriad (including compute nodes in running jobs) running the command: 

scp\u00a0ssh.rd.ucl.ac.uk:~/data_file.tgz\u00a0~/Scratch/\n
"},{"location":"Supplementary/Connecting_to_RDSS/#from-myriad-to-rdss","title":"From Myriad to RDSS","text":"

From Myriad, send data to your project space on RDSS by running the command:

scp\u00a0data_file.tgz\u00a0ccaaxyz@ssh.rd.ucl.ac.uk:<path_to_project_space>\n

The RDSS support pages provide more information:

https://www.ucl.ac.uk/isd/live-storage-access-guide

"},{"location":"Supplementary/GPU_Clusters/","title":"GPU clusters","text":"

UCL users may be able to access the following GPU clusters.

"},{"location":"Supplementary/GPU_Clusters/#national-gpu-clusters","title":"National GPU clusters","text":"

There are two nationally-accessible EPSRC Tier 2 HPC centres with GPUs.

Access is generally managed through calls to an EPSRC Resource Allocation Panel

There may also be pump-priming/proof of concept access available.

General information about machines with external access is available at HPC-UK.

"},{"location":"Supplementary/GPU_Clusters/#csd3","title":"CSD3","text":""},{"location":"Supplementary/GPU_Clusters/#jade","title":"JADE","text":""},{"location":"Supplementary/GPU_Nodes/","title":"GPU Nodes","text":""},{"location":"Supplementary/GPU_Nodes/#node-types","title":"Node Types","text":"

You can view the hardware specifications for GPU node types in Myriad.

There are several types of GPU nodes available in Myriad.

"},{"location":"Supplementary/GPU_Nodes/#available-modules","title":"Available modules","text":"

You can see all the available CUDA modules by typing

module load beta-modules\nmodule\u00a0avail\u00a0cuda\n

The ones that become visible once you load beta-modules have been built with newer compilers.

"},{"location":"Supplementary/GPU_Nodes/#sample-cuda-code","title":"Sample CUDA code","text":"

There are samples in some CUDA install locations, e.g. 

/shared/ucl/apps/cuda/7.5.18/gnu-4.9.2/samples\n/shared/ucl/apps/cuda/8.0.61/gnu-4.9.2/samples\n

which are further documented by NVIDIA here. In general, you should look at their CUDA docs: http://docs.nvidia.com/cuda/

"},{"location":"Supplementary/GPU_Nodes/#sample-jobscripts","title":"Sample jobscripts","text":"

You can see sample jobscripts here.

Use this in your script to request up to 2 GPUs. 

#$\u00a0-l\u00a0gpu=2\n

Load GCC and the relevant CUDA module. 

module\u00a0unload\u00a0compilers\u00a0mpi\nmodule\u00a0load\u00a0compilers/gnu/4.9.2\nmodule\u00a0load\u00a0cuda/7.5.18/gnu-4.9.2\n
"},{"location":"Supplementary/GPU_Nodes/#running-the-sample-code","title":"Running the sample code","text":"

To get started, here's how you would compile one of the CUDA samples and run it in an interactive session on a GPU node.

You can compile CUDA code on the login nodes like this (which do not have GPUs) if they do not require all the CUDA libraries to be present at compile time. If they do, you'll get an error saying it cannot link the CUDA libraries, and ERROR: CUDA could not be found on your system and you will need tro do your compiling on the GPU node as well.

1. Load the cuda module 

module\u00a0unload\u00a0compilers\u00a0mpi\nmodule\u00a0load\u00a0compilers/gnu/4.9.2 \nmodule\u00a0load\u00a0cuda/7.5.18/gnu-4.9.2\n

2. Copy the samples directory to somewhere in your home (or to Scratch if you're building on the GPU node or are going to want a job to write anything in the same directory).

cp\u00a0-r\u00a0/shared/ucl/apps/cuda/7.5.18/gnu-4.9.2/NVIDIA_CUDA-7.5_Samples/\u00a0~/cuda\n

3. Choose an example: eigenvalues in this case, and build using the provided makefile - if you have a look at it you can see it is using nvcc and g++. 

cd\u00a0NVIDIA_CUDA-7.5_Samples/6_Advanced/eigenvalues/\nmake\n

4. Request an interactive job with a GPU and wait to be given access to the node. You will see your prompt change to indicate that you are on a different node than the login node once your qrsh request has been scheduled, and you can then continue. Load the cuda module on the node and run the program. 

qrsh\u00a0-l\u00a0mem=1G,h_rt=0:30:0,gpu=1\u00a0-now\u00a0no\n\n# wait for interactive job to start\n\nmodule\u00a0unload\u00a0compilers\u00a0mpi\nmodule\u00a0load\u00a0compilers/gnu/4.9.2\nmodule\u00a0load\u00a0cuda/7.5.18\ncd\u00a0~/cuda/NVIDIA_CUDA-7.5_Samples/6_Advanced/eigenvalues/\n./eigenvalues\n

5. Your output should look something like this: 

Starting\u00a0eigenvalues  \nGPU\u00a0Device\u00a00:\u00a0\"Tesla\u00a0M2070\"\u00a0with\u00a0compute\u00a0capability\u00a02.0\n\nMatrix\u00a0size:\u00a02048\u00a0x\u00a02048\u00a0  \nPrecision:\u00a00.000010  \nIterations\u00a0to\u00a0be\u00a0timed:\u00a0100  \nResult\u00a0filename:\u00a0'eigenvalues.dat'  \nGerschgorin\u00a0interval:\u00a0-2.894310\u00a0/\u00a02.923303  \nAverage\u00a0time\u00a0step\u00a01:\u00a026.739325\u00a0ms  \nAverage\u00a0time\u00a0step\u00a02,\u00a0one\u00a0intervals:\u00a09.031162\u00a0ms  \nAverage\u00a0time\u00a0step\u00a02,\u00a0mult\u00a0intervals:\u00a00.004330\u00a0ms  \nAverage\u00a0time\u00a0TOTAL:\u00a035.806992\u00a0ms  \nTest\u00a0Succeeded!\n
"},{"location":"Supplementary/GPU_Nodes/#building-your-own-code","title":"Building your own code","text":"

As above, if the code you are trying to compile needs to link against libcuda, it must also be built on a GPU node because only the GPU nodes have the correct libraries.

The NVIDIA examples don't require this, but things like Tensorflow do.

"},{"location":"Supplementary/GPU_Nodes/#tensorflow","title":"Tensorflow","text":"

Tensorflow is installed: type module avail tensorflow to see the available versions.

Modules to load for the non-MKL GPU version: 

module\u00a0unload\u00a0compilers\u00a0mpi \nmodule\u00a0load\u00a0compilers/gnu/4.9.2  \nmodule\u00a0load\u00a0python3/3.7\nmodule\u00a0load\u00a0cuda/10.0.130/gnu-4.9.2  \nmodule\u00a0load\u00a0cudnn/7.4.2.24/cuda-10.0\nmodule\u00a0load\u00a0tensorflow/2.0.0/gpu-py37\n

Modules to load the most recent version we have installed with GPU support (2.11.0):

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load python/3.9.6-gnu-10.2.0\nmodule load cuda/11.2.0/gnu-10.2.0\nmodule load cudnn/8.1.0.77/cuda-11.2\nmodule load tensorflow/2.11.0/gpu\n
"},{"location":"Supplementary/GPU_Nodes/#pytorch","title":"PyTorch","text":"

PyTorch is installed: type module avail pytorch to see the versions available.

Modules to load the most recent release we have installed (May 2022) are:

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load python3/3.9-gnu-10.2.0\nmodule load cuda/11.3.1/gnu-10.2.0\nmodule load cudnn/8.2.1.32/cuda-11.3\nmodule load pytorch/1.11.0/gpu\n

If you want the CPU only version then use:

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load python3/3.9-gnu-10.2.0\nmodule load pytorch/1.11.0/cpu\n
"},{"location":"Supplementary/GPU_Nodes/#using-mpi-and-gpus","title":"Using MPI and GPUs","text":"

It is possible to run MPI programs that use GPUs but only within a single node, so you can request up to 4 GPUs and 36 cores on Myriad.

"},{"location":"Supplementary/GPU_Nodes/#looking-for-more-gpus","title":"Looking for more GPUs?","text":""},{"location":"Supplementary/Glossary/","title":"Research Computing Glossary","text":"Bash A shell and scripting language, which is the default command processor on most Linux operating systems. Cluster A cluster consists of a set of computer nodes connected together over a fast local area network. A message passing protocol such as MPI allows individual nodes to work together as a single system. Core A core refers to a processing unit within a node. A node may have multiple cores which can work in parallel on a single task, operating on the same data in memory. This kind of parallelism is coordinated using the OpenMP library. Alternatively, cores may work independently on different tasks. Cores may or may not also share cache. Interconnect The interconnect is the network which is used to transfer data between nodes in a cluster. Different types of interconnect operate at different bandwidths and with different amounts of latency, which affects the suitability of a collection of nodes for jobs which use message passing (MPI). Job In the context of Batch Processing, a job refers to a computational task to be performed such as a single simulation or analysis. Job Script A job script is essentially a special kind of script used to specify the parameters of a job. Users can specify the data to input, program to use, and the computing resources required. The job script is specified when a job is submitted to SGE, which reads lines starting with #$. MPI The Message Passing Interface (MPI) system is a set of portable libraries which can be incorporated into programs in order to control parallel computation. Specifically it coordinates effort between nodes which do not share the same memory address space cf. OpenMP. Node In cluster computing, a node refers to a computational unit which is capable of operating independently of other parts of the cluster. As a minimum it consists of one (or more) processing cores, has its own memory, and runs its own operating system. OpenMP Open Multi-Processing. OpenMP supports multithreading, a process whereby a master thread generates a number of slave threads to run a task which is divided among them. OpenMP applies to processes running on shared memory platforms, i.e. jobs running on a single node. Hybrid applications may make use of both OpenMP and MPI. Process A process is a single instance of a program that is running on a computer. A single process may consist of many threads acting concurrently, and there may multiple instances of a program running as separate processes. Script A shell script enables users to list commands to be run consecutively by typing them into a text file instead of typing them out live. The first line of the script uses the shebang notation #! to designate the scripting language interpreter program to be used to interpret the commands, e.g. bash. Shebang \"Shebang\" is a common abbreviation for \"hash-bang\" \u2014 the character sequence #! \u2014 which is placed at the start of a script to specify the interpreter that should be used. When the shebang is found in the first line of a script, the program loader reads the rest of the line as the path to the required interpreter (e.g. /bin/bash is the usual path to the bash shell). The specified interpreter is then run with the path to the script passed as an argument to it. Shell A command line interpreter which provides an interface for users to type instructions to be interpreted by the operating system and display output via the monitor. Users type specific shell commands in order to run processes, e.g. ls to list directory contents. Son of Grid Engine (SGE or SoGE) The queuing system used by many cluster computing systems (including, currently, all the ones we run) to organise and schedule jobs. Once jobs are submitted to SGE, it takes care of executing them when the required resources become available. Job priority is subject to the local fair use policy. Sun Grid Engine (SGE) The original software written by Sun Microsystems that was later modified to make Son of Grid Engine (among other products, like Univa Grid Engine). Documentation may refer to Sun Grid Engine instead of Son of Grid Engine, and for most user purposes, the terms are interchangeable. Thread A thread refers to a serial computational process which can run on a single core. The number of threads generated by a parallel job may exceed the number of cores available though, in which case cores may alternate between running different threads. Threads are a software concept whereas cores are physical hardware."},{"location":"Supplementary/Hostkeys/","title":"Hostkeys","text":"

These are the current hostkey fingerprints for our clusters. The MD5 or SHA256 at the front is letting you know what type of fingerprint it is - your ssh client may not include that part in its output.

"},{"location":"Supplementary/Hostkeys/#myriad","title":"Myriad","text":"
ED25519 key fingerprint is MD5:92:6d:97:46:eb:8d:0a:4b:8a:65:cb:0d:65:79:bb:7f\nECDSA key fingerprint is SHA256:7FTryal3mIhWr9CqM3EPPeXsfezNk8Mm8HPCCAGXiIA\nRSA key fingerprint is 29:a7:45:04:83:86:ec:95:fa:25:dc:7a:f4:93:78:c1\n
"},{"location":"Supplementary/Hostkeys/#kathleen","title":"Kathleen","text":"
ED25519 key fingerprint is MD5:92:6d:97:46:eb:8d:0a:4b:8a:65:cb:0d:65:79:bb:7f\nECDSA key fingerprint is SHA256:rCKAb0yOWXK8+GClKy/pdbwrUbrGMvFkMciZLVcbaTA\nRSA key fingerprint is 5a:cf:95:a2:e4:05:8a:36:46:dc:65:0a:f2:8b:ab:e1\n
"},{"location":"Supplementary/Hostkeys/#young","title":"Young","text":"
ED25519 key fingerprint MD5 is 92:6d:97:46:eb:8d:0a:4b:8a:65:cb:0d:65:79:bb:7f\nECDSA key fingerprint is SHA256:3zwMU9C8d9rgmYJ9qDElo15NnWyF2I4xy2X/VIAmFdo\nRSA key fingerprint is 06:17:f3:f2:0c:3e:0d:df:1d:04:fb:53:dc:77:60:56\n
"},{"location":"Supplementary/Hostkeys/#thomas","title":"Thomas","text":"
ED25519 key fingerprint is MD5:92:6d:97:46:eb:8d:0a:4b:8a:65:cb:0d:65:79:bb:7f\nECDSA key fingerprint is SHA256:r48udIRDfBEIJG+jiIJFs/56ZayaKUdusFd+JQ3jsO4\nRSA key fingerprint is SHA256:AZ88UVU3BfZkSBOsMw5VKgbDi47o3dpEabPlIB9GtcM\n
"},{"location":"Supplementary/Hostkeys/#michael","title":"Michael","text":"
ED25519 key fingerprint is MD5:b3:d9:60:a8:73:62:d5:91:ef:2b:ba:1e:d3:68:7c:ec\nECDSA key fingerprint is SHA256:3PMLXp6ny0dECycvx4D7+t0sNgsSsLvSO5QUYmzkbhs\nRSA key fingerprint is 85:31:4b:cf:1a:ec:64:e4:b2:98:28:4a:46:b2:c1:90\n
"},{"location":"Supplementary/Installing_PGI/","title":"Installing PGI","text":""},{"location":"Supplementary/Installing_PGI/#pgi-compiler-suite-installation-at-ucl","title":"PGI Compiler Suite Installation at UCL","text":"

UCL has two floating licences for PGI Fortran/C/C++ Server for Linux, purchased primarily for building Gaussian 03 and Gaussian 09 on UCL computers. To install follow the procedure below. If you are installing on a system outside the Institutional Firewall, please connect to the UCL VPN service first.

tar\u00a0xvzf\u00a0pgilinux-2013-139.tar.gz\n
./install`\n
export\u00a0LM_LICENSE_FILE=27000@lic-pgi.ucl.ac.uk\n\nexport\u00a0LM_LICENSE_FILE=$LM_LICENSE_FILE:27000@lic-pgi.ucl.ac.uk\n

The PGI compilers should now be installed and working.

"},{"location":"Supplementary/NVIDIA_Containers/","title":"Using NVIDIA Grid Cloud Containers","text":"

NVIDIA's NGC Container Registry stores a lot of containers1 with various applications pre-configured to be run with GPUs. Each container can be set up completely differently, so you'll need to read the instructions in the registry to work out how to use it.

Many of the instructions for using these containers assume you are using the Docker toolset to run the container, but this is not installed on our services2. The Singularity container runtime can be used instead for most workloads, but there are some limitations, that mostly affect containers that try to run web services.

In general, if a container's instructions do not have specific instructions for using Singularity, and would tell you to run, for example:

docker run --gpus all nvcr.io/some/container\n

You should instead use:

singularity run --nv https://nvcr.io/some/container\n

For jobs using MPI, this is more complicated, because the MPI traffic has to be passed between the container and the operating system underneath. Containers built to use MPI will usually contain instructions for using Singularity, however.

"},{"location":"Supplementary/NVIDIA_Containers/#worked-example-namd-3","title":"Worked Example: NAMD 3","text":"

The NAMD authors publish a NAMD container on the NGC Container Registry, and we should be able to download this and run it on our cluster inside a batch job.

The page about how to use the container is here: https://catalog.ngc.nvidia.com/orgs/hpc/containers/namd

Following it through, you can download the benchmark example to check later whether your container works:

mkdir ngc_namd_experiment\ncd ngc_namd_experiment\nwget -O - https://gitlab.com/NVHPC/ngc-examples/raw/master/namd/3.0/get_apoa1.sh | bash\n

There are a couple of typos in the instructions: you'll need to use the tag 3.0-alpha3-singlenode instead of 3.0_alpha3-singlenode:

export NAMD_TAG=\"3.0-alpha3-singlenode\"\n
"},{"location":"Supplementary/NVIDIA_Containers/#creating-the-container-image","title":"Creating the Container Image","text":"

Before you use Singularity to create the container image, you should load the Singularity module to set up some directories where things are stored. Not doing this can cause you problems, because the default places often do not have space to store the large files needed.

module load singularity-env\n

Once you've done that, you can download the container's files and build them into the usable container:

singularity build ${NAMD_TAG}.sif docker://nvcr.io/hpc/namd:${NAMD_TAG}\n

This can take a while: Singularity has to download quite a few file collections and assemble them into a single usable set. You may see some of the following warnings:

WARNING: 'nodev' mount option set on /lustre, it could be a source of failure during build process\n2022/02/03 14:06:28  warn xattr{var/log/apt/term.log} ignoring ENOTSUP on setxattr \"user.rootlesscontainers\"\n2022/02/03 14:06:28  warn xattr{/home/uccaiki/Scratch/.singularity/tmp/rootfs-5ac43e37-84fa-11ec-8784-0894ef553d4e/var/log/apt/term.log} destination filesystem does not support xattrs, further warnings will be suppressed\n

These indicate that various capabilities are not available because of how we're building the container. For HPC use, they don't present a problem, but they could be problematic if you were building a web server into a container.

When Singularity has finished, you should see the following message:

INFO:    Creating SIF file...\nINFO:    Build complete: 3.0-alpha3-singlenode.sif\n

This file is the container image, which contains the files needed to run NAMD. You can see what NAMD gets when running \"inside\" the container, by running ls with it:

singularity exec 3.0-alpha3-singlenode.sif ls /\n

gives:

WARNING: Bind mount '/home/uccaiki => /home/uccaiki' overlaps container CWD /home/uccaiki/ngc_namd_experiment, may not be available\nbin  boot  dev  environment  etc  home  host_pwd  lib  lib64  lustre  media  mnt  opt  proc  root  run  sbin  scratch  singularity  srv  sys  tmp  tmpdir  usr  var\n

The warning you get is telling you that your current working directory overlaps with a directory being \"bound\" into the container. Binding brings a directory into the container's view of the filesystem, so that, for example, programs can still access your home directory as usual. In this case it's not a problem, because it's warning you that your home directory is being bound into the container in the same place it would usually be, and that means the same files are visible.

By default, the clusters have Singularity configured to bind your home and Scratch directories into the container, as well as the per-job temporary storage allocated to jobs under $TMPDIR.

The NAMD instructions make an alternative suggestion when setting up this environment variable to use to run Singularity, binding your data directory into a fixed place in the container:

SINGULARITY=\"$(which singularity) exec --nv -B $(pwd):/host_pwd ${NAMD_TAG}.sif\"\n

The option -B $(pwd):/host_pwd handles this, binding wherever you run the command from to the fixed location /host_pwd inside the container.

So, for example, if you run:

ls\n$SINGULARITY ls /host_pwd\n

In both cases, you should see the same files, because you're looking at the same underlying directory.

"},{"location":"Supplementary/NVIDIA_Containers/#running-on-a-single-node","title":"Running on a Single Node","text":"

At this point you're ready to run NAMD inside the container, but you need a job script to submit to the scheduler which can set up the number of cores and GPUs correctly.

#!/bin/bash -l\n\n# Start with our resource requirements:\n#  1 hour's maximum runtime\n#$ -l h_rt=1:00:00\n\n# 2 GPUs\n#$ -l gpu=2\n\n# 36 processor cores\n#$ -pe smp 36\n\n# Start with current working directory the same as where we submitted the job from\n#$ -cwd\n\n# Make sure Singularity looks for our stored container data in the right places\nmodule load singularity-env\n\n# Set the variables we need for this example to run\nNAMD_TAG=\"3.0-alpha3-singlenode\"\nSINGULARITY=\"$(which singularity) exec --nv -B $(pwd):/host_pwd ${NAMD_TAG}.sif\"\n\n# This is where the benchmark's data ends up inside the container\nINPUT=\"/host_pwd/apoa1/apoa1_nve_cuda.namd\"\n\n# Show us some info we can refer to later\nprintf \"Running NAMD using:\\n Cores: %d\\n GPUs: %d\\n Container image: %s\\nWorking directory: %s\\n Input: %s\\n\" \\\n\"$NSLOTS\" \\\n\"$GPU\" \\\n\"${NAMD_TAG}.sif\" \\\n\"$(pwd)\" \\\n\"$INPUT\"\n\n# Run NAMD\n\"$SINGULARITY\" namd3  +ppn \"$NSLOTS\" +setcpuaffinity +idlepoll \"$INPUT\"\n

Copy this into a file, and submit it to the queue, e.g.:

qsub ngc_namd_experiment.sh\n

This should take about 5 minutes to run on a 36-core, 2-GPU node.

  1. A container is a way of bundling up a collection of files and instructions to run as a kind of \"altered view\" of the computer's files and systems.\u00a0\u21a9

  2. This is for a variety of reasons, but primarily that Docker presents a much larger range of security and misuse risks when used by inexperienced or hostile users. Singularity represents a much better fit for common HPC workloads and use-cases.\u00a0\u21a9

"},{"location":"Supplementary/Points_of_Contact/","title":"MMM Points of Contact","text":""},{"location":"Supplementary/Points_of_Contact/#user-management-tools","title":"User management tools","text":"

This page contains tools and information for the nominated Points of Contact.

Other system-specific information is at Thomas, Michael or Young.

These commands can all be run as thomas-command or michael-command or young-command: they run the same thing and the different names are for convenience.

"},{"location":"Supplementary/Points_of_Contact/#displaying-user-information","title":"Displaying user information","text":"

thomas-show, michael-show or young-show is a tool that enables you to find a lot of information about users. Access to the database is given to points of contact individually, contact rc-support@ucl.ac.uk if you try to use this and get an access denied.

At the top level, --user shows all information for one user, in multiple tables. --contacts shows all points of contact - useful for getting the IDs, and --institutes is the same. --allusers will show everyone's basic info. --getmmm will show the most recently used mmm username.

thomas-show\u00a0-h  \nusage:\u00a0thomas-show\u00a0[-h]\u00a0[--user\u00a0username]\u00a0[--contacts]\u00a0[--institutes]  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0[--allusers]\u00a0[--getmmm]  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0{recentusers,getusers,whois}\u00a0...\n\nShow\u00a0data\u00a0from\u00a0the\u00a0Thomas\u00a0database.\u00a0Use\u00a0[positional\u00a0argument\u00a0-h]\u00a0for\u00a0more  \nhelp.\n\npositional\u00a0arguments:  \n\u00a0{recentusers,getusers,whois}  \n\u00a0\u00a0\u00a0recentusers\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0the\u00a0n\u00a0newest\u00a0users\u00a0(5\u00a0by\u00a0default)  \n\u00a0\u00a0\u00a0getusers\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0all\u00a0users\u00a0with\u00a0this\u00a0project,\u00a0institute,\u00a0contact  \n\u00a0\u00a0\u00a0whois\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Search\u00a0for\u00a0users\u00a0matching\u00a0the\u00a0given\u00a0requirements\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit  \n\u00a0--user\u00a0username\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0all\u00a0current\u00a0info\u00a0for\u00a0this\u00a0user  \n\u00a0--contacts\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0all\u00a0allowed\u00a0values\u00a0for\u00a0contact  \n\u00a0--institutes\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0all\u00a0allowed\u00a0values\u00a0for\u00a0institute  \n\u00a0--allusers\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0all\u00a0current\u00a0users  \n\u00a0--getmmm\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0the\u00a0highest\u00a0mmm\u00a0username\u00a0used\n
"},{"location":"Supplementary/Points_of_Contact/#show-recent-users","title":"Show recent users","text":"

thomas-show recentusers shows you the most recently-added N users, default 5.

thomas-show\u00a0recentusers\u00a0-h  \nusage:\u00a0thomas-show\u00a0recentusers\u00a0[-h]\u00a0[-n\u00a0N]\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit  \n\u00a0-n\u00a0N\n
"},{"location":"Supplementary/Points_of_Contact/#show-users-with-a-given-project-institute-contact","title":"Show users with a given project, institute, contact","text":"

thomas-show getusers will search for exact matches to the given project, institute, contact combination.

thomas-show\u00a0getusers\u00a0-h  \nusage:\u00a0thomas-show\u00a0getusers\u00a0[-h]\u00a0[-p\u00a0PROJECT]\u00a0[-i\u00a0INST_ID]\u00a0[-c\u00a0POC_ID]\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit  \n\u00a0-p\u00a0PROJECT,\u00a0--project\u00a0PROJECT  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Project\u00a0name  \n\u00a0-i\u00a0INST_ID,\u00a0--institute\u00a0INST_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Institute\u00a0ID  \n\u00a0-c\u00a0POC_ID,\u00a0--contact\u00a0POC_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Point\u00a0of\u00a0Contact\u00a0ID\n
"},{"location":"Supplementary/Points_of_Contact/#search-for-users-based-on-partial-information","title":"Search for users based on partial information","text":"

thomas-show whois can be used to search for partial matches to username, name, email fragments, including all of those in combination.

thomas-show\u00a0whois\u00a0-h  \nusage:\u00a0thomas-show\u00a0whois\u00a0[-h]\u00a0[-u\u00a0USERNAME]\u00a0[-e\u00a0EMAIL]\u00a0[-n\u00a0GIVEN_NAME]  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0[-s\u00a0SURNAME]\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit  \n\u00a0-u\u00a0USERNAME,\u00a0--user\u00a0USERNAME  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0UCL\u00a0username\u00a0of\u00a0user\u00a0contains  \n\u00a0-e\u00a0EMAIL,\u00a0--email\u00a0EMAIL  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Email\u00a0address\u00a0of\u00a0user\u00a0contains  \n\u00a0-n\u00a0GIVEN_NAME,\u00a0--name\u00a0GIVEN_NAME  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Given\u00a0name\u00a0of\u00a0user\u00a0contains  \n\u00a0-s\u00a0SURNAME,\u00a0--surname\u00a0SURNAME  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Surname\u00a0of\u00a0user\u00a0contains\n
"},{"location":"Supplementary/Points_of_Contact/#adding-user-information-and-new-projects","title":"Adding user information and new projects","text":"

thomas-add will add information to the database. Access to the database is given to points of contact individually, contact rc-support@ucl.ac.uk if you try to use this and get an access denied.

Please note that all options have a --debug flag that will allow you to see the query generated without committing the changes to the database - double-check that the information you are adding is correct.

thomas-add\u00a0-h  \nusage:\u00a0thomas-add\u00a0[-h]\u00a0{user,project,projectuser,poc,institute}\u00a0...\n\nAdd\u00a0data\u00a0to\u00a0the\u00a0Thomas\u00a0database.\u00a0Use\u00a0[positional\u00a0argument\u00a0-h]\u00a0for\u00a0more\u00a0help.\n\npositional\u00a0arguments:  \n\u00a0{user,project,projectuser,poc,institute}  \n   csv                 Add all users from the provided CSV file\n\u00a0\u00a0\u00a0user\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Adding\u00a0a\u00a0new\u00a0user\u00a0with\u00a0their\u00a0initial\u00a0project  \n\u00a0\u00a0\u00a0project\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Adding\u00a0a\u00a0new\u00a0project  \n\u00a0\u00a0\u00a0projectuser\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Adding\u00a0a\u00a0new\u00a0user-project-contact\u00a0relationship  \n\u00a0\u00a0\u00a0poc\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Adding\u00a0a\u00a0new\u00a0Point\u00a0of\u00a0Contact  \n\u00a0\u00a0\u00a0institute\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Adding\u00a0a\u00a0new\u00a0institute/consortium\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit\n
"},{"location":"Supplementary/Points_of_Contact/#add-a-new-user","title":"Add a new user","text":"

thomas-add user allows you to add a new user, with their initial project and point of contact. As of 27 June 2022 this now goes ahead and creates their account automatically within 10 minutes - first prompting you that the information you have entered is correct. You do not need to email us separately about creating accounts unless something has gone wrong. The user's initial project must already exist (create with thomas-add project first).

The user will be allocated the next free mmmxxxx username - you should only specify username yourself if they are an existing UCL user, or on Young if they previously had a Thomas or Michael account you should give them the same username. If they already have an account on this cluster with a different institution, just add them as a projectuser instead using their existing username.

You can get your poc_id by looking at thomas-show --contacts.

thomas-add\u00a0user\u00a0-h  \nusage:\u00a0thomas-add\u00a0user\u00a0[-h]\u00a0-u\u00a0USERNAME\u00a0-n\u00a0GIVEN_NAME\u00a0[-s\u00a0SURNAME]\u00a0-e  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0EMAIL_ADDRESS\u00a0-k\u00a0\"SSH_KEY\"\u00a0-p\u00a0PROJECT_ID\u00a0-c\u00a0POC_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0[--debug]\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit  \n\u00a0-u\u00a0USERNAME,\u00a0--user\u00a0USERNAME  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0UCL\u00a0username\u00a0of\u00a0user  \n\u00a0-n\u00a0GIVEN_NAME,\u00a0--name\u00a0GIVEN_NAME  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Given\u00a0name\u00a0of\u00a0user  \n\u00a0-s\u00a0SURNAME,\u00a0--surname\u00a0SURNAME  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Surname\u00a0of\u00a0user\u00a0(optional)  \n\u00a0-e\u00a0EMAIL_ADDRESS,\u00a0--email\u00a0EMAIL_ADDRESS  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Institutional\u00a0email\u00a0address\u00a0of\u00a0user  \n\u00a0-k\u00a0\"SSH_KEY\",\u00a0--key\u00a0\"SSH_KEY\"  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0User's\u00a0public\u00a0ssh\u00a0key\u00a0(quotes\u00a0necessary)  \n\u00a0-p\u00a0PROJECT_ID,\u00a0--project\u00a0PROJECT_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Initial\u00a0project\u00a0the\u00a0user\u00a0belongs\u00a0to  \n\u00a0-c\u00a0POC_ID,\u00a0--contact\u00a0POC_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Short\u00a0ID\u00a0of\u00a0the\u00a0user's\u00a0Point\u00a0of\u00a0Contact  \n --noconfirm           Don't ask for confirmation on user account creation\n\u00a0--verbose\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0SQL\u00a0queries\u00a0that\u00a0are\u00a0being\u00a0submitted  \n\u00a0--debug\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0SQL\u00a0query\u00a0submitted\u00a0without\u00a0committing\u00a0the\u00a0change\n
"},{"location":"Supplementary/Points_of_Contact/#ssh-key-formats","title":"SSH key formats","text":"

It will verify the provided ssh key by default. Note that it has to be in the form ssh-xxx keystartshere. If someone has sent in a key which has line breaks and header items, make it into this format by adding the key type to the start and removing the line breaks from the key body.

This key:

----\u00a0BEGIN\u00a0SSH2\u00a0PUBLIC\u00a0KEY\u00a0----  \nComment:\u00a0\"comment\u00a0goes\u00a0here\"  \nAAAAB3NzaC1yc2EAAAABJQAAAQEAlLhFLr/4LGC3cM1xgRZVxfQ7JgoSvnVXly0K  \n7MNufZbUSUkKtVnBXAOIjtOYe7EPndyT/SAq1s9RGZ63qsaVc/05diLrgL0E0gW+  \n9VptTmiUh7OSsXkoKQn1RiACfH7sbKi6H373bmB5/TyXNZ5C5KVmdXxO+laT8IdW  \n7JdD/gwrBra9M9vAMfcxNYVCBcPQRhJ7vOeDZ+e30qapH4R/mfEyKorYxrvQerJW  \nOeLKjOH4rSnAAOLcEqPmJhkLL8k6nQAAK3P/E1PeOaB2xD7NNPqfIsjhAJLZ+2wV  \n3eUZATx9vnmVF0YafOjvzcoK2GqUrhNAvi7k0f+ihh8twkfthj==  \n----\u00a0END\u00a0SSH2\u00a0PUBLIC\u00a0KEY\u00a0----\n

should be converted into

ssh-rsa\u00a0AAAAB3NzaC1yc2EAAAABJQAAAQEAlLhFLr/4LGC3cM1xgRZVxfQ7JgoSvnVXly0K7MNufZbUSUkKtVnBXAOIjtOYe7EPndyT/SAq1s9RGZ63qsaVc/05diLrgL0E0gW+9VptTmiUh7OSsXkoKQn1RiACfH7sbKi6H373bmB5/TyXNZ5C5KVmdXxO+laT8IdW7JdD/gwrBra9M9vAMfcxNYVCBcPQRhJ7vOeDZ+e30qapH4R/mfEyKorYxrvQerJWOeLKjOH4rSnAAOLcEqPmJhkLL8k6nQAAK3P/E1PeOaB2xD7NNPqfIsjhAJLZ+2wV3eUZATx9vnmVF0YafOjvzcoK2GqUrhNAvi7k0f+ihh8twkfthj==\n

Other types of keys (ed25519 etc) will say what they are in the first line, and you should change the ssh-rsa appropriately. The guide linked at Creating an ssh key in Windows also shows where users can get the second format out of PuTTY.

"},{"location":"Supplementary/Points_of_Contact/#add-new-users-in-bulk-from-a-csv-file","title":"Add new users in bulk from a CSV file","text":"

young-add csv allows you to add users in bulk using a CSV file of specific format and headers. As of 27 June 2022 the accounts will be all created and activated automatically within 10 minutes.

The CSV is comma-separated with a header line of 

email,given_name,surname,username,project_ID,ssh_key\n

You can leave username empty for it to allocate them a new username, but if they have an existing mmm username you should fill it in. It may be useful to show users with a given institute on Thomas if you are migrating users from one service to another.

You can download a CSV template here. Replace the example data.

young-add csv will try to automatically get your Point of Contact ID based on your username. If it can't, or if you have more than one, it will give you a list to choose from. (All users in one CSV upload will be added using the same Point of Contact ID).

It will prompt you for confirmation on each user account creation unless you give the --noconfirm option.

The project you are adding the user to must already exist.

The SSH key must be formatted as shown in SSH key formats.

If you check your CSV file on the cluster with cat -v and it shows that it is beginning with M-oM-;M-? and ending with ^M you probably need to run dos2unix on it first.

"},{"location":"Supplementary/Points_of_Contact/#add-a-new-project","title":"Add a new project","text":"

thomas-add project will create a new project, associated with an institution. It will not show in Gold until it also has a user in it.

A project ID should begin with your institute ID, followed by an underscore and a project name.

thomas-add\u00a0project\u00a0-h  \nusage:\u00a0thomas-add\u00a0project\u00a0[-h]\u00a0-p\u00a0PROJECT_ID\u00a0-i\u00a0INST_ID\u00a0[--debug]\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit  \n\u00a0-p\u00a0PROJECT_ID,\u00a0--project\u00a0PROJECT_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0A\u00a0new\u00a0unique\u00a0project\u00a0ID  \n\u00a0-i\u00a0INST_ID,\u00a0--institute\u00a0INST_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Institute\u00a0ID\u00a0this\u00a0project\u00a0belongs\u00a0to  \n\u00a0--debug\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0SQL\u00a0query\u00a0submitted\u00a0without\u00a0committing\u00a0the\u00a0change\n
"},{"location":"Supplementary/Points_of_Contact/#add-a-new-projectuser-pairing","title":"Add a new project/user pairing","text":"

thomas-add projectuser will add an existing user to an existing project. Creating a new user for an existing project also creates this relationship. After a new project-user relationship is added, a cron job will pick that up within 15 minutes and create that project for that user in Gold, with no allocation.

thomas-add\u00a0projectuser\u00a0-h  \nusage:\u00a0thomas-add\u00a0projectuser\u00a0[-h]\u00a0-u\u00a0USERNAME\u00a0-p\u00a0PROJECT_ID\u00a0-c\u00a0POC_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0[--debug]\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit  \n\u00a0-u\u00a0USERNAME,\u00a0--user\u00a0USERNAME  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0An\u00a0existing\u00a0UCL\u00a0username  \n\u00a0-p\u00a0PROJECT_ID,\u00a0--project\u00a0PROJECT_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0An\u00a0existing\u00a0project\u00a0ID  \n\u00a0-c\u00a0POC_ID,\u00a0--contact\u00a0POC_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0An\u00a0existing\u00a0Point\u00a0of\u00a0Contact\u00a0ID  \n\u00a0--debug\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0SQL\u00a0query\u00a0submitted\u00a0without\u00a0committing\u00a0the\u00a0change\n
"},{"location":"Supplementary/Points_of_Contact/#deactivating-information","title":"Deactivating information","text":"

This tool is only partly functional at present. It allows you to deactivate (not delete) some entities that may no longer exist or may have been created in error.

"},{"location":"Supplementary/Points_of_Contact/#deactivate-a-projectuser","title":"Deactivate a projectuser","text":"

Use this when the user should no longer be a member of the given project. It does not deactivate the user account, just their membership in this project. You can confirm the change by looking at young-show --user - it will say 'deactivated' rather than 'active' next to their listing for this project.

young-deactivate projectuser -h\nusage: thomas_deactivate.py projectuser [-h] -u USERNAME -p PROJECT [--debug]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -u USERNAME, --user USERNAME\n                        An existing username\n  -p PROJECT, --project PROJECT\n                        An existing project ID\n  --debug               Show SQL query submitted without committing the change\n
"},{"location":"Supplementary/Points_of_Contact/#gold-resource-allocation","title":"Gold resource allocation","text":"

We are currently using Gold to manage allocations. Thomas and Michael share one Gold database, so all the projects exist on both, but they are only active on the correct cluster. Young has its own database.

"},{"location":"Supplementary/Points_of_Contact/#reporting-from-gold","title":"Reporting from Gold","text":"

There are wrapper scripts for a number of Gold commands (these exist in the userscripts module, loaded by default).

These are all set to report in cpu-hours with the -h flag, as that is our main unit. If you wish to change anything about the wrappers, they live in /shared/ucl/apps/cluster-scripts/ so you can take a copy and add your preferred options.

They all have a --man option to see the man pages for that command.

Here are some basic useful options and what they do. They can all be given more options for more specific searches.

gusage\u00a0-p\u00a0project_name\u00a0[-s\u00a0start_time]\u00a0\u00a0#\u00a0Show\u00a0the\u00a0Gold\u00a0usage\u00a0per\u00a0user\u00a0in\u00a0this\u00a0project,\u00a0in\u00a0the\u00a0given\u00a0timeframe\u00a0if\u00a0specified.  \ngbalance\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0#\u00a0Show\u00a0the\u00a0balance\u00a0for\u00a0every\u00a0project,\u00a0split\u00a0into\u00a0total,\u00a0reserved\u00a0and\u00a0available.  \nglsuser\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0#\u00a0Shows\u00a0all\u00a0the\u00a0users\u00a0in\u00a0Gold.  \nglsproject\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0#\u00a0Shows\u00a0all\u00a0the\u00a0projects\u00a0and\u00a0which\u00a0users\u00a0are\u00a0in\u00a0them.  \nglsres\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0#\u00a0Show\u00a0all\u00a0the\u00a0current\u00a0reservatioms,\u00a0inc\u00a0user\u00a0and\u00a0project.\u00a0The\u00a0Name\u00a0column\u00a0is\u00a0the\u00a0SGE\u00a0job\u00a0ID.  \ngstatement\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0#\u00a0Produce\u00a0a\u00a0reporting\u00a0statement\u00a0showing\u00a0beginning\u00a0and\u00a0end\u00a0balances,\u00a0credits\u00a0and\u00a0debits.\n\n#\u00a0Less\u00a0useful\u00a0commands  \nglstxn\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0#\u00a0Show\u00a0all\u00a0Gold\u00a0transactions.\u00a0Filter\u00a0or\u00a0it\u00a0will\u00a0take\u00a0forever\u00a0to\u00a0run.  \nglsalloc\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0#\u00a0Show\u00a0all\u00a0the\u00a0allocations.\n

These can be run by any user. The date format is YYYY-MM-DD.

Eg. gstatement -p PROJECT -s 2017-08-01 will show all credits and debits for the given project since the given date, saying which user and job ID each charge was associated with.

"},{"location":"Supplementary/Points_of_Contact/#transferring-gold","title":"Transferring Gold","text":"

As the point of contact, you can transfer Gold from your allocation account into other project accounts. As before, we've put -h in the wrapper so it is always working in cpu-hours.

gtransfer\u00a0--fromProject\u00a0xxx_allocation\u00a0--toProject\u00a0xxx_subproject\u00a0cpu_hours\n

You can also transfer in the opposite direction, from the subproject back into your allocation account.

Note that you are able to transfer your allocation into another institute's projects, but you cannot transfer it back again - only the other institute's point of contact (or rc-support) can give it back, so be careful which project you specify.

"},{"location":"Supplementary/Points_of_Contact/#when-two-allocations-are-active","title":"When two allocations are active","text":"

There is now an overlap period of a week when two allocations can be active. By default, gtransfer will transfer from active allocations in the order of earliest expiring first. To transfer from the new allocation only, you need to specify the allocation id.

gtransfer\u00a0-i\u00a0allocation_ID\u00a0--fromProject\u00a0xxx_allocation\u00a0--toProject\u00a0xxx_subproject\u00a0cpu_hours\n

glsalloc -p xxx_allocation shows you all allocations that ever existed for your institute, and the first column is the id.

Id\u00a0\u00a0Account\u00a0Projects\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0StartTime\u00a0\u00a0EndTime\u00a0\u00a0\u00a0\u00a0Amount\u00a0\u00a0\u00a0\u00a0\u00a0Deposited\u00a0\u00a0Description\u00a0\u00a0\u00a0\u00a0  \n---\u00a0-------\u00a0---------------------\u00a0----------\u00a0----------\u00a0----------\u00a0----------\u00a0--------------\u00a0  \n87\u00a0\u00a038\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0UKCP_allocation\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a02017-08-07\u00a02017-11-05\u00a0212800.00\u00a03712800.00  \n97\u00a0\u00a038\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0UKCP_allocation\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a02017-10-30\u00a02018-02-04\u00a03712800.00\u00a03712800.00\n
"},{"location":"Supplementary/Troubleshooting/","title":"Troubleshooting","text":"

This page lists some common problems encountered by users, with methods to investigate or solve them.

"},{"location":"Supplementary/Troubleshooting/#why-is-my-job-in-eqw-status","title":"Why is my job in Eqw status?","text":"

If your job goes straight into Eqw state, there was an error in your jobscript that meant your job couldn't be started. The standard qstat job information command will give you a truncated version of the error:

qstat\u00a0-j\u00a0<job_ID>\n

To see the full error instead:

qexplain\u00a0<job_ID>\n

The qexplain script is part of our userscripts set -- if you try to use it and get an error that it doesn't exist, load the userscripts module:

module load userscripts\n

The most common reason jobs go into this error state is that a file or directory your job is trying to use doesn't exist. Creating it after the job is in the Eqw state won't make the job run: it'll still have to be deleted and re-submitted.

"},{"location":"Supplementary/Troubleshooting/#binbash-invalid-option-error","title":"\"/bin/bash: invalid option\" error","text":"

This is a sign that your jobscript is a DOS-formatted text file and not a Unix one - the line break characters are different. Type dos2unix <yourscriptname> in your terminal to convert it.

Sometimes the offending characters will be visible in the error. You can see here it's trying to parse ^M as an option.

"},{"location":"Supplementary/Troubleshooting/#i-think-i-deleted-my-scratch-space-how-do-i-restore-it","title":"I think I deleted my Scratch space, how do I restore it?","text":"

You may have accidentally deleted or replaced the link to your Scratch space. Do an ls -al in your home - if set up correctly, it should look like this:

lrwxrwxrwx\u00a0\u00a0\u00a01\u00a0username\u00a0group\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a024\u00a0Apr\u00a014\u00a0\u00a02022\u00a0Scratch\u00a0->\u00a0/scratch/scratch/username\n

where username is your UCL user ID and group is your primary group.

If this link is not present, you can recreate it with

ln\u00a0-s\u00a0/scratch/scratch/$(whoami)\u00a0Scratch\n

If you have actually deleted the files stored in your Scratch space, there is unfortunately no way to restore them.

"},{"location":"Supplementary/Troubleshooting/#which-mkl-library-files-should-i-use-to-build-my-application","title":"Which MKL library files should I use to build my application?","text":"

Depending on which whether you wish to use BLAS/LAPACK/ScaLAPACK/etc... there is a specific set of libraries that you need to pass to your compilation command line. Fortunately, Intel have released a tool that allows you to determine which libraries to link and in which order for a number of compilers and operating systems:

http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/

"},{"location":"Supplementary/Troubleshooting/#unable-to-run-job-jsv-stderr-perl-warning-setting-locale-failed","title":"Unable to run job: JSV stderr: perl: warning: Setting locale failed.","text":"

This error is generally because your SSH client is passing LANG through as part of the SSH command, and is passing something that conflicts with what Myriad has it set to. You may be more likely to come across this with newer versions of macOS - if your client is different, have a look for an equivalent option.

In Mac OS X Terminal, click Settings and under International untick the box that says \"Set locale environment variables on startup\".

Per session, you can try LANG=C ssh userid@myriad.rc.ucl.ac.uk

"},{"location":"Supplementary/Troubleshooting/#what-can-i-do-to-minimise-the-time-i-need-to-wait-for-my-jobs-to-run","title":"What can I do to minimise the time I need to wait for my job(s) to run?","text":"
  1. Minimise the amount of wall clock time you request.
  2. Use job arrays instead of submitting large numbers of jobs (see our job script examples).
  3. Plan your work so that you can do other things while your jobs are being scheduled.
"},{"location":"Supplementary/Troubleshooting/#what-is-my-project-code-short-string-project-id","title":"What is my project code (short string) / project ID?","text":"

Prior to July 2014, every user had a project code. Now all users belong to the default project \"AllUsers\" and no longer have to specify this. If you see older job script examples mentioning a project ID, you can delete that section. Only projects with access to paid or specialised resources need to give a project code in order to use those resources. If you do not know yours, contact rc-support.

"},{"location":"Supplementary/X-Forwarding/","title":"X-Forwarding","text":"

X is a system and protocol that lets remote computers push interactive windows to your local computer over a network. We use a method known as X-Forwarding, together with an SSH client, to direct the network messages for X over the same connection you use for the command-line.

The setup steps for getting this working on Windows, Linux, and macOS are each different, and we've put them below.

"},{"location":"Supplementary/X-Forwarding/#windows","title":"Windows","text":"

Windows doesn't natively have the ability to receive X windows, so you need to install an X server separately.

There are a few choices; UCL has a site-wide license (covering UCL-owned and personal computers) for one called Exceed, which is pretty reliable and seems to handle 3D content well, but there's also Xming, which is free and open-source if that's a concern.

Exceed is installed on all UCL's centrally-managed Windows computers.

"},{"location":"Supplementary/X-Forwarding/#installing-exceed","title":"Installing Exceed","text":"

Exceed comes in two parts: the main package, and the add-on that handles certain types of rendered 3D content.

You can download both parts from the Exceed page in the UCL Software Database. First, install the main package, then the 3D package.

"},{"location":"Supplementary/X-Forwarding/#using-putty-with-exceed","title":"Using PuTTY with Exceed","text":"

When you run the Exceed program, it will automatically wait in the background until you run a remote application, so you only have to configure PuTTY to pass the data through.

To do this, first fill in the details for your PuTTY connection as normal with the hostname and any other details you'd normally use to connect, and then, in the pane labelled \"Category\" on the left, click the \"+\" next to \"SSH\" and then select \"X11\".

The main pane should change to one labelled \"Options controlling SSH X11 forwarding\". Check the box labelled \"Enable X11 Forwarding\".

You can now click \"Open\" to start the connection, or you can return to the \"Session\" options to save this setup.

"},{"location":"Supplementary/X-Forwarding/#macos","title":"macOS","text":"

Like Windows, macOS doesn't come with an X server to receive X windows. The most commonly used X server for macOS is XQuartz. If you download and install that, you can follow the Linux instructions below. When you connect with X-Forwarding enabled, the XQuartz server program should start automatically, ready to present remote windows.

"},{"location":"Supplementary/X-Forwarding/#linux","title":"Linux","text":"

Almost all Linux versions that have a graphical desktop use an X server to provide it, so you don't have to install a separate one.

You still have to set up your SSH client's connection to \"tunnel\" the X windows from the remote computer, though. You can do this by simply adding the -X option to your ssh command line, so for example to connect to Myriad with X-Forwarding:

ssh -X ccaaxyz@myriad.rc.ucl.ac.uk\n

To use X-Forwarding from outside UCL, you must either use the VPN, or use the SSH Gateway system, with the appropriate flags for both ssh steps, for example:

[me@my_computer ~]$ ssh\u00a0-X\u00a0ccaaxyz@ssh-gateway.ucl.ac.uk\n[...]\n[ccaaxyz@ejp-gateway-01 ~]$ ssh -X ccaaxyz@myriad.rc.ucl.ac.uk\n

Note

This assumes you use a Linux distribution that uses Xorg as its display server. If your distribution uses Wayland instead, and you aren't sure how to make this work, please contact us, letting us know what version of which distribution you're using.

"},{"location":"Supplementary/X-Forwarding/#checking-your-setup","title":"Checking your Setup","text":"

There are some simple programs on the system that use X, which can be used to check whether your setup is working correctly.

If these work, you have a working X connection. If not, you should see an error when you try to run them that may look something like:

xterm: Xt error: Can't open display:\n

Or:

Error: couldn't open display (null)\n

If you see these, please check you have followed all the appropriate steps above, and if you still have problems, contact rc-support@ucl.ac.uk for assistance.

"},{"location":"Supplementary/Young_GPU_Nodes/","title":"Young GPU Nodes","text":""},{"location":"Supplementary/Young_GPU_Nodes/#pilot-access","title":"Pilot access","text":"

A group of nominated pilot users had access to these nodes during the pilot, which lasted from 15 July to 5 September 2022. They are now available to all.

"},{"location":"Supplementary/Young_GPU_Nodes/#gpu-specs","title":"GPU specs","text":"

The nodes are listed in Young's node types table

There are 6 nodes which each have 64 AMD EPYC CPU cores and 8 Nvidia A100-SXM4-40GB GPUs. They have 1T RAM and 200G local disk is available to request as tmpfs unlike the rest of Young. The AMD equivalent of hyperthreading is not enabled.

"},{"location":"Supplementary/Young_GPU_Nodes/#request-gpus","title":"Request GPUs","text":"
# Submit a job to the GPU nodes by adding a request for a number of GPUs per node\n#$ -l gpu=8\n\n# Only Free jobs are available at present. Use your normal projects\n#$ -P Free\n#$ -A Inst_Project\n

At the start of the pilot, jobs did not share nodes and users always had access to all GPUs on each node. This has since been altered and device cgroups are implemented (as of 10 Aug 2022) so jobs can share nodes on the GPU nodes and each only have access to the number of GPUs they requested.

For example, 8 separate single-GPU jobs from different users can be running on one node, or 2 4-GPU jobs. Multi-node parallel GPU jobs do not share nodes, so a job asking for two nodes and some number of GPUs per node over those two nodes will block out the entire 16 GPUs even if using fewer than that.

"},{"location":"Supplementary/Young_GPU_Nodes/#exclusive-use-of-nodes","title":"Exclusive use of nodes","text":"

If you are using fewer than 8 GPUs but want to make sure nothing else is running on the same node as you, add this to your jobscript:

#$ -ac exclusive\n

This would generally only be done if you are benchmarking or investigating speeds and want to rule out anything else running on the rest of the node as possibly affecting your timings.

"},{"location":"Supplementary/Young_GPU_Nodes/#cuda-versions","title":"CUDA versions","text":"

The newer CUDA installs we have are made visible by first loading beta-modules but can then be used alongside any other compiler.

module load beta-modules\nmodule avail cuda\n\n# pick one of the 11.x CUDA installs\nmodule load cuda/11.3.1/gnu-10.2.0\n# or\nmodule load cuda/11.2.0/gnu-10.2.0\n
"},{"location":"Supplementary/Young_GPU_Nodes/#choosing-a-cuda-version","title":"Choosing a CUDA version","text":"

The drivers we have installed on the GPU nodes are version 460.27.03 which is CUDA 11.2. CUDA 11 has minor version compatibility so in most cases you can use the 11.3.1 runtime, but not all functionality is available.

If your code builds but when running it you get an error like this:

CUDA RUNTIME API error: Free failed with error cudaErrorUnsupportedPtxVersion \n

then use the cuda/11.2.0/gnu-10.2.0 module to build and run your program instead.

"},{"location":"Supplementary/Young_GPU_Nodes/#building-with-cuda","title":"Building with CUDA","text":"

If the code you are trying to build only needs to link to the CUDA runtime libraries, libcudart.so then you can build it on the login nodes which do not have GPUs.

If it needs the full libcuda.so to be available, you need to build it on a GPU node. You can submit it as a job or request an interactive session with qrsh. Eg:

qrsh -l gpu=8,h_rt=2:0:0,tmpfs=10G,mem=1G -pe smp 4 -P Free -A Inst_Project -now no\n
"},{"location":"Supplementary/Young_GPU_Nodes/#nvidia-documentation","title":"NVIDIA documentation","text":"

NVIDIA has some useful information at these locations:

"},{"location":"Supplementary/Young_GPU_Nodes/#run-on-a-specific-device-or-limit-the-number-visible","title":"Run on a specific device or limit the number visible","text":"

If you want to tell your code to run on a specific device or devices, you can set CUDA_VISIBLE_DEVICES to the ids between 0 and 7. If the code only uses one GPU it will usually default to running on device 0, but if it is running on all GPUs that belong to your job and you don't want it to, you can limit it.

# run on gpu 1\nexport CUDA_VISIBLE_DEVICES=1\n\n# run on gpus 0 and 4\nexport CUDA_VISIBLE_DEVICES=0,4\n
"},{"location":"Supplementary/Young_GPU_Nodes/#cuda-utility-devicequery","title":"CUDA utility deviceQuery","text":"

CUDA has a number of small utilities that come with its examples which can be useful: you can take a copy of the samples directory from the corresponding CUDA version - for example /shared/ucl/apps/cuda/11.3.1/gnu-10.2.0/samples/ and build those utilities with their corresponding CUDA module loaded.

samples/1_Utilities/deviceQuery will give you a small utility that will confirm that setting CUDA_VISIBLE_DEVICES is working - you can run it before and after setting it. The devices will have been renamed as 0 and 1 in its output, but the location IDs will be the same as when you could see all of them.

"},{"location":"Supplementary/Young_GPU_Nodes/#setting-ppn","title":"Setting PPN","text":"

You will also be able to set the number of cpu slots per node that you want. Instead of -pe smp or -pe mpi, you would request:

-pe ppn=<slots per node> <total slots>\n
# this would give you 8 slots per node and 16 slots total (so is using 2 nodes)\n# along with 8 GPUs per node (16 GPUs in total).\n#$ -pe ppn=8 16\n#$ -l gpu=8\n

Like -pe mpi this will also create a suitable machinefile for you so MPI will know how many cores on which nodes it can use. gerun (our mpirun wrapper) will use it automatically for Intel MPI as usual and our OpenMPI modules shouldn't need it since they have scheduler integration, but you can find it in $TMPDIR/machines if you are using mpirun and need it.

"},{"location":"Supplementary/Young_GPU_Nodes/#request-tmpfs","title":"Request tmpfs","text":"

The GPU nodes do have local disks and you can request an amount of tmpfs up to the maximum 200G like this:

# Request a $TMPDIR of 20G\n#$ -l tmpfs=20G\n

In the job, you refer to this using $TMPDIR. Many programs will use this environment variable for temporary files automatically, or you may need to tell them to do it explicitly with a command line argument.

$TMPDIR is deleted at the end of the job, so if you need any data that is being written to there, copy it back to your Scratch at the end of the job.

"},{"location":"Supplementary/Young_GPU_Nodes/#software-of-interest","title":"Software of interest","text":"

GPU software we have installed that may be of particular interest to users of Young.

You can also use NVIDIA Grid Cloud Containers via Singularity which provide pre-configured GPU applications. Our page gives an example of using the NAMD 3 container.

"},{"location":"Walkthroughs/Logging_In/","title":"Logging In","text":"

Here are walkthroughs showing all the steps to get logged in to one of our clusters from computers with different operating systems.

These instructions assume you have already applied for an account and received the email saying it has been created and that you can log in.

We use Myriad as the cluster we are logging in to. The same steps also apply to Kathleen.

To log in, you need an SSH client to be installed on the computer you are logging in from.

"},{"location":"Walkthroughs/Logging_In/#logging-in-from-windows","title":"Logging in from Windows","text":"

There are several choices of SSH client for Windows, with one now included in the Windows Command Prompt on Windows 10 or later. You can also log in via Desktop@UCL Anywhere, which provides a Windows environment inside the UCL network.

"},{"location":"Walkthroughs/Logging_In/#putty","title":"PuTTY","text":"

PuTTY will provide you with a graphical interface to configure your SSH connection and then open a terminal window you can type into and press return to submit.

"},{"location":"Walkthroughs/Logging_In/#windows-command-prompt","title":"Windows Command Prompt","text":"

Launch the Command Prompt from the Windows Start menu. It will give you a prompt that you can type commands into.

Replace \"uccacxx\" with your own central UCL username.

ssh uccacxx@ssh-gateway.ucl.ac.uk\n

If your computer has never connected to the Gateway before, it has no existing record of the host fingerprint which identifies it, so it will ask if you want to accept it and continue.

Type \"yes\" to accept the fingerprint. It will save it and check it next time. If the fingerprint is different, it can be an indication that something else is pretending to be the Gateway (or that it has changed after a major update). If concerned, contact rc-support@ucl.ac.uk or the main Service Desk.

It now informs you that it has added the Gateway to your list of known hosts.

You have now contacted the Gateway and it displays a small splash screen and asks for your UCL password. Nothing will show up when typing in this box - no placeholders or bullet point characters. Press return at the end to submit.

If you have a typo in your password or have changed it within the last couple of hours and the new one hasn't propagated yet, it will ask again.

Once the correct password has been entered, it will show you a longer message about the system.

From the Gateway, we want to ssh in to Myriad:

ssh uccacxx@myriad.rc.ucl.ac.uk\n

Your user on the Gateway will also not have connected to Myriad before, so you will get a similar prompt about Myriad's host fingerprint. You can check this against our current key fingerprints.

After saying \"yes\" you will be prompted for your password again, and after typing it in you will be logged in to Myriad and see Myriad's message containing information about the system and where to get help.

At the bottom you can see that the prompt on Myriad looks like

[uccacxx@login13 ~]$\n

It shows you your username, which Myriad login node you are on, and where you are (~ is a short way to reference your home directory). You can now look at what software is available and write jobscripts.

"},{"location":"Walkthroughs/Logging_In/#desktopucl-anywhere","title":"Desktop@UCL Anywhere","text":"

You can log in to this from a web browser or download a Citrix client. Once logged in, Desktop@UCL is inside the UCL network so we can log straight into a cluster with no need for a gateway machine or VPN.

You can use PuTTY or the Windows Command Prompt on Desktop@UCL Anywhere to log in to Myriad. It has a version of the Command Prompt called \"SSH Command Prompt\" that may set up some additional configuration for SSH usage - either should work.

If you want to use SSH keys for your connection, you need to use PuTTY and PuTTYgen. The OpenSSH Authentication Agent is not running as a service on Desktop@UCL Anywhere or on UCL managed laptops, so you cannot use ssh-add in the Windows Command Prompt.

"},{"location":"Walkthroughs/Machine_Learning/","title":"Machine Learning","text":"

Myriad is a traditional HPC system with a batch scheduler. This means that there are some additional steps to think about when running Machine Learning toolkits. Here are some videos that walk through installing and using various machine learning toolkits on Myriad.

"}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Research Computing Services","text":"

This documentation is maintained by the Research Computing team for the purpose of sharing information about our services, including user guides, service updates and account request and renewal support.

"},{"location":"#documentation","title":"Documentation","text":"

General User Information:

Clusters:

"},{"location":"#email-support","title":"Email Support","text":"

For support for any of our services, contact us at: rc-support@ucl.ac.uk

We will endeavour to answer queries on any aspect of computing related to your research, whatever your skill level or requirements.

"},{"location":"#how-to","title":"How to...","text":""},{"location":"#training","title":"Training","text":"

Due to the on-going COVID-19 pandemic, our ordinary training courses have been suspended indefinitely.

We have an online Moodle course \"Introduction to the Unix Shell\" and infrequently provide a training course aimed at getting users up and running on one of our main clusters. \"Introduction to the Unix Shell (Moodle) (UCL users), GitHub-Pages (non-UCL users), additional resources can be found on UCL mediacentral.

We have an online Moodle course \"Introduction to High Performance Computing at UCL\" aimed at getting users comfortable with using HPC at UCL. \"Introduction to High Performance Computing at UCL\" (Moodle) (UCL users), UCL mediacentral.

Please see our Research IT Training page for details.

"},{"location":"Account_Services/","title":"Account Services","text":""},{"location":"Account_Services/#cluster-access-application","title":"Cluster access application","text":"

There is a single online form for applying to the majority of our clusters. Use of these services is subject to a common set of terms and conditions. All granted applications give you a Myriad account while access to other systems depends on what types of work you tell us you will be doing.

"},{"location":"Account_Services/#account-sponsors","title":"Account sponsors","text":"

If you are a student or postdoctoral researcher, your application must be approved by a permanent member of staff (normally your supervisor or PI). This will be automatically determined when you begin your application, and when you submit it, an email will be sent to this person asking them to approve the application before the account can be created.

Please note that the form requires your sponsor's UCL username and not their UPI.

Permanent members of staff do not need a sponsor and their accounts will be automatically approved.

"},{"location":"Account_Services/#apply-for-an-account","title":"Apply for an account","text":"

The online application form can be found here:

You will need a UCL user name and password. These are the same credentials used to access core services like Portico and HR systems.

"},{"location":"Account_Services/#application-process","title":"Application process","text":"

The application process has these steps:

  1. Enter your UCL username and password to access the application form.

  2. Complete the application form, reading the instructions carefully. Tip: hover your mouse over text boxes for more information.

  3. When you successfully submit the form, your sponsor will be sent an email message asking them to approve your application. If you do not require a sponsor, your application will be automatically approved.

  4. Your sponsor should click on the link in the email, log in and approve the account.

  5. You will then receive an email when your account is approved, if you have a sponsor. (This does not mean your account has been created yet).

  6. You should receive an email once we have created your account. Please note that there may be a delay of up to one working day between an application being approved and your account being created.

If your sponsor does not receive the email to approve your account, send them the link to your application directly (it will look like dashboard.rc.ucl.ac.uk/computing/requests/xxxx) and there will be a button at the top to approve it.

Please note: most delays in the application process are caused by sponsors missing or not receiving the approval email, so check with your sponsor first if your application is stuck waiting for approval. Research Computing can only create your account after it is approved.

"},{"location":"Account_Services/#accounts-for-visitors","title":"Accounts for visitors","text":"

UCL visitors are welcome to apply for accounts on Research Computing Services. Please note that:

  1. Applicants must have a central UCL account. UCL Visitor accounts can be arranged by your Departmental Administrator, so you should speak to them first.
  2. Account applications should specify the UCL grant under which the work is being carried out, if possible, as well as an associated UCL Group or Researcher.
  3. Account applications may not be submitted on behalf of another, except to cover accessibility requirements, as the account application process includes agreeing to relevant legal terms and conditions.
"},{"location":"Account_Services/#accounts-for-honorary-staff","title":"Accounts for Honorary staff","text":"

UCL Staff Members may nominate Honorary members (named individuals) to be provided with access to Research Computing services where this is beneficial to UCL's research interests.

Nomination should be made via the CRAG, explaining the benefit arising to UCL.

Proposals will be reviewed by the CRAG at their monthly meeting and on a case by case basis.

All accounts thus provided are subject to all other 'standard' T&C's relating to their use of Research Computing Services.

"},{"location":"Account_Services/#charges-for-use-of-research-computing-services","title":"Charges for use of Research Computing services","text":"

Research Computing services are free at point of use by default. There are no direct charges for your usage under standard resource allocation policy as defined by the CRAG.

Several methods are available to researchers who wish to gain access to additional resources, or obtain 'priority' use, including chargeable options. Details are available at Additional Resource_Requests and Purchasing in Myriad.

"},{"location":"Additional_Resource_Requests/","title":"Additional Resource Requests","text":"

We recognise that researchers may sometimes require a higher throughput of work than it is possible to achieve with free \u2018fair share\u2019 usage of Myriad and Kathleen. There a couple of ways of obtaining additional Myriad resources beyond this fair share:

"},{"location":"Additional_Resource_Requests/#make-a-special-request-to-the-crag-for-free-access-to-additional-resources","title":"Make a special request to the CRAG for free access to additional resources","text":"

Users who wish to request additional resources or reserve resources beyond those provided can complete the additional resource request form in collaboration with your supervisor or the project's principal investigator. This includes requests for increased storage quotas.

The completed form should be sent to the Research Computing Platforms team at rc-support@ucl.ac.uk, for technical review. If successful, your case will be presented to the CRAG for consideration at the next meeting of the Group. The CRAG meets monthly, usually on the second Tuesday of the month, and users will be informed of the Group\u2019s decision as soon as possible after their next meeting.

Note that an application to the CRAG for additional resources is only likely to be approved if the impact on other users is not deemed to be significant, or of long duration.

"},{"location":"Additional_Resource_Requests/#examples-of-requests","title":"Examples of requests","text":""},{"location":"Additional_Resource_Requests/#request-hosting-of-shared-datasets","title":"Request hosting of shared datasets","text":"

We have provision for hosting shared datasets for users on Myriad. These can be datasets that are freely accessible by all users, or ones limited to groups.

Hosted datasets:

They are likely to be managed by a role account - access to the role account will be by ssh key.

To apply for a hosted dataset, please send this form to rc-support@ucl.ac.uk.

"},{"location":"Additional_Resource_Requests/#purchase-dedicated-compute-nodes-or-priority-cycles-on-myriad","title":"Purchase dedicated compute nodes or priority cycles on Myriad","text":"

Researchers may purchase additional resources to be used as part of the Myriad High Performance Computing cluster if the free service does not meet their needs. See Paid-for resources: Purchasing in Myriad for details.

"},{"location":"Additional_Resource_Requests/#further-information","title":"Further information","text":"

For further advice or information on future hardware options, please contact rc-support@ucl.ac.uk.

"},{"location":"Contact_Us/","title":"Contact and Support","text":"

Users should direct any queries relating to their use of Research Computing services to the Research Computing Support Team at rc-support@ucl.ac.uk (see below). The team will respond to your question as quickly as possible, giving priority to requests that are deemed urgent on the basis of the information provided.

Availability: 9:30am - 4:30pm, Monday - Friday, except on Bank Holidays and College Closures.

We aim to provide to you with a useful response within 24 hours.

Please do not email individuals unless you are explicitly asked to do so; always use the rc-support email address provided as this is the best way for your request to be processed.

"},{"location":"Contact_Us/#drop-in-sessions","title":"Drop-In Sessions","text":"

Research IT Services holds drop-in sessions roughly every two weeks which at least one member of the Research Computing team usually attends. More details and dates for these sessions are available on the the RITS pages.

If you have a particularly complex problem, it may be useful to email the support address, rc-support@ucl.ac.uk, beforehand so that the person attending can prepare.

"},{"location":"Contact_Us/#location","title":"Location","text":"

The Research Computing Team are located at:

38-50 Bidborough Street Floor 3 London WC1H 9BT

We are keen to collaborate and welcome visitors to our offices to talk about all things research computing. However, we do not operate a walk-up service desk: if you are frustrated by slow response to a support ticket, we are sorry but please do send reminders as there is probably a good reason why your request is not being processed.

"},{"location":"Example_Jobscripts/","title":"Example Jobscripts","text":"

On this page we describe some basic example scripts to submit jobs to our clusters.

After creating your script, submit it to the scheduler with:

qsub my_script.sh

"},{"location":"Example_Jobscripts/#service-differences","title":"Service Differences","text":"

These scripts are applicable to all our clusters, but node sizes (core count, memory, and temporary storage sizes) differ between machines, so please check those details on the cluster-specific pages. Some clusters are diskless and have no temporary space that can be requested.

"},{"location":"Example_Jobscripts/#working-directories-and-output","title":"Working Directories and Output","text":"

The parallel filesystems we use to provide the home and scratch filesystems perform best when reading or writing single large files, and worst when operating on many different small files. To avoid causing problems, many of the scripts below are written to create all their files in the temporary $TMPDIR storage, and compress and copy them to the scratch area at the end of the job.

This can be a problem if your job is not finishing and you need to see the output, or if your job is crashing or failing to produce what you expected. Feel free to modify the scripts to read from or write to Scratch directly, however, your performance will generally not be as good as writing to $TMPDIR, and you may impact the general performance of the machine if you do this with many jobs simultaneously. This is particularly the case with single-core jobs, because that core is guaranteed to be writing out data.

Please be aware that some clusters are diskless (eg Kathleen) and have no $TMPDIR available for use - in those you must remove the request for tmpfs in your script. Check the cluster-specific pages.

Note that there is also the option of using the Local2Scratch process (see below), which takes place after the job has finished, in the clean-up step. This gives you the option of always getting the contents of $TMPDIR back, at the cost of possibly getting incomplete files and not having any control over where the files go.

"},{"location":"Example_Jobscripts/#note-about-projects","title":"Note about Projects","text":"

Projects are a system used in the scheduler and the accounting system to track budgets and access controls.

Most users of UCL's internal clusters will not need to specify a project and will default to the AllUsers project. Users of the Thomas and Michael services should refer to the specific pages for those machines, and the information they were given when they registered.

To specify a project ID in a job script, use the -P object as below:

#$\u00a0-P\u00a0<your_project_id>\n
"},{"location":"Example_Jobscripts/#resources","title":"Resources","text":"

The lines starting with #$ -l are where you are requesting resources like wallclock time (how long your job is allowed to run), memory, and possibly tmpfs (local hard disk space on the node, if it is not diskless). More details on how to request resources can be found here.

If you have no notion of how much you should request for any of these, have a look at How do I estimate what resources to request in my jobscript?

Useful resources:

"},{"location":"Example_Jobscripts/#serial-job-script-example","title":"Serial Job Script Example","text":"

The most basic type of job a user can submit is a serial job. These jobs run on a single processor (core) with a single thread.

Shown below is a simple job script that runs /bin/date (which prints the current date) on the compute node, and puts the output into a file.

#!/bin/bash -l\n\n# Batch script to run a serial job under SGE.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set the name of the job.\n#$ -N Serial_Job\n\n# Set the working directory to somewhere in your scratch space.  \n#  This is a necessary step as compute nodes cannot write to $HOME.\n# Replace \"<your_UCL_id>\" with your UCL user ID.\n#$ -wd /home/<your_UCL_id>/Scratch/workspace\n\n# Your work should be done in $TMPDIR \ncd $TMPDIR\n\n# Run the application and put the output into a file called date.txt\n/bin/date > date.txt\n\n# Preferably, tar-up (archive) all output files onto the shared scratch area\ntar -zcvf $HOME/Scratch/workspace/files_from_job_$JOB_ID.tar.gz $TMPDIR\n\n# Make sure you have given enough time for the copy to complete!\n
"},{"location":"Example_Jobscripts/#multi-threaded-job-example","title":"Multi-threaded Job Example","text":"

For programs that can use multiple threads, you can request multiple processor cores using the -pe smp <number> option. One common method for using multiple threads in a program is OpenMP, and the $OMP_NUM_THREADS environment variable is set automatically in a job of this type to tell OpenMP how many threads it should use. Most methods for running multi-threaded applications should correctly detect how many cores have been allocated, though (via a mechanism called cgroups).

Note that this job script works directly in scratch instead of in the temporary $TMPDIR storage.

#!/bin/bash -l\n\n# Batch script to run an OpenMP threaded job under SGE.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM for each core/thread \n# (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set the name of the job.\n#$ -N Multi-threaded_Job\n\n# Request 16 cores.\n#$ -pe smp 16\n\n# Set the working directory to somewhere in your scratch space.\n# Replace \"<your_UCL_id>\" with your UCL user ID\n#$ -wd /home/<your_UCL_id>/Scratch/output\n\n# 8. Run the application.\n$HOME/my_program/example\n
"},{"location":"Example_Jobscripts/#mpi-job-script-example","title":"MPI Job Script Example","text":"

The default MPI implementation on our clusters is the Intel MPI stack. MPI programs don\u2019t use a shared memory model so they can be run across multiple nodes. This script differs considerably from the serial and OpenMP jobs in that MPI programs need to be invoked by a program called gerun. This is our wrapper for mpirun and takes care of passing the number of processors and a file called a machine file.

Important: If you wish to pass a file or stream of data to the standard input (stdin) of an MPI program, there are specific command-line options you need to use to control which MPI tasks are able to receive it. (-s for Intel MPI, --stdin for OpenMPI.) Please consult the help output of the mpirun command for further information. The gerun launcher does not automatically handle this.

If you use OpenMPI, you need to make sure the Intel MPI modules are removed and the OpenMPI modules are loaded, either in your jobscript or in your shell start-up files (e.g. ~/.bashrc).

#!/bin/bash -l\n\n# Batch script to run an MPI parallel job under SGE with Intel MPI.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM per process (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space per node \n# (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set the name of the job.\n#$ -N MadScience_1_16\n\n# Select the MPI parallel environment and 16 processes.\n#$ -pe mpi 16\n\n# Set the working directory to somewhere in your scratch space.\n# Replace \"<your_UCL_id>\" with your UCL user ID :\n#$ -wd /home/<your_UCL_id>/Scratch/output\n\n# Run our MPI job.  GERun is a wrapper that launches MPI jobs on our clusters.\ngerun $HOME/src/science/simulate\n
"},{"location":"Example_Jobscripts/#array-job-script-example","title":"Array Job Script Example","text":"

If you want to submit a large number of similar serial jobs then it may be easier to submit them as an array job. Array jobs are similar to serial jobs except we use the -t option to get Sun Grid Engine to run 10,000 copies of this job numbered 1 to 10,000. Each job in this array will have the same job ID but a different task ID. The task ID is stored in the $SGE_TASK_ID environment variable in each task. All the usual SGE output files have the task ID appended. MPI jobs and parallel shared memory jobs can also be submitted as arrays.

#!/bin/bash -l\n\n# Batch script to run a serial array job under SGE.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set up the job array.  In this instance we have requested 10000 tasks\n# numbered 1 to 10000.\n#$ -t 1-10000\n\n# Set the name of the job.\n#$ -N MyArrayJob\n\n# Set the working directory to somewhere in your scratch space. \n# Replace \"<your_UCL_id>\" with your UCL user ID :)\n#$ -wd /home/<your_UCL_id>/Scratch/output\n\n# Run the application.\n\necho \"$JOB_NAME $SGE_TASK_ID\"\n
"},{"location":"Example_Jobscripts/#array-job-script-example-using-parameter-file","title":"Array Job Script Example Using Parameter File","text":"

Often a user will want to submit a large number of similar jobs but their input parameters don't match easily on to an index from 1 to n. In these cases it's possible to use a parameter file. To use this script a user needs to construct a file with a line for each element in the job array, with parameters separated by spaces.

For example: 

0001 1.5 3 aardvark\n0002 1.1 13 guppy\n0003 1.23 5 elephant\n0004 1.112 23 panda\n0005 ...\n

Assuming that this file is stored in ~/Scratch/input/params.txt (you can call this file anything you want) then the user can use awk/sed to get the appropriate variables out of the file. The script below does this and stores them in $index, $variable1, $variable2 and $variable3. So for example in task 4, $index = 0004, $variable1 = 1.112, $variable2 = 23 and $variable3 = panda.

Since the parameter file can be generated automatically from a user's datasets, this approach allows the simple automation, submission and management of thousands or tens of thousands of tasks.

#!/bin/bash -l\n\n# Batch script to run an array job.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set up the job array.  In this instance we have requested 1000 tasks\n# numbered 1 to 1000.\n#$ -t 1-1000\n\n# Set the name of the job.\n#$ -N array-params\n\n# Set the working directory to somewhere in your scratch space.\n# Replace \"<your_UCL_id>\" with your UCL user ID :)\n#$ -wd /home/<your_UCL_id>/Scratch/output\n\n# Parse parameter file to get variables.\nnumber=$SGE_TASK_ID\nparamfile=/home/<your_UCL_id>/Scratch/input/params.txt\n\nindex=\"`sed -n ${number}p $paramfile | awk '{print $1}'`\"\nvariable1=\"`sed -n ${number}p $paramfile | awk '{print $2}'`\"\nvariable2=\"`sed -n ${number}p $paramfile | awk '{print $3}'`\"\nvariable3=\"`sed -n ${number}p $paramfile | awk '{print $4}'`\"\n\n# Run the program (replace echo with your binary and options).\n\necho \"$index\" \"$variable1\" \"$variable2\" \"$variable3\"\n
"},{"location":"Example_Jobscripts/#example-array-job-using-local2scratch","title":"Example Array Job Using Local2Scratch","text":"

Users can automate the transfer of data from $TMPDIR to their scratch space by adding the text #Local2Scratch to their script on a line alone as a special comment. During the clean-up phase of the job, a tool checks whether the script contains that text, and if so, files are transferred from $TMPDIR to a directory in scratch with the structure <job id>/<job id>.<task id>.<queue>/.

The example below does this for a job array, but this works for any job type.

#!/bin/bash -l\n\n# Batch script to run an array job under SGE and \n#  transfer the output to Scratch from local.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set up the job array.  In this instance we have requested 10000 tasks\n# numbered 1 to 10000.\n#$ -t 1-10000\n\n# Set the name of the job.\n#$ -N local2scratcharray\n\n# Set the working directory to somewhere in your scratch space.\n# Replace \"<your_UCL_id>\" with your UCL user ID :)\n#$ -wd /home/<your_UCL_id>/Scratch/output\n\n# Automate transfer of output to Scratch from $TMPDIR.\n#Local2Scratch\n\n# Run the application in TMPDIR.\ncd $TMPDIR\nhostname > hostname.txt\n
"},{"location":"Example_Jobscripts/#array-job-script-with-a-stride","title":"Array Job Script with a Stride","text":"

If each task for your array job is very small, you will get better use of the cluster if you can combine a number of these so each has a couple of hours' worth of work to do. There is a startup cost associated with the amount of time it takes to set up a new job. If your job's runtime is very small, this cost is proportionately high, and you incur it with every array task.

Using a stride will allow you to leave your input files numbered as before, and each array task will run N inputs.

For example, a stride of 10 will give you these task IDs: 1, 11, 21...

Your script can then have a loop that runs task IDs from $SGE_TASK_ID to $SGE_TASK_ID + 9, so each task is doing ten times as many runs as it was before.

#!/bin/bash -l\n\n# Batch script to run an array job with strided task IDs under SGE.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set up the job array.  In this instance we have requested task IDs\n# numbered 1 to 10000 with a stride of 10.\n#$ -t 1-10000:10\n\n# Set the name of the job.\n#$ -N arraystride\n\n# Set the working directory to somewhere in your scratch space.\n# Replace \"<your_UCL_id>\" with your UCL user ID :)\n#$ -wd /home/<your_UCL_id>/Scratch/output\n\n# Automate transfer of output to Scratch from $TMPDIR.\n#Local2Scratch\n\n# Do your work in $TMPDIR \ncd $TMPDIR\n\n# 10. Loop through the IDs covered by this stride and run the application if \n# the input file exists. (This is because the last stride may not have that\n# many inputs available). Or you can leave out the check and get an error.\nfor (( i=$SGE_TASK_ID; i<$SGE_TASK_ID+10; i++ ))\ndo\nif [ -f \"input.$i\" ]\nthen\necho \"$JOB_NAME\" \"$SGE_TASK_ID\" \"input.$i\"\nfi\ndone\n
"},{"location":"Example_Jobscripts/#hybrid-mpi-plus-openmp-jobscript-example","title":"Hybrid MPI plus OpenMP jobscript example","text":"

This is a type of job where you have a small number of MPI processes, and each one of those launches a number of threads. One common form of this is to have only one MPI process per node which handles communication between nodes, and the work on each node is done in a shared memory style by threads.

When requesting resources for this type of job, what you are asking the scheduler for is the physical number of cores and amount of memory per core that you need. Whether you end up running MPI processes or threads on that core is up to your code. (The -pe mpi xx request is telling it you want an MPI parallel environment and xx number of cores, not that you want xx MPI processes - this can be confusing).

"},{"location":"Example_Jobscripts/#setting-number-of-threads","title":"Setting number of threads","text":"

You can either set $OMP_NUM_THREADS for the number of OpenMP threads yourself, or allow it to be worked out automatically by setting it to OMP_NUM_THREADS=$(ppn). That is a helper script on our clusters which will set $OMP_NUM_THREADS to $NSLOTS/$NHOSTS, so you will use threads within a node and MPI between nodes and don't need to know in advance what size of node you are running on. Gerun will then run $NSLOTS/$OMP_NUM_THREADS processes, round-robin allocated (if supported by the MPI).

#!/bin/bash -l\n\n# Batch script to run a hybrid parallel job under SGE.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM per core (must be an integer)\n#$ -l mem=1G\n\n# Request 15 gigabytes of TMPDIR space per node (default is 10 GB - remove if cluster is diskless)\n#$ -l tmpfs=15G\n\n# Set the name of the job.\n#$ -N MadIntelHybrid\n\n# Select the MPI parallel environment and 80 cores.\n#$ -pe mpi 80\n\n# Set the working directory to somewhere in your scratch space. \n# This directory must exist.\n#$ -wd /home/<your_UCL_id/scratch/output/\n\n# Automatically set threads using ppn. On a cluster with 40 cores\n# per node this will be 80/2 = 40 threads.\nexport OMP_NUM_THREADS=$(ppn)\n\n# Run our MPI job with the default modules. Gerun is a wrapper script for mpirun. \ngerun $HOME/src/madscience/madhybrid\n

If you want to specify a specific number of OMP threads yourself, you would alter the relevant lines above to this:

# Request 80 cores and run 4 MPI processes per 40-core node, each spawning 10 threads\n#$ -pe mpi 80\nexport OMP_NUM_THREADS=10\ngerun your_binary\n
"},{"location":"Example_Jobscripts/#gpu-job-script-example","title":"GPU Job Script Example","text":"

To use NVIDIA GPUs with the CUDA libraries, you need to load the CUDA runtime libraries module or else set up the environment yourself. The script below shows what you'll need to unload and load the appropriate modules.

You also need to use the -l gpu=<number> option to request the GPUs from the scheduler.

#!/bin/bash -l\n\n# Batch script to run a GPU job under SGE.\n\n# Request a number of GPU cards, in this case 2 (the maximum)\n#$ -l gpu=2\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM (must be an integer followed by M, G, or T)\n#$ -l mem=1G\n\n# Request 15 gigabyte of TMPDIR space (default is 10 GB)\n#$ -l tmpfs=15G\n\n# Set the name of the job.\n#$ -N GPUJob\n\n# Set the working directory to somewhere in your scratch space.\n# Replace \"<your_UCL_id>\" with your UCL user ID :)\n#$ -wd /home/<your_UCL_id>/Scratch/output\n\n# Change into temporary directory to run work\ncd $TMPDIR\n\n# load the cuda module (in case you are running a CUDA program)\nmodule unload compilers mpi\nmodule load compilers/gnu/4.9.2\nmodule load cuda/7.5.18/gnu-4.9.2\n\n# Run the application - the line below is just a random example.\nmygpucode\n\n# 10. Preferably, tar-up (archive) all output files onto the shared scratch area\ntar zcvf $HOME/Scratch/files_from_job_$JOB_ID.tar.gz $TMPDIR\n\n# Make sure you have given enough time for the copy to complete!\n
"},{"location":"Example_Jobscripts/#job-using-mpi-and-gpus","title":"Job using MPI and GPUs","text":"

It is possible to run MPI programs that use GPUs but our clusters currently only support this within a single node. The script below shows how to run a program using 2 gpus and 12 cpus.

#!/bin/bash -l\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 12 cores, 2 GPUs, 1 gigabyte of RAM per CPU, 15 gigabyte of TMPDIR space\n#$ -l mem=1G\n#$ -l gpu=2\n#$ -pe mpi 12\n#$ -l tmpfs=15G\n\n# Set the name of the job.\n#$ -N GPUMPIrun\n\n# Set the working directory to somewhere in your scratch space.\n#$ -wd /home/<your user id>/Scratch/output/\n\n# Run our MPI job. You can choose OpenMPI or IntelMPI for GCC.\nmodule unload compilers mpi\nmodule load compilers/gnu/4.9.2\nmodule load mpi/openmpi/1.10.1/gnu-4.9.2\nmodule load cuda/7.5.18/gnu-4.9.2\n\ngerun myGPUapp\n
"},{"location":"Experienced_Users/","title":"Quick Start Guide for Experienced HPC Users","text":""},{"location":"Experienced_Users/#what-services-are-available","title":"What Services are available?","text":""},{"location":"Experienced_Users/#ucl","title":"UCL","text":""},{"location":"Experienced_Users/#external","title":"External","text":""},{"location":"Experienced_Users/#how-do-i-get-access","title":"How do I get access?","text":"

UCL services: Fill in the sign-up form

Tier 2 services: Contact your point of contact.

"},{"location":"Experienced_Users/#how-do-i-connect","title":"How do I connect?","text":"

All connections are via SSH, and you use your UCL credentials to log in (external users should use the mmmXXXX account with the SSH key they have provided to their point of contact). UCL services can only be connected to by users inside the UCL network which may mean using the institutional VPN or \"bouncing\" off another UCL machine when accessing them from outside the UCL network. The Tier 2 services (Thomas and Michael) are externally accessible.

"},{"location":"Experienced_Users/#login-hosts","title":"Login hosts","text":"Service General alias Direct login node addresses Grace grace.rc.ucl.ac.uk login01.ext.grace.ucl.ac.uk login02.ext.grace.ucl.ac.uk Kathleen kathleen.rc.ucl.ac.uk login01.kathleen.rc.ucl.ac.uk login02.kathleen.rc.ucl.ac.uk Myriad myriad.rc.ucl.ac.uk login12.myriad.rc.ucl.ac.uk login13.myriad.rc.ucl.ac.uk Thomas thomas.rc.ucl.ac.uk login03.thomas.rc.ucl.ac.uk login04.thomas.rc.ucl.ac.uk Michael michael.rc.ucl.ac.uk login10.michael.rc.ucl.ac.uk login11.michael.rc.ucl.ac.uk

Generally you should connect to the general alias as this is load-balanced across the available login nodes, however if you use screen or tmux you will want to use the direct hostname so that you can reconnect to your session.

"},{"location":"Experienced_Users/#software-stack","title":"Software stack","text":"

All UCL services use the same software stack based upon RHEL 7.x with a standardised set of packages, exposed to the user through environment modules (the module command). By default this has a set of useful tools loaded, as well as the Intel compilers and MPI but users are free to change their own environment.

"},{"location":"Experienced_Users/#batch-system","title":"Batch System","text":"

UCL services use Grid Engine to manage jobs. This install is somewhat customised and so scripts for non-UCL services may not work.

We recommend that when launching MPI jobs you use our gerun parallel launcher instead of mpirun as it inherits settings from the job and launches the appropriate number of processes with the MPI implementation you have chosen. It abstracts away a lot of the complexity between different version of MPI.

# using gerun\ngerun myMPIprogram\n\n# using mpirun when a machinefile is needed (eg Intel MPI)\nmpirun -np $NSLOTS -machinefile $PE_HOSTFILE myMPIprogram\n

$NSLOTS is an environment variable containing the value you gave to -pe mpi so you do not need to re-specify it.

"},{"location":"Experienced_Users/#troubleshooting-gerun","title":"Troubleshooting gerun","text":"

If you need to see what gerun is doing because something is not working as expected, look at the error file for your job, default name $JOBNAME.e$JOB_ID. It will contain debug information from gerun about where it ran and the exact mpirun command it used.

You may need to use mpirun directly with different options if your program has sufficiently complex process placement requirements, or is using something like GlobalArrays and requires a different process layout than it is being given.

"},{"location":"Experienced_Users/#script-sections","title":"Script sections","text":""},{"location":"Experienced_Users/#shebang","title":"Shebang","text":"

It's important that you add the -l option to bash in the shebang so that login scripts are parsed and the environment modules environment is set up. #! must be the first two characters in the file, no previous white space.

#!/bin/bash -l\n
"},{"location":"Experienced_Users/#resources-you-can-request","title":"Resources you can request","text":""},{"location":"Experienced_Users/#number-of-cores","title":"Number of cores","text":"

For MPI:

#$ -pe mpi <number of cores>\n

For threads:

#$ -pe smp <number of cores>\n

For single core jobs you don't need to request a number of cores. For hybrid codes use the MPI example and set OMP_NUM_THREADS to control the number of threads per node. gerun will launch the right number of processes in the right place if you use it.

"},{"location":"Experienced_Users/#amount-of-ram-per-core","title":"Amount of RAM per core","text":"
#$ -l mem=<integer amount of RAM in G or M>\n

e.g. #$ -l mem=4G requests 4 gigabytes of RAM per core.

"},{"location":"Experienced_Users/#run-time","title":"Run time","text":"
#$ -l h_rt=<hours:minutes:seconds>\n

e.g. #$ -l h_rt=48:00:00 requests 48 hours.

"},{"location":"Experienced_Users/#working-directory","title":"Working directory","text":"

Either a specific working directory:

#$ -wd /path/to/working/directory\n

or the directory the script was submitted from:

#$ -cwd\n
"},{"location":"Experienced_Users/#gpus-myriad-only","title":"GPUs (Myriad only)","text":"
#$ -l gpu=<number of GPUs>\n
"},{"location":"Experienced_Users/#enable-hyperthreads-kathleen-only","title":"Enable Hyperthreads (Kathleen only)","text":"
#$ -l threads=1\n

The -l threads= request is not a true/false setting, instead you are telling the scheduler you want one slot to block one virtual cpu instead of the normal situation where it blocks two. If you have a script with a threads request and want to override it on the command line or set it back to normal, the usual case is -l threads=2. (Setting threads to 0 does not disable hyperthreading!)

With Hyperthreads enabled you need to request twice as many cores and then control threads vs MPI ranks with OMP_NUM_THREADS. E.g. 

#$ -pe mpi 160\n#$ -l threads=1\nexport OMP_NUM_THREADS=2\n

Would use 80 cores, with two threads (on Hyperthreads) per core. If you use gerun to launch your MPI process, it will take care of the division for you, but if you're using mpirun or mpiexec directly, you'll have to take care to use the correct number of MPI ranks per node yourself.

Note that memory requests are now per virtual core with hyperthreading enabled. If you asked for #$ -l mem=4Gon a node with 80 virtual cores and 192G RAM then you are requiring 320G RAM in total which will not fit on that node and so you would be given a sparse process layout across more nodes to meet this requirement.

"},{"location":"Experienced_Users/#temporary-local-disk-every-machine-except-kathleen","title":"Temporary local disk (every machine EXCEPT Kathleen)","text":"
#$ -l tmpdir=<size in G>\n

e.g. #$ -l tmpdir=10G requests 10 gigabytes of temporary local disk.

"},{"location":"Experienced_Users/#the-rest-of-the-script","title":"The rest of the script","text":"

You need to load any module dependencies, set up any custom environment variables or paths you need and then run the rest of your workflow.

"},{"location":"Experienced_Users/#submitting-your-jobscript","title":"Submitting your jobscript","text":"

Job scripts can be submitted with qsub, jobs can be monitored with qstat and deleted with qdel.

"},{"location":"Experienced_Users/#interactive-jobs","title":"Interactive jobs","text":"

If you need to run an interactive job, possibly with X forwarding, you can do so using qrsh. Please see our page on interactive jobs for more details.

"},{"location":"Interactive_Jobs/","title":"Interactive Job Sessions","text":"

For an interactive session, you reserve some compute nodes via the scheduler and then are logged in live, just like on the login nodes. These can be used for live visualisation, software debugging, or to work up a script to run your program without having to submit each attempt separately to the queue and wait for it to complete.

"},{"location":"Interactive_Jobs/#requesting-access","title":"Requesting Access","text":"

You will be granted an interactive shell after running a command that checks with the scheduler whether the resources you wish to use in your tests/analysis are available. Interactive sessions are requested using the qrsh command. It typically takes the form:

qrsh\u00a0-pe\u00a0mpi\u00a08\u00a0-l\u00a0mem=512M,h_rt=2:00:00\u00a0-now\u00a0no\n

In this example you are asking to run eight parallel processes within an MPI environment, 512MB RAM per process, for a period of two hours.

All job types we support on the system are supported via an interactive session (see our examples section). Likewise, all qsub options are supported like regular job submission with the difference that with qrsh they must be given at the command line, and not with any job script (or via -@).

In addition the -now option is useful when a cluster is busy. By default qrsh and qlogin jobs will run on the next scheduling cycle or give up. The -now no option tells it to keep waiting until it gets scheduled. Pressing Ctrl+C (i.e. the control key and the C key at the same time) will safely cancel the request if it doesn't seem to be able to get you a session.

More resources can be found here:

"},{"location":"Interactive_Jobs/#interactive-x-sessions","title":"Interactive X sessions","text":"

You can get an interactive X session from the head node of the job back to the login node. The way to do this is to run the qrsh command in the following generic fashion:

qrsh\u00a0<options> <command>\u00a0<arguments\u00a0to\u00a0<command>>\n

Where <command> is either a command to launch an X terminal like Xterm or Mrxvt or a GUI application like XMGrace or GaussView.

To make effective use of the X forwarding you will need to have logged in to the login node with ssh -X or some equivalent method. Here is an example of how you can get a X terminal session with the qrsh command:

qrsh\u00a0-l\u00a0mem=512M,h_rt=0:30:0\u00a0\\\n\u00a0\u00a0\u00a0\"/shared/ucl/apps/mrxvt/0.5.4/bin/mrxvt\u00a0-title\u00a0'User\u00a0Test\u00a0Node'\"\n
"},{"location":"Interactive_Jobs/#working-on-the-nodes","title":"Working on the nodes","text":"

If you want to run a command on one of your allocated nodes which is not the headnode, you can use a standard ssh command: 

ssh\u00a0<hostname> <command>\u00a0[args]\n

Where <hostname> can be obtained by inspecting the file $TMPDIR/machines.

"},{"location":"Interactive_Jobs/#gpu-test-nodes","title":"GPU test nodes","text":"

You can also run GPU jobs interactively simply by adding the -l gpu=1 or -l gpu=2 options to the qrsh command as normal.

For more information, please contact us on rc-support@ucl.ac.uk

"},{"location":"Job_Results/","title":"Where do my results go?","text":"

After submitting your job, you can use the command qstat to view the status of all the jobs you have submitted. Once you can no longer see your job on the list, this means your job has completed. To view details on jobs that have completed, you can run jobhist. There are various ways of monitoring the output of your job.

"},{"location":"Job_Results/#output-and-error-files","title":"Output and error files","text":"

When writing your job script you can either tell it to start in the directory you submit it from (-cwd), or from a particular directory (-wd <dir>), or from your home directory (the default). When your job runs, it will create files in this directory for the job's output and errors:

File Name Contents myscript.sh Your job script. myscript.sh.o12345 Output from the job. (stdout) myscript.sh.e12345 Errors, warnings, and other messages from the job that aren't mixed into the output. (stderr) myscript.sh.po12345 Output from the setup script run before a job. (\"prolog\") myscript.sh.pe12345 Output from the clean-up script run after a job. (\"epilog\")

Normally there should be nothing in the .po and .pe files, and that's fine. If you change the name of the job in the queue, using the -N option, your output and error files will use that as the filename stem instead.

Most programs will also produce separate output files, in a way that is particular to that program. Often these will be in the same directory, but that depends on the program and how you ran it.

"},{"location":"Job_Results/#grid-engine-commands","title":"Grid Engine commands","text":"

The following commands can be used to submit and monitor a job.

"},{"location":"Job_Results/#qsub","title":"Qsub","text":"

This command submits your job to the batch queue. You can also use options on the command-line to override options you have put in your job script.

Command Action qsub myscript.sh Submit the script as-is qsub -N NewName myscript.sh Submit the script but change the job's name qsub -l h_rt=24:0:0 myscript.sh Submit the script but change the maximum run-time qsub -hold_jid 12345 myscript.sh Submit the script but make it wait for job 12345 to finish qsub -ac allow=XYZ myscript.sh Submit the script but only let it run on node classes X, Y, and Z"},{"location":"Job_Results/#qstat","title":"Qstat","text":"

This command shows the status of your jobs. When you run qstat with no options, all of your jobs currently running will be displayed. By adding in the option -f -j <job-ID> you will get more detail on the specified job.

"},{"location":"Job_Results/#qdel","title":"Qdel","text":"

This command deletes your job from the queue. When deleting a job will need to run qdel <job-ID>, however qdel '*' can be used to delete all jobs. To delete a batch of jobs, creating a file with the list of job IDs that you would like to delete and placing it in the following commands will delete the following jobs: cat <filename> | xargs qdel

"},{"location":"Job_Results/#qsub-emailing","title":"Qsub emailing","text":"

We also have a mailing system that can be implemented to send emails with reminders of the status of your job through qsub. In your jobscript, or when you use qsub to submit your job, you can use the option -m. You can specify when you want an email sent to you by using the below options after qsub -m:

b Mail is sent at the beginning of the job. e Mail is sent at the end of the job. a Mail is sent when the job is aborted or rescheduled. s Mail is sent when the job is suspended. n No mail is sent. (The default.)

You can use more than one of these options by putting them together after the -m option; for example, adding the following to your job script would mean you get an email when the job begins and when it ends:

#$ -m be\n

Further resources can be found here:

"},{"location":"New_Users/","title":"Guide for New Users","text":""},{"location":"Planned_Outages/","title":"Planned Outages","text":"

The second Tuesday of every month is a maintenance day, when the following clusters should be considered at risk from 8:00AM: Myriad, Kathleen, Thomas, Young, Michael, Aristotle and the Data Science Platform. We won\u2019t necessarily perform maintenance every month, and notice by email will not always be given about maintenance day work that only puts services at risk.

Full details of outages are emailed to the cluster-specific user lists.

Generally speaking, an outage will last from the morning of the first date listed until mid-morning of the end date listed. The nodes may need to be emptied of jobs in advance ('drained'), so jobs may remain in the queue for longer before an outage begins.

If there is a notable delay in bringing the system back we will contact you after approximately midday - please don't email us at 9am on the listed end days!

After an outage, the first day or two back should be considered 'at risk'; that is, things are more likely to go wrong without warning and we might need to make adjustments.

Date Service Status Reason"},{"location":"Planned_Outages/#previous-outages","title":"Previous Outages","text":"Date Service Status Reason 29 Aug -> 1 Sept 2023 Kathleen Completed Our vendors need us to do some filesystem testing on Kathleen to prepare for the replacement of its Scratch filesystem with a new filesystem later this year. We will drain jobs for 8am on Tues 29th. Depending on how long it takes to gather this information, we may not be running jobs for the rest of the week. We will update you on Thursday around lunchtime. (Jobs were restarted on Mon 4 Sept 12:00) 11 July 2023 Myriad Completed Full shutdown for replacement of both Lustre controllers in the filesystem. There will be no login access, no jobs running. Jobs will be drained for 8am. Myriad will be returned to service during the day on Weds 12 July. 10 -> 15 May 2023 Myriad Completed Power work taking place in the datacentre that Myriad is in which may result in some cabinets losing power. We will be draining jobs for 8am on Weds 10 May and re-enabling them again once the work is complete, likely to be during Tues 16 May. 21 -> 23 Apr 2023 Young, Michael, Kathleen Completed Electrical works in the datacentre require us to switch off all three clusters. Jobs will be drained for 12:00pm on Friday 21 April. The clusters will be powered down shortly after 12:00pm. You won't be able to log in or access data from then or throughout the weekend. We will bring the clusters back up during Monday 24 April. This may take a few hours to most of the day. 31 Jan 2023 -> 1st Feb 2023 Myriad Completed Full shutdown of the system to perform firmware updates to the storage system, including all controllers and drives. Jobs will be drained for 08:00 on 31st. It is intended to make the system available again in the evening of the 1st. 23 Dec 2022 -> 3-4 Jan 2023 Young, Michael, Kathleen Complete Full shutdown during UCL Christmas closing to prevent risk of filesystem damage from instability or power issues. Jobs will be drained for 9am on 23rd, shutting down at 1pm. Will be brought back up throughout 3 Jan with intent to have jobs running at some point on 4 Jan. 13 Dec 2022 -> 14 Dec 2022 Myriad Postponed Upgrade of firmware on all Lustre storage disks. Jobs will be drained for 8am on Tues 13 Dec and we expect to re-enable jobs on Weds 14 Dec. Discovered that we would need to update the storage controller firmware as well, and possibly the OSSs (object store servers) and Lustre itself. We don't want to do this right before the Christmas closure so are postponing the update until later. 1 Nov 2022 -> 2 Nov 2022 Myriad Complete Upgrade of firmware on all Lustre storage disks. No reading or writing of files will be possible during the upgrade interval. Jobs will be drained for 8am on Tues 1 Nov and we expect to re-enable jobs on Weds 2 Nov. 27 Sep 2022 -> 28 Sep 2022 Myriad Completed Revised dates Moving switches. Jobs will be drained for 8am on 27 Sept and access to the login nodes disabled. Access should be restored during the day on Thurs 29. Myriad should be considered at risk for the rest of the week. 12 Apr 2022 -> 19 Apr 2022 Kathleen, Young, Michael Completed Datacentre work. Clusters will be drained of jobs and access to the login nodes disabled, for 4:00pm Tuesday 12th April. Clusters will be brought back into service during the day on Tuesday 19th April. They should be considered AT RISK that day. 1 Apr 2022 -> 4 Apr 2022 Kathleen, Young, Michael Completed Datacentre work. Clusters will be will be drained of jobs and access to the login nodes disabled, for 4pm Friday 1st April. Clusters will be brought back into service on Monday the 4th April. They should be considered AT RISK that day. 7 Jan 2022 -> 10 Jan 2022 Kathleen, Young, Michael Completed Datacentre power outage. Jobs will be drained for 2pm on Fri 7. Login nodes will be powered down. Clusters will be brought back online during Mon 10, should be considered at risk that day. 19 Nov 2021 Myriad Completed One of our suppliers will be onsite to replace a component. Queues are being drained. 9 Nov 2021 Myriad Completed Continued work to improve the file system. Myriad will be unavailable for the whole day. Queues are being drained for 8am. 12 Oct 2021 Myriad Completed Filesystem load tests before expansion. No jobs will be running and access to login nodes will be blocked during the test, expected to finish by end of day. 29 Sept 2021 -> 30 Sept 2021, 5 Oct 2021 Young Completed Extended: expected back in service midday Tues 5 Oct. Queues drained so ZFS can be upgraded to patch a bug causing metadata server issues. No jobs will be running. 14 Sept 2021 Myriad Planned Queues drained for 8am so that the central software installs can be redistributed over more servers. This is to help mitigate the effects of current filesystem issues. 03 Aug 2021 Young, Michael Completed Shortly after 8am: ~10 mins Gold outage for security updates. New job submission and Gold commands will not work during this. 12 July 2021 Myriad Completed Drain of d97a (Economics) nodes for 8am to update network config. May require less than the full day. Jobs will remain in queue until drain is over or run on other nodes if they can. 15 Jun 2021 -> 18 Jun 2021 Myriad Completed Datacentre network outage. No jobs will run on the 15th, and Myriad will be at risk for the rest of the week. 14 Jun 2021 Kathleen, Young and Data Science Platform Completed Datacentre network outage 8.30-9am. No login access and no jobs will run. 08 Jun 2021 Thomas Completed Storage will be at risk during an essential controller reboot. 22 May 2021 -> 24 May 2021 Michael Completed Datacentre network outage. Queues to be drained. Full outage, no access. 22 May 2021 -> 24 May 2021 Young Completed Gold and job submission expected to be unavailable while database uncontactable due to datacentre network outage. 22 May 2021 -> 24 May 2021 Myriad and Kathleen Completed Gold (i.e. high priority) job submission expected to be unavailable while database uncontactable due to datacentre network outage. 29 Mar 2021 -> 30 Mar 2021 Myriad Completed A number of GPU nodes will be reserved for the NVidia AI Bootcamp on the 29th and 30th March. Some users may experience longer than usual waiting times for GPUs, from Monday the 22nd until after the event, especially for long jobs. We apologise for any inconvenience this may cause. 30 Mar 2021 Kathleen Completed Essential Lustre filesystem maintenance. Queue draining starts 26th March, SSH access to login nodes disabled on the 30th. Full outage, no access. 23 Feb 2021 8-9:30am Young Completed Gold and job submission expected to be unavailable for two 10-min periods while network switches upgraded. 09 Feb 2021 8-1pm Young Completed Gold and job submission outage while we migrate our database server. 08 Jan 2021 -> 12 Jan 2021 Kathleen Completed Electrical work in datacentre. Logins delayed until Tues morning while tests completed. 08 Jan 2021 -> 12 Jan 2021 MMM Michael Completed Electrical work in datacentre. Logins delayed until Tues morning while tests completed. 08 Jan 2021 -> 12 Jan 2021 MMM Young Completed Electrical work in datacentre. Logins delayed until Tues morning while tests completed. 02 Jul 2020 -> 09 Jul 2020 Michael Rollback Lustre software upgrade to fix bug. Full outage, no access. Upgrade was unusable, downgrading to previous. 01 May 2020 -> 11 May 2020 Myriad Completed Storage upgrade. 20 Mar 2020 -> 30 Mar 2020 Myriad Stopped, Postponed Issues found with new storage during outage. Myriad returned to service 24th. Sync data; switch to new storage. (Important: new metadata servers). Begins midday. Full outage, no access. 16 Mar 2020 -> (was 23) 26 Mar 2020 Michael Completed Firmware upgrades to bridge old and new networks. Extended to get jobs working on new nodes. 2 Mar 2020 -> 9 Mar 2020 Michael Completed Installation of phase 2 hardware, network bridging 10 Feb 2020 -> 17 Feb 2020 Myriad Cancelled Storage upgrade to 3PB"},{"location":"Planned_Outages/#retirements","title":"Retirements","text":"

These clusters or services are being retired. Notable dates are below.

"},{"location":"Planned_Outages/#grace","title":"Grace","text":""},{"location":"Planned_Outages/#mmm-hub-thomas","title":"MMM Hub Thomas","text":"

This service has been retired as part of the MMM Hub. Some remains are still running temporarily for UCL users only.

"},{"location":"Planned_Outages/#rstudio-rstudiorcuclacuk","title":"RStudio (rstudio.rc.ucl.ac.uk)","text":""},{"location":"Remote_Access/","title":"Remote Access to Research Computing Resources","text":"

UCL's Research Computing services are accessible from inside the UCL firewall. If you wish to connect from outside, you need to either connect through a VPN or use SSH to log in to a machine accessible from outside and use that to \"jump\" through into the UCL network.

"},{"location":"Remote_Access/#connecting-to-the-jump-boxes","title":"Connecting to the jump boxes","text":"

You can connect to the jump boxes by connecting with your SSH client to:

ssh-gateway.ucl.ac.uk\n

Once connected you can then log on to the UCL RC service you are using as normal.

You can configure your ssh client to automatically connect via these jump boxes so that you make the connection in one step.

"},{"location":"Remote_Access/#single-step-logins-using-tunnelling","title":"Single-step logins using tunnelling","text":""},{"location":"Remote_Access/#linux-unix-mac-os-x","title":"Linux / Unix / Mac OS X","text":""},{"location":"Remote_Access/#on-the-command-line","title":"On the command line","text":"
# Log in to Kathleen, jumping via jump box\n# Replace ccxxxxx with your own username.\nssh -o ProxyJump=ccxxxxx@ssh-gateway.ucl.ac.uk ccxxxxx@kathleen.rc.ucl.ac.uk\n

or

# Copy 'my_file', from the machine you are logged in to, into your Scratch on Kathleen\n# Replace ccxxxxx with your own username.\nscp -o ProxyJump=ccxxxxx@ssh-gateway.ucl.ac.uk my_file ccxxxxx@kathleen.rc.ucl.ac.uk:~/Scratch/\n

This tunnels through the jump box service in order to get you to your destination - you'll be asked for your password twice, once for each machine. You can use this to log in or to copy files.

You may also need to do this if you are trying to reach one cluster from another and there is a firewall in the way.

Useful resources can be found here:

"},{"location":"Remote_Access/#using-a-config-file","title":"Using a config file","text":"

You can create a config which does this without you needing to type it every time.

Inside your ~/.ssh directory on your local machine, add the below to your config file (or create a file called config if you don't already have one).

Generically, it should be of this form where <name> can be anything you want to call this entry. You can use these as short-hand names when you run ssh.

Host <name>\n   User <remote_user_id>\n   HostName <remote_hostname>\n   proxyCommand ssh -W <remote_hostname>:22 <remote_user_id>@ssh-gateway.ucl.ac.uk\n

This proxyCommand option causes the commands you type in your client to be forwarded on over a secure channel to the specified remote host.

On newer versions of OpenSSH, you can use ProxyJump <remote_user_id>@ssh-gateway.ucl.ac.uk instead of this proxyCommand line.

Here are some examples - you can have as many of these as you need in your config file.

Host myriad\n   User ccxxxxx\n   HostName myriad.rc.ucl.ac.uk\n   proxyCommand ssh -W myriad.rc.ucl.ac.uk:22 ccxxxxx@ssh-gateway.ucl.ac.uk\n\nHost kathleen01\n   User ccxxxxx\n   HostName login01.kathleen.rc.ucl.ac.uk\n   proxyCommand ssh -W login01.kathleen.rc.ucl.ac.uk:22 ccxxxxx@ssh-gateway.ucl.ac.uk\n\nHost aristotle\n   User ccxxxxx\n   HostName aristotle.rc.ucl.ac.uk\n   proxyCommand ssh -W aristotle.rc.ucl.ac.uk:22 ccxxxxx@ssh-gateway.ucl.ac.uk\n

You can now just type ssh myriad or scp file1 aristotle:~ and you will go through the jump box. You'll be asked for login details twice since you're logging in to two machines, the jump box and your endpoint.

"},{"location":"Remote_Access/#file-storage-on-the-gateway-servers","title":"File storage on the Gateway servers","text":"

The individual servers in the pool for the Gateway service have extremely limited file storage space, intentionally, and should not be used for storing files - if you need to transfer files you should use the two-step process above. This storage should only be used for SSH configuration files.

This storage is not mirrored across the jump boxes which means if you write a file to your home directory, you will not be able to read it if you are allocated to another jump box next time you log in.

"},{"location":"Remote_Access/#key-management","title":"Key management","text":"

Warning

If you use SSH keys you absolutely MUST NOT STORE UNENCRYPTED PRIVATE KEYS ON THIS OR ANY OTHER MULTI-USER COMPUTER. We will be running regular scans of the filesystem to identify and then block unencrypted key pairs across our services.

There are currently two servers in the pool, internally named ejp-gateway01 and ejp-gateway02.

Because the /home filesystem is not shared across the jump boxes, you need to sync SSH configuration files like ~/.ssh/authorized_keys across all the available jump boxes so that the change takes effect whichever jump box you are allocated to.

You can see which machine you are logged into by the bash prompt.

So for example, if on ejp-gateway02 then do:

[ccaaxxx@ad.ucl.ac.uk@ejp-gateway02 ~]$ scp -r ~/.ssh ejp-gateway01:\n\nPassword:\nknown_hosts 100% 196 87.1KB/s 00:00\nauthorized_keys 100% 0 0.0KB/s 00:00\n[ccaaxxx@ad.ucl.ac.uk@ejp-gateway02 ~]$\n

and similarly if on ejp-gateway01 do scp -r ~/.ssh ejp-gateway02:

"},{"location":"Status_page/","title":"Status of machines","text":"

This page outlines that status of each of the machines managed by the Research Computing team at UCL. We endeavour to keep this page as up-to-date as possible but there might be some delay. Also there are spontaneous errors that we have to deal with (i.e. high load on login nodes) but feel free to report them to rc-support@ucl.ac.uk. Finally, details of our planned outages can be found here.

"},{"location":"Status_page/#myriad","title":"Myriad","text":""},{"location":"Status_page/#kathleen","title":"Kathleen","text":""},{"location":"Status_page/#young","title":"Young","text":""},{"location":"Status_page/#michael","title":"Michael","text":""},{"location":"Status_page/#thomas","title":"Thomas","text":""},{"location":"Terms_and_Conditions/","title":"Terms and Conditions","text":"
  1. All use of Research Computing Platforms is subject to the UCL Computing Regulations.
  2. All users will be required to renew their account once per year. Users will receive a reminder one month prior to suspension of their Myriad account sent to their Live@UCL e-mail address. Funding information will need to be provided upon application, and publication information upon renewal.
  3. Users are forbidden from performing production runs on the login nodes.
  4. The Research Computing Platform Services Team reserve the right to suspend or ban without prior warning any use of the system which impairs its operation.
  5. With the exception of in cases where there is imminent harm or risk to the service, the Research Computing Platform Services Team will not access your files without permission.1
  6. Official service notifications are sent to the myriad-users (or the equivalent for other services) mailing list. Users are automatically subscribed to this list using their Live@UCL e-mail address and should read notices sent there.
  7. The Research Computing Platform Services Team reserve the right to suspend users' accounts, without notice, in the event of a user being the subject of any UCL disciplinary procedure, or where a user is found to be in breach of UCL\u2019s Computing Regulations or best practice guidelines regarding password management, as provided by Information Services Division.
  8. Users are required to acknowledge their use of Myriad and associated research computing services in any publications describing research that has been conducted, in any part, on Myriad. The following words should be used: \"The authors acknowledge the use of the UCL Myriad High Performance Computing Facility (Myriad@UCL), and associated support services, in the completion of this work\".
  9. All support requests should be sent by e-mail to rc-support@ucl.ac.uk.

  1. When you submit a job script, a copy is made, and that copy is public. This may appear as an exception to this term. Please do not include passwords or API keys in your job scripts.\u00a0\u21a9

"},{"location":"UCL_Service_For_Me/","title":"Which service(s) at UCL are right for me?","text":"

Depending on the type of jobs you would like to run each cluster can have different requirements that your jobs must meet. When you submit your user account application form you will be given access to the cluster depending on resources selected.

Currently we categorise intended workloads into ones that use:

Each cluster has its own specifications for the types of jobs that they run which is all dependable on list above. The cluster machines we have available are:

Myriad is designed to be most suitable for serial work, including large numbers of serial jobs, and multi-threaded jobs (using e.g. OpenMP). It also includes a small number of GPUs for development or testing work. It went into service in July 2018. See Myriad.

Kathleen is intended for multi-node jobs (using e.g. MPI) and went into service in Feb 2020 as a replacement for Grace. We recommend using Kathleen if you intend to use more than 36 cores per job. See Kathleen.

Thomas is the UK's Tier 2 Materials and Molecular Modelling Hub. It is accessible by members of partner institutions and relevant consortia, and is for materials and molecular modelling work only. It has separate access procedures from UCL's central clusters. Access is managed by a Point of Contact from the relevant institution or consortia, not by Research Computing. See Thomas.

Michael is an extension to the UCL-hosted Hub for Materials and Molecular Modelling, an EPSRC-funded Tier 2 system providing large scale computation to UK researchers; and delivers computational capability for the Faraday Institution, a national institute for electrochemical energy storage science and technology. Access is managed by a Point of Contact from the Faraday Institution, not by Research Computing. See Michael.

"},{"location":"howto/","title":"How do I?","text":"

I have an account, now:

"},{"location":"howto/#how-do-i-log-in","title":"How do I log in?","text":"

Logging in is most straightforward if you are inside the UCL firewall. If you are logging in from home or other external networks then you first have to get on to the UCL network.

"},{"location":"howto/#linux-unix-mac-os-x","title":"Linux / Unix / Mac OS X","text":"

Use the terminal and type the below command to secure shell (ssh) into the machine you wish to access. Replace <your_UCL_user_id> with your central UCL username, and <system_name> with the name of the machine you want to log in to, eg. myriad, kathleen, aristotle. 

ssh <your_UCL_user_id>@<system_name>.rc.ucl.ac.uk\n
"},{"location":"howto/#windows","title":"Windows","text":"

On Windows you need something that will give you a suitable terminal and ssh - usually PuTTY, or on Windows 10 you can use OpenSSH from a command prompt and type the same ssh command as the Linux instructions.

"},{"location":"howto/#using-putty","title":"Using PuTTY","text":"

PuTTY is a common SSH client on Windows and is available on Desktop@UCL. You can find it under: Start > P > PuTTY 0.76 (64-bit) > PuTTY 0.76 or type \"putty\" in the toolbar's search box.

You will need to create an entry for the host you are connecting to with the settings below. If you want to save your settings, give them an easily-identifiable name in the \"Saved Sessions\" box and press \"Save\". Then you can select it and \"Load\" next time you use PuTTY.

You will then be asked to enter your username and password. Only enter your username, not @<system_name>.rc.ucl.ac.uk. The password field will remain entirely blank when you type in to it - it does not show placeholders to indicate you have typed something.

The first time you log in to a new server, you'll get a popup telling you that the server's host key is not cached in the registry - this is normal and is because you have never connected to this server before. If you want to, you can check the host fingerprint against our current key fingerprints.

"},{"location":"howto/#logging-in-from-outside-the-ucl-firewall","title":"Logging in from outside the UCL firewall","text":"

You will need to either use the UCL Virtual Private Network or ssh in to UCL's Gateway system ssh-gateway.ucl.ac.uk first. From there you can then ssh in to our systems. 

ssh <your_UCL_user_id>@ssh-gateway.ucl.ac.uk\nssh <your_UCL_user_id>@<system_name>.rc.ucl.ac.uk\n

Advanced: If you find you need to go via the Gateway often, you can set up this jump automatically, see Single-step logins using tunnelling

"},{"location":"howto/#china-connect","title":"China Connect","text":"

If you are on China Connect, you will still need to ssh in to a UCL gateway first as described above.

"},{"location":"howto/#login-problems","title":"Login problems","text":"

If you experience difficulties with your login, please make sure that you are typing your UCL user ID and your password correctly. If you have recently updated your password, it takes some hours to propagate to all UCL systems.

If you still cannot get access but can access other UCL services like the SSH Gateway, please contact us on rc-support@ucl.ac.uk. Your account may have expired, or you may have gone over quota.

If you cannot access anything, please see UCL MyAccount - you may need to request a password reset from the Service Desk.

If you get a host key error message, you will need to delete old host keys - continue reading!

"},{"location":"howto/#remote-host-identification-has-changed","title":"Remote host identification has changed","text":"

When you log in via SSH, it keeps a record of the host key for the server you logged in to in your .ssh/known_hosts file in your home directory, on the machine you are logging in from. This helps make sure you are connecting directly to the server you think you are, but can cause warnings to show up if the host key on that machine has genuinely changed (usually because of an update or reinstall).

Check the host key warning against our current key fingerprints:

The error message looks like this if you are using OpenSSH in a terminal:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\n@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @\n@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\nIT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!\nSomeone could be eavesdropping on you right now (man-in-the-middle attack)!\nIt is also possible that a host key has just been changed.\nThe fingerprint for the ECDSA key sent by the remote host is\nSHA256:7FTryal3mIhWr9CqM3EPPeXsfezNk8Mm8HPCCAGXiIA.\nPlease contact your system administrator.\nAdd correct host key in /Users/uccaxxx/.ssh/known_hosts to get rid of this message.\nOffending ECDSA key in /Users/uccaxxx/.ssh/known_hosts:11\nECDSA host key for myriad.rc.ucl.ac.uk has changed and you have requested strict checking.\nHost key verification failed.\nKilled by signal 1.\n

This tells you that the old key is in line 11 of your known_hosts file. Sometimes it will give you a direct command you can run to remove that specific key:

ssh-keygen -R myriad.rc.ucl.ac.uk\n

or you can manually delete line 11 yourself in a text editor.

If you are logging in via the Gateway, you will need to remove the old key there too. On the Gateway, nano and vim are available text editors. If you are not already familiar with vim, use nano - it has the command shortcuts shown at the bottom, where ^O means press Ctrl and then the letter o.

# to open the file for editing in nano\nnano ~/.ssh/known_hosts\n

Once you have removed the old host key you will be able to ssh in again. The first time you log in to an unknown server you will get a message like this:

The authenticity of host 'myriad.rc.ucl.ac.uk can't be established.\nECDSA key fingerprint is SHA256:7FTryal3mIhWr9CqM3EPPeXsfezNk8Mm8HPCCAGXiIA.\nAre you sure you want to continue connecting (yes/no)?\n

Typing yes will allow you to continue logging in.

PuTTY will display a warning and you can choose to continue or not.

WinSCP will say Server's host key does not match the one that WinSCP has in cache. and you will have the option to update the key.

"},{"location":"howto/#mac-os-x-connection-failures","title":"Mac OS X connection failures","text":"

If you are on Mac OS X and getting many ssh connection failures and broken pipe messages when trying to log in, try adding an ssh timeout to your ssh command:

ssh -o ConnectTimeout=10 <your_UCL_user_id>@myriad.rc.ucl.ac.uk\n

This has particularly been a problem with Mac OS X Big Sur when using the VPN.

"},{"location":"howto/#how-do-i-transfer-data-onto-the-system","title":"How do I transfer data onto the system?","text":"

You can transfer data to and from our systems using any program capable of using the Secure Copy (SCP) protocol. This uses the same SSH system as you use to log in to a command line session, but then transfers data over it. This means that if you can use SSH to connect to a system, you can usually use SCP to transfer files to it.

"},{"location":"howto/#copying-files-using-linux-or-mac-os-x","title":"Copying files using Linux or Mac OS X","text":"

You can use the command-line utilities scp, sftp or rsync to copy your data about. You can also use a graphical client (Transmit, CyberDuck, FileZilla).

"},{"location":"howto/#scp","title":"scp","text":"

This will copy a data file from somewhere on your local machine to a specified location on the remote machine (Myriad etc).

scp <local_data_file> <remote_user_id>@<remote_hostname>:<remote_path>\n
# Example: copy myfile from your local current directory into Scratch on Myriad\nscp myfile ccxxxxx@myriad.rc.ucl.ac.uk:~/Scratch/\n

This will do the reverse, copying from the remote machine to your local machine. (This is still run from your local machine).

scp <remote_user_id>@<remote_hostname>:<remote_path><remote_data_file> <local_path>\n
# Example: copy myfile from Myriad into the Backups directory in your local current directory\nscp ccxxxxx@myriad.rc.ucl.ac.uk:~/Scratch/myfile Backups/\n
"},{"location":"howto/#sftp","title":"sftp","text":"

You can use sftp to log in to the remote machine, navigate through directories and use put and get to copy files from and to your local machine. lcd and lls are local equivalents of cd and ls so you can navigate through your local directories as you go. 

sftp <remote_user_id>@<remote_hostname>\ncd <remote_path>\nget <remote_file>\nlcd <local_path>\nput <local_file>\n
# Example: download a copy of file1 into your local current directory,\n# change local directory and upload a copy of file2\nsftp ccxxxxx@myriad.rc.ucl.ac.uk\ncd Scratch/files\nget file1\nlcd ../files_to_upload\nput file2\n
"},{"location":"howto/#rsync","title":"rsync","text":"

rsync is used to remotely synchronise directories, so can be used to only copy files which have changed. Have a look at man rsync as there are many options.

"},{"location":"howto/#copying-files-using-windows-and-winscp","title":"Copying files using Windows and WinSCP","text":"

WinSCP is a graphical client that you can use for scp or sftp.

  1. The login/create new session screen will open if this is the first time you are using WinSCP.
  2. You can choose SFTP or SCP as the file protocol. If you have an unstable connection with one, you may wish to try the other. SCP is probably generally better.
  3. Fill in the hostname of the machine you wish to connect to, your username and password.
  4. Click Save and give your settings a useful name.
  5. You'll then be shown your list of Stored sessions, which will have the one you just created.
  6. Select the session and click Login.
"},{"location":"howto/#mobaxterm","title":"MobaXterm","text":"

If using MobaXterm, you may need to set a password for the left side file manager panel separately as well as for the main panel, to allow you to drag and drop files and have them transferred to the cluster.

"},{"location":"howto/#transferring-files-from-outside-the-ucl-firewall","title":"Transferring files from outside the UCL firewall","text":"

As when logging in, when you are outside the UCL firewall you will need a method to connect inside it before you copy files.

You can use the UCL Virtual Private Network and scp direct to our systems or you can do some form of SSH tunnelling.

"},{"location":"howto/#single-step-logins-using-tunnelling","title":"Single-step logins using tunnelling","text":""},{"location":"howto/#linux-unix-mac-os-x_1","title":"Linux / Unix / Mac OS X","text":""},{"location":"howto/#on-the-command-line","title":"On the command line","text":"
# Log in to Myriad, jumping via the Gateway (replace ccxxxxx with your own username)\nssh -o ProxyJump=ccxxxxx@ssh-gateway.ucl.ac.uk ccxxxxx@myriad.rc.ucl.ac.uk\n

or

# Copy 'my_file' from the machine you are logged in to into your Scratch on Grace\n# Replace ccxxxxx with your own username.\nscp -o ProxyJump=ccxxxxx@ssh-gateway.ucl.ac.uk my_file ccxxxxx@myriad.rc.ucl.ac.uk:~/Scratch/\n

This tunnels through the Gateway in order to get you to your destination - you'll be asked for your password twice, once for each machine. You can use this to log in or to copy files.

You may also need to do this if you are trying to reach one cluster from another and there is a firewall in the way.

"},{"location":"howto/#using-a-config-file","title":"Using a config file","text":"

You can create a config file which does this without you needing to type it every time.

Inside your ~/.ssh directory on your local machine, add the below to your config file (or create a file called config if you don't already have one).

Generally, it should be of this form where <name> can be anything you want to call this entry.

Host <name>\n   User <remote_user_id>\n   HostName <remote_hostname>\n   proxyCommand ssh -W <remote_hostname>:22 <remote_user_id>@ssh-gateway.ucl.ac.uk\n

This causes the commands you type in your client to be forwarded on over a secure channel to the specified remote host.

Here are some examples - you can have as many of these as you need in your config file.

Host myriad\n   User ccxxxxx\n   HostName myriad.rc.ucl.ac.uk\n   proxyCommand ssh -W myriad.rc.ucl.ac.uk:22 ccxxxxx@ssh-gateway.ucl.ac.uk\n\nHost myriad12\n   User ccxxxxx\n   HostName login12.myriad.rc.ucl.ac.uk\n   proxyCommand ssh -W login12.myriad.rc.ucl.ac.uk:22 ccxxxxx@ssh-gateway.ucl.ac.uk\n\nHost aristotle\n   User ccxxxxx\n   HostName aristotle.rc.ucl.ac.uk\n   proxyCommand ssh -W aristotle.rc.ucl.ac.uk:22 ccxxxxx@ssh-gateway.ucl.ac.uk\n

You can now just type ssh myriad or scp file1 aristotle:~ and you will go through the Gateway. You'll be asked for login details twice since you're logging in to two machines, a Gateway server and your endpoint.

"},{"location":"howto/#windows-winscp","title":"Windows - WinSCP","text":"

WinSCP can also set up SSH tunnels.

  1. Create a new session as before, and tick the Advanced options box in the bottom left corner.
  2. Select Connection > Tunnel from the left pane.
  3. Tick the Connect through SSH tunnel box and enter the hostname of the gateway you are tunnelling through, for example ssh-gateway.ucl.ac.uk
  4. Fill in your username and password for that host. (Central UCL ones for the Gateway).
  5. Select Session from the left pane and fill in the hostname you want to end up on after the tunnel.
  6. Fill in your username and password for that host and set the file protocol to SCP.
  7. Save your settings with a useful name.
"},{"location":"howto/#creating-a-tunnel-that-other-applications-can-use","title":"Creating a tunnel that other applications can use","text":"

Some applications do not read your SSH config file and also cannot set up tunnels themselves, but can use one that you have created separately. FileZilla in particular is something you may want to do this with to transfer your files directly to the clusters from outside UCL using a graphical client.

"},{"location":"howto/#ssh-tunnel-creation-using-a-terminal","title":"SSH tunnel creation using a terminal","text":"

You can do this in Linux, Mac OS X and the Windows Command Prompt on Windows 10 and later.

Set up a tunnel between a port on your local computer (this is using 3333 as it is unlikely to be in use, but you can pick different ones) to Myriad's port 22 (which is the standard port for ssh), going via a UCL gateway.

# replace ccxxxxx with your UCL username\nssh -L 3333:myriad.rc.ucl.ac.uk:22 ccxxxxx@ssh-gateway.ucl.ac.uk \n

You may also want to use the -N option to tell it not to execute any remote commands and -f to put this command into the background if you want to continue to type other commands into the same terminal.

The tunnel now exists, and localhost:3333 on your computer connects to Myriad.

You can do this with ports other than 22 if you are not wanting to ssh in but to instead connect with a local browser to something running on Myriad. Here the port remains as 3333, something could be launched on that port on Myriad and your browser could be pointed at localhost:3333 to connect to it.

# replace ccxxxxx with your UCL username\nssh -L 3333:myriad.rc.ucl.ac.uk:3333 ccxxxxx@ssh-gateway.ucl.ac.uk\n

Do not leave things like this running for long periods on the login nodes.

"},{"location":"howto/#ssh-tunnel-creation-using-putty","title":"SSH tunnel creation using PuTTY","text":"

On Windows you can also set up a tunnel using PuTTY.

"},{"location":"howto/#connect-to-your-tunnel-with-an-application-like-filezilla","title":"Connect to your tunnel with an application (like FileZilla)","text":"

You can then tell your application to connect to localhost:3333 instead of Myriad. If it has separate boxes for hostname and port, put localhost as the hostname and 3333 as the port.

"},{"location":"howto/#managing-your-quota","title":"Managing your quota","text":"

After using lquota to see your total usage, you may wish to find what is using all your space.

du is a command that gives you information about your disk usage. Useful options are:

du -ch <dir>\ndu -h --max-depth=1\n

The first will give you a summary of the sizes of directory tree and subtrees inside the directory you specify, using human-readable sizes with a total at the bottom. The second will show you the totals for all top-level directories relative to where you are, plus the grand total. These can help you track down the locations of large amounts of data if you need to reduce your disk usage.

"},{"location":"howto/#how-do-i-connect-to-ucl-group-folders-n-drive-or-other-smb-mounts","title":"How do I connect to UCL group folders, N drive or other smb mounts?","text":"

You may have data stored in a UCL group folder (S drive) that you normally mount using smb. You can use smbclient to copy the files across onto Myriad (you do want them to be copied onto Myriad before you run any jobs using them, otherwise the compute node will be sitting there waiting for the copy to complete before it can do anything useful).

If the address you are trying to mount looks like smb://ad.ucl.ac.uk/groupfolders then you would do this:

smbclient //ad.ucl.ac.uk/groupfolders \n

This will give you a prompt where you can access that storage in an ftp-like way, where you can use get commands to copy files from there on to Myriad, or put commands to copy data into there from Myriad.

You can look at man smbclient on Myriad for the manual.

If you get an error like this:

\\ad.ucl.ac.ukgroupfolders: Not enough '\\' characters in service\n

then you need to change the format from \\\\ad.ucl.ac.uk\\groupfolders to //ad.ucl.ac.uk/groupfolders instead.

"},{"location":"howto/#ucl-n-drive","title":"UCL N drive","text":"

The N drive (Windows filestore, Filestore@UCL) can also be accessed using smb. To find out what smb address to use, look at Adding your home N drive to a Linux machine. Then use smbclient as above to connect to it.

"},{"location":"howto/#how-do-i-connect-out-to-an-ftp-server","title":"How do I connect out to an FTP server?","text":"

You cannot connect in to Myriad using FTP (we only allow SFTP access) but you can connect out to FTP servers run by other people.

Load the GNU inetutils module which provides ftp, telnet and tftp clients.

module load inetutils/1.9.4\n\n# connect to your desired server\nftp servername.ac.uk\n

You can then use put and get commands to put data on the remote FTP server or download it from there to Myriad.

"},{"location":"howto/#how-do-i-submit-a-job-to-the-scheduler","title":"How do I submit a job to the scheduler?","text":"

To submit a job to the scheduler you need to write a jobscript that contains the resources the job is asking for and the actual commands you want to run. This jobscript is then submitted using the qsub command.

qsub myjobscript\n

It will be put in to the queue and will begin running on the compute nodes at some point later when it has been allocated resources.

"},{"location":"howto/#passing-in-qsub-options-on-the-command-line","title":"Passing in qsub options on the command line","text":"

The #$ lines in your jobscript are options to qsub. It will take each line which has #$ as the first two characters and use the contents beyond that as an option.

You can also pass options directly to the qsub command and this will override the settings in your script. This can be useful if you are scripting your job submissions in more complicated ways.

For example, if you want to change the name of the job for this one instance of the job you can submit your script with:

qsub -N NewName myscript.sh\n

Or if you want to increase the wall-clock time to 24 hours:

qsub -l h_rt=24:0:0 myscript.sh\n

You can submit jobs with dependencies by using the -hold_jid option. For example, the command below submits a job that won't run until job 12345 has finished:

qsub -hold_jid 12345 myscript.sh\n

You may specify node type with the -ac allow= flags as below: 

qsub -ac allow=XYZ myscript.sh\n

That would restrict the job to running on nodes of type X, Y or Z (the older Legion nodes).

Note that for debugging purposes, it helps us if you have these options inside your jobscript rather than passed in on the command line whenever possible. We (and you) can see the exact jobscript that was submitted for every job that ran but not what command line options you submitted it with.

"},{"location":"howto/#checking-your-previous-jobscripts","title":"Checking your previous jobscripts","text":"

If you want to check what you submitted for a specific job ID, you can do it with the scriptfor utility.

scriptfor 12345\n

As mentioned above, this will not show any command line options you passed in.

"},{"location":"howto/#how-do-i-monitor-a-job","title":"How do I monitor a job?","text":""},{"location":"howto/#qstat","title":"qstat","text":"

The qstat command shows the status of your jobs. By default, if you run it with no options, it shows only your jobs (and no-one else\u2019s). This makes it easier to keep track of your jobs.

The output will look something like this:

job-ID  prior   name       user         state submit/start at     queue                          slots ja-task-ID \n-----------------------------------------------------------------------------------------------------------------\n123454 2.00685 DI_m3      ccxxxxx      Eqw   10/13/2017 15:29:11                                    12 \n123456 2.00685 DI_m3      ccxxxxx      r     10/13/2017 15:29:11 Yorick@node-x02e-006               24 \n123457 2.00398 DI_m2      ucappka      qw    10/12/2017 14:42:12                                    1 \n

This shows you the job ID, the numeric priority the scheduler has assigned to the job, the name you have given the job, your username, the state the job is in, the date and time it was submitted at (or started at, if it has begun), the head node of the job, the number of 'slots' it is taking up, and if it is an array job the last column shows the task ID.

The queue name (Yorick here) is generally not useful. The head node name (node-x02e-006) is useful - the node-x part tells you this is an X-type node.

If you want to get more information on a particular job, note its job ID and then use the -f and -j flags to get full output about that job. Most of this information is not very useful.

qstat -f -j 12345\n
"},{"location":"howto/#job-states","title":"Job states","text":"

Many jobs cycling between Rq and Rr generally means there is a dodgy compute node which is failing pre-job checks, but is free so everything tries to run there. In this case, let us know and we will investigate.

If a job stays in t or dr state for a long time, the node it was on is likely to be unresponsive - again let us know and we'll investigate.

A job in Eqw will remain in that state until you delete it - you should first have a look at what the error was with qexplain.

"},{"location":"howto/#qexplain","title":"qexplain","text":"

This is a utility to show you the non-truncated error reported by your job. qstat -j will show you a truncated version near the bottom of the output.

qexplain 123454\n
"},{"location":"howto/#qdel","title":"qdel","text":"

You use qdel to delete a job from the queue.

qdel 123454\n

You can delete all your jobs at once:

qdel '*'\n
"},{"location":"howto/#more-scheduler-commands","title":"More scheduler commands","text":"

Have a look at man qstat and note the commands shown in the SEE ALSO section of the manual page. Exit the manual page and then look at the man pages for those. (You will not be able to run all commands).

"},{"location":"howto/#nodesforjob","title":"nodesforjob","text":"

This is a utility that shows you the current percentage load, memory used and swap used on the nodes your job is running on. If your job is sharing the node with other people's jobs, it will show you the total resources in use, not just those used by your job. This is a snapshot of the current time and resource usage may change over the course of your job. Bear in mind that memory use in particular can increase over time as your job runs.

If a cluster has hyperthreading enabled and you aren't using it, full load will show as 50% and not 100% - this is normal and not a problem.

For a parallel job, very low (or zero) usage of any of the nodes suggests your job is either not capable of running over multiple nodes, or not partitioning its work effectively - you may be asking for more cores than it can use, or asking for a number of cores that doesn't fit well into the node sizes, leaving many idle.

[uccacxx@login02 ~]$ nodesforjob 1234\nNodes for job 1234:\n  Primary:\n    node-r99a-238:  103.1 % load, 12.9 % memory used, 0.1% swap used\n  Secondaries:\n    node-r99a-206:  1.7 % load, 1.6 % memory used, 0.1% swap used\n    node-r99a-238:  103.1 % load, 12.9 % memory used, 0.1% swap used\n    node-r99a-292:  103.1 % load, 12.9 % memory used, 0.1% swap used\n    node-r99a-651:  1.6 % load, 3.2 % memory used, 0.1% swap used\n

The above example shows a multi-node job, so all the usage belongs to this job itself. It is running on four nodes, and node-r99a-238 is the head node (the one that launched the job) and shows up in both Primary and Secondaries. The load is very unbalanced - it is using two nodes flat out, and two are mostly doing nothing. Memory use is low. Swap use is essentially zero.

"},{"location":"howto/#jobhist","title":"jobhist","text":"

Once a job ends, it no longer shows up in qstat. To see information about your finished jobs - when they started, when they ended, what node they ran on - type jobhist.

[uccacxx@login02 ~]$ jobhist\n        FSTIME        |       FETIME        |   HOSTNAME    |  OWNER  | JOB NUMBER | TASK NUMBER | EXIT STATUS |   JOB NAME    \n----------------------+---------------------+---------------+---------+------------+-------------+-------------+---------------\n  2020-06-17 16:31:12 | 2020-06-17 16:34:19 | node-h00a-010 | uccacxx |    3854822 |           0 |           0 | m_job   \n  2020-06-17 16:56:50 | 2020-06-17 16:56:52 | node-d00a-023 | uccacxx |    3854836 |           0 |           1 | k_job  \n  2020-06-17 17:21:12 | 2020-06-17 17:21:46 | node-d00a-012 | uccacxx |    3854859 |           0 |           0 | k_job\n

FSTIME - when the job started running on the node FETIME - when the job ended HOSTNAME - the head node of the job (if it ran on multiple nodes, it only lists the first) TASK NUMBER - if it was an array job, it will have a different number here for each task

This shows jobs that finished in the last 24 hours by default. You can search for longer as well:

jobhist --hours=200\n

If a job ended and didn't create the files you expect, check the start and end times to see whether it ran out of wallclock time.

If a job only ran for seconds and didn't produce the expected output, there was probably something wrong in your script - check the .o and .e files in the directory you submitted the job from for errors.

"},{"location":"howto/#how-do-i-run-interactive-jobs","title":"How do I run interactive jobs?","text":"

Sometimes you need to run interactive programs, sometimes with a GUI. This can be achieved through qrsh. We have a detailed guide to running interactive jobs.

"},{"location":"howto/#how-do-i-estimate-what-resources-to-request-in-my-jobscript","title":"How do I estimate what resources to request in my jobscript?","text":"

It can be difficult to know where to start when estimating the resources your job will need. One way you can find out what resources your jobs need is to submit one job which requests far more than you think necessary, and gather data on what it actually uses. If you aren't sure what 'far more' entails, request the maximum wallclock time and job size that will fit on one node, and reduce this after you have some idea.

Run your program as:

 /usr/bin/time --verbose myprogram myargs\n

where myprogram myargs is however you normally run your program, with whatever options you pass to it.

When your job finishes, you will get output about the resources it used and how long it took - the relevant one for memory is maxrss (maximum resident set size) which roughly tells you the largest amount of memory it used.

If your job is not completing successfully or you need to know how the memory usage changes throughout the job, there is a tool called Ruse that can measure this for you.

Run your program as:

module load ruse/2.0\n\n# sample the current memory usage every 120s and output each step to stdout\nruse --stdout --time=120 -s myprogram myargs\n

where myprogram myargs is however you normally run your program, with whatever options you pass to it.

Remember that memory requests in your jobscript are always per core, so check the total you are requesting is sensible - if you increase it too much you may end up with a job that cannot be submitted.

You can also look at nodesforjob while a job is running to see a snapshot of the memory, swap and load on the nodes your job is running on.

"},{"location":"howto/#how-can-i-see-what-types-of-node-a-cluster-has","title":"How can I see what types of node a cluster has?","text":"

As well as looking at the cluster-specific page in this documentation for more details (for example Myriad), you can run nodetypes, which will give you basic information about the nodes that exist in that cluster.

[uccacxx@login12 ~]$ nodetypes\n    3 type * nodes: 36 cores, 188.4G RAM\n    7 type B nodes: 36 cores,   1.5T RAM\n   66 type D nodes: 36 cores, 188.4G RAM\n    9 type E nodes: 36 cores, 188.4G RAM\n    1 type F nodes: 36 cores, 188.4G RAM\n   55 type H nodes: 36 cores, 188.4G RAM\n    3 type I nodes: 36 cores,   1.5T RAM\n    2 type J nodes: 36 cores, 188.4G RAM\n

This shows how many of each letter-labelled nodetype the cluster has, then the number of cores and amount of memory the node is reporting it has. It also shows the cluster has some utility nodes - those are part of the infrastructure. The * nodes are the login nodes.

"},{"location":"howto/#how-do-i-run-a-graphical-program","title":"How do I run a graphical program?","text":"

To run a graphical program on the cluster and be able to view the user interface on your own local computer, you will need to have an X-Windows Server installed on your local computer and use X-forwarding.

"},{"location":"howto/#x-forwarding-on-linux","title":"X-forwarding on Linux","text":"

Desktop Linux operating systems already have X-Windows installed, so you just need to ssh in with the correct flags.

You need to make sure you use either the -X or -Y (look at man ssh for details) flags on all ssh commands you run to establish a connection to the cluster.

For example, connecting from outside of UCL:

ssh -X <your_UCL_user_id>@ssh-gateway.ucl.ac.uk\n

and then

ssh -X <your_UCL_user_id>@myriad.rc.ucl.ac.uk\n

A video walkthrough of running remote applications using X11, X-forwarding on compute nodes.

"},{"location":"howto/#x-forwarding-on-mac-os-x","title":"X-forwarding on Mac OS X","text":"

You will need to install XQuartz to provide an X-Window System for Mac OS X. (Previously known as X11.app).

You can then follow the Linux instructions using Terminal.app.

"},{"location":"howto/#x-forwarding-on-windows","title":"X-forwarding on Windows","text":"

You will need:

Exceed is available on Desktop@UCL machines and downloadable from the UCL software database. Xming is open source (and mentioned here without testing).

"},{"location":"howto/#exceed-on-desktopucl","title":"Exceed on Desktop@UCL","text":"
  1. Load Exceed. You can find it under Start > All Programs > Applications O-P > Open Text Exceed 14 > Exceed
  2. Open PuTTY (Applications O-P > PuTTY)
  3. In PuTTY, set up the connection with the host machine as usual:
  4. Then, from the Category menu, select Connection > SSH > X11 for 'Options controlling SSH X11 forwarding'.
  5. Return to the session menu and save these settings with a new identifiable name for reuse in future.
  6. Click 'Open' and login to the host as usual
  7. To test that X-forwarding is working, try running nedit which is a text editor in our default modules.

If nedit works, you have successfully enabled X-forwarding for graphical applications.

"},{"location":"howto/#installing-xming","title":"Installing Xming","text":"

Xming is a popular open source X server for Windows. These are instructions for using it alongside PuTTY. Other SSH clients and X servers are available. We cannot verify how well it may be working.

  1. Install both PuTTY and Xming if you have not done so already. During Xming installation, choose not to install an SSH client.
  2. Open Xming - the Xming icon should appear on the task bar.
  3. Open PuTTY
  4. Set up PuTTY as shown in the Exceed section.
"},{"location":"Background/Cluster_Computing/","title":"Cluster Computing","text":""},{"location":"Background/Cluster_Computing/#what-is-a-cluster","title":"What is a cluster?","text":"

In this context, a cluster is a collection of computers (often referred to as \"nodes\"). They're networked together with some shared storage and a scheduling system that lets people run programs on them without having to enter commands \"live\".

"},{"location":"Background/Cluster_Computing/#why-would-i-want-to-use-one","title":"Why would I want to use one?","text":"

Some researchers have programs that require a lot of compute power, like simulating weather patterns or the quantum behaviour of molecules.

Others have a lot of data to process, or need to simulate a lot of things at once, like simulating the spread of disease or assembling parts of DNA into a genome.

Often these kinds of work are either impossible or would take far too long to do on a desktop or laptop computer, as well as making the computer unavailable to do everyday tasks like writing documents or reading papers.

By running the programs on the computers in a cluster, researchers can use many powerful computers at once, without locking up their own one.

"},{"location":"Background/Cluster_Computing/#how-do-i-use-it","title":"How do I use it?","text":"

Most people use something like the following workflow:

Most people connect using a program called a Secure Shell Client (\"ssh client\" for short), but some programs, like Matlab and Comsol, run on your own computer and can be set up to send work to the cluster automatically. That can be a little tricky to get working, though.

The ssh client gives you a command prompt when you can enter text commands, but you can also tell it to pass graphical windows through the network connection, using a system called X-Forwarding. This can be useful for visualising your data without transferring all the files back, but the network connection makes it a bit slower to use than running it on your own computer. You'll need an X server on your own computer to use this: check our page on X-Forwarding for details.

Please be aware that login nodes are shared resources, so users should not be running memory intensive jobs nor jobs with long runtimes in the login node. Doing so may negatively impact the performance of the login node for other users.

"},{"location":"Clusters/Acknowledging_RC_Systems/","title":"Acknowledging the Use of RC Systems","text":"

To keep running our services, we depend on being able to demonstrate that they are used in published research.

When preparing papers describing work that has used any of our clusters or services, please use the terms below, especially the \"service@UCL\" label, so that we can easily search for them.

"},{"location":"Clusters/Acknowledging_RC_Systems/#kathleen","title":"Kathleen","text":"

\"The authors acknowledge the use of the UCL Kathleen High Performance Computing Facility (Kathleen@UCL), and associated support services, in the completion of this work.\"

"},{"location":"Clusters/Acknowledging_RC_Systems/#myriad","title":"Myriad","text":"

\"The authors acknowledge the use of the UCL Myriad High Performance Computing Facility (Myriad@UCL), and associated support services, in the completion of this work.\"

"},{"location":"Clusters/Acknowledging_RC_Systems/#aristotle","title":"Aristotle","text":"

\"The authors acknowledge the use of the UCL Aristotle Computing Facility (Aristotle@UCL), and associated support services, in the completion of this work.\"

"},{"location":"Clusters/Acknowledging_RC_Systems/#mmm-hub-young","title":"MMM Hub Young","text":"

Please find the appropriate wording at Acknowledging the use of Young in publications.

"},{"location":"Clusters/Acknowledging_RC_Systems/#mmm-hub-thomas","title":"MMM Hub Thomas","text":"

Please find the appropriate wording at Acknowledging the use of Thomas in publications.

"},{"location":"Clusters/Kathleen/","title":"Kathleen","text":"

Kathleen is a compute cluster designed for extensively parallel, multi-node batch-processing jobs, having high-bandwidth connections between each individual node. It is named after Professor Dame Kathleen Lonsdale, a pioneering chemist and activist, and was installed in December 2019. It went into service at the end of January 2020.

"},{"location":"Clusters/Kathleen/#accounts","title":"Accounts","text":"

Kathleen accounts can be applied for via the Research Computing sign up process.

As Kathleen is intended for multi-node jobs, users who specify that they will need to use multi-node jobs (e.g. with MPI) will be given access to Kathleen.

"},{"location":"Clusters/Kathleen/#logging-in","title":"Logging in","text":"

Please use your UCL username and password to connect to Kathleen with an SSH client.

ssh uccaxxx@kathleen.rc.ucl.ac.uk\n

If using PuTTY, put kathleen.rc.ucl.ac.uk as the hostname and your seven-character username (with no @ after) as the username when logging in, eg. uccaxxx. When entering your password in PuTTY no characters or bulletpoints will show on screen - this is normal.

If you are outside the UCL firewall you will need to follow the instructions for Logging in from outside the UCL firewall.

"},{"location":"Clusters/Kathleen/#logging-in-to-a-specific-node","title":"Logging in to a specific node","text":"

You can access a specific Kathleen login node by using their dedicated addresses instead of the main kathleen.rc.ucl.ac.uk address, for example:

ssh\u00a0uccaxxx@login01.kathleen.rc.ucl.ac.uk\n

The main address will unpredictably direct you to either one of these (to balance load), so if you need multiple sessions on one, this lets you do that.

"},{"location":"Clusters/Kathleen/#copying-data-onto-kathleen","title":"Copying data onto Kathleen","text":"

You will need to use an SCP or SFTP client to copy data onto Kathleen. Please refer to the page on How do I transfer data onto the system?

You can connect directly in both directions between Grace and Kathleen.

If you find you cannot connect directly from one cluster to another, this is generally because of firewalls in between and so you need to use tunnelling with the scp command.

"},{"location":"Clusters/Kathleen/#quotas","title":"Quotas","text":"

On Kathleen you have a single 250GB quota by default which covers your home and Scratch.

This is a hard quota: once you reach it, you will no longer be able to write more data. Keep an eye on it, as this will cause jobs to fail if they cannot create their .o or .e files at the start, or their output files partway through.

You can check your quota on Kathleen by running:

lquota\n

which will give you output similar to this:

     Storage        Used        Quota   % Used   Path\n      lustre  146.19 MiB   250.00 GiB       0%   /home/uccaxxx\n

You can apply for quota increases using the form at Additional Resource Requests.

Here are some tips for managing your quota and finding where space is being used.

"},{"location":"Clusters/Kathleen/#job-sizes-and-durations","title":"Job sizes and durations","text":"

Please consider that Kathleen nodes have 40 physical cores - 2 nodes is 80 cores. Jobs do not share nodes, so although asking for 41 cores is possible, it means you are wasting the other 39 cores on your second node!

For interactive and batch jobs:

Cores Max. Duration 41-240 48h 241-480 24h 481-5760 12h

These are numbers of physical cores.

If you have a workload that requires longer jobs than this, you may be able to apply to our governance group for access to a longer queue. Applications will be expected to demonstrate that their work cannot be run using techniques like checkpointing that would allow their workload to be broken up into smaller parts. Please see the section on Additional Resource Requests for more details.

"},{"location":"Clusters/Kathleen/#node-types","title":"Node types","text":"

Kathleen's compute capability comprises 192 diskless compute nodes each with two 20-core Intel Xeon Gold 6248 2.5GHz processors, 192 gigabytes of 2933MHz DDR4 RAM, and an Intel OmniPath network.

Two nodes identical to these, but with two 1 terabyte hard-disk drives added, serve as the login nodes.

"},{"location":"Clusters/Kathleen/#hyperthreading","title":"Hyperthreading","text":"

Kathleen has hyperthreading enabled and you can choose on a per-job basis whether you want to use it.

Hyperthreading lets you use two virtual cores instead of one physical core - some programs can take advantage of this.

If you do not ask for hyperthreading, your job only uses one thread per core as normal.

The -l threads= request is not a true/false setting, instead you are telling the scheduler you want one slot to block one virtual cpu instead of the normal situation where it blocks two. If you have a script with a threads request and want to override it on the command line or set it back to normal, the usual case is -l threads=2. (Setting threads to 0 does not disable hyperthreading!)

# request hyperthreading in this job\n#$ -l threads=1\n\n# request the number of virtual cores\n#$ -pe mpi 160\n\n# request 2G RAM per virtual core\n#$ -l mem=2G\n\n# set number of OpenMP threads being used per MPI process \nexport OMP_NUM_THREADS=2\n

This job would be using 80 physical cores, using 80 MPI processes each of which would create two threads (on Hyperthreads).

Note that memory requests are now per virtual core with hyperthreading enabled. If you asked for #$ -l mem=4Gon a node with 80 virtual cores and 192G RAM then you are requiring 320G RAM in total which will not fit on that node and so you would be given a sparse process layout across more nodes to meet this requirement.

# request hyperthreading in this job\n#$ -l threads=1\n\n# request the number of virtual cores\n#$ -pe mpi 160\n\n# request 2G RAM per virtual core\n#$ -l mem=2G\n\n# set number of OpenMP threads being used per MPI process\n# (a whole node's worth)\nexport OMP_NUM_THREADS=80\n

This job would still be using 80 physical cores, but would use one MPI process per node which would create 80 threads on the node (on Hyperthreads).

"},{"location":"Clusters/Kathleen/#diskless-nodes","title":"Diskless nodes","text":"

Kathleen nodes are diskless (have no local hard drives) - there is no $TMPDIR available on Kathleen, so you should not request -l tmpfs=10G in your jobscripts or your job will be rejected at submit time.

If you need temporary space, you should use somewhere in your Scratch.

"},{"location":"Clusters/Kathleen/#loading-and-unloading-modules","title":"Loading and unloading modules","text":"

Kathleen has a newer version of modulecmd which tries to manage module dependencies automatically by loading or unloading prerequisites for you whenever possible.

If you get an error like this:

[uccaxxx@login01.kathleen ~]$ module unload compilers mpi\nUnloading compilers/intel/2018/update3\n  ERROR: compilers/intel/2018/update3 cannot be unloaded due to a prereq.\n    HINT: Might try \"module unload default-modules/2018\" first.\n\nUnloading mpi/intel/2018/update3/intel\n  ERROR: mpi/intel/2018/update3/intel cannot be unloaded due to a prereq.\n    HINT: Might try \"module unload default-modules/2018\" first.\n

You can use the -f option to force the module change. It will carry it out and warn you about modules it thinks are dependent.

[uccaxxx@login01.kathleen ~]$ module unload -f compilers mpi\nUnloading compilers/intel/2018/update3\n  WARNING: Dependent default-modules/2018 is loaded\n\nUnloading mpi/intel/2018/update3/intel\n  WARNING: Dependent default-modules/2018 is loaded\n

Otherwise you will need to unload default-modules/2018 to swap compiler and MPI module, but that will leave you without gerun in your path. You can then do either of these things:

# load everything that was in default-modules except the compiler and mpi\nmodule unload default-modules/2018\nmodule load rcps-core/1.0.0\nmodule load whatever\n

or

# just reload gerun\nmodule unload default-modules/2018\nmodule load gerun\nmodule load whatever\n
"},{"location":"Clusters/Michael/","title":"Michael","text":"

Michael is an extension to the UCL-hosted Hub for Materials and Molecular Modelling, an EPSRC-funded Tier 2 system providing large scale computation to UK researchers; and delivers computational capability for the Faraday Institution, a national institute for electrochemical energy storage science and technology.

"},{"location":"Clusters/Michael/#applying-for-an-account","title":"Applying for an account","text":"

Michael accounts belong to you as an individual and are applied for via David Scanlon who is the point of contact for Michael.

You will need to supply an SSH public key, which is the only method used to log in.

"},{"location":"Clusters/Michael/#creating-an-ssh-key-pair","title":"Creating an ssh key pair","text":"

An ssh key consists of a public and a private part, typically named id_rsa and id_rsa.pub by default. The public part is what we need. You must not share your private key with anyone else. You can copy it onto multiple machines belonging to you so you can log in from all of them (or you can have a separate pair for each machine).

"},{"location":"Clusters/Michael/#creating-an-ssh-key-in-linuxunixmac-os-x","title":"Creating an ssh key in Linux/Unix/Mac OS X","text":"
ssh-keygen\u00a0-t\u00a0rsa\n

The defaults should give you a reasonable key. If you prefer to use ed25519 instead, and/or longer keys, you can. You can also tell it to create one with a different name, so it doesn't overwrite any existing key.

You will be asked to add a passphrase for your key. A blank passphrase is not recommended; if you use one please make sure that no one else ever has access to your local computer account. How often you are asked for a passphrase depends on how long your local ssh agent keeps it.

You may need to run ssh-add to add the key to your agent so you can use it. If you aren't sure what keys your agent can see, running ssh-add -L will show all the public parts of the keys it is aware of.

"},{"location":"Clusters/Michael/#creating-an-ssh-key-in-windows","title":"Creating an ssh key in Windows","text":"

Have a look at Key-Based SSH Logins With PuTTY which has step-by-step instructions. You can choose whether to use Pageant or not to manage your key. You can again pick RSA, ED25519, ECDSA etc but do not pick SSH-1 as that is a very old and insecure key type. As above, DSA is no longer accepted. The key must be at least 2048-bit.

If you are using Windows 10, then you probably have OpenSSH installed and could instead run ssh-keygen in a terminal per the Linux instructions and use the ssh command to log in instead of PuTTY.

"},{"location":"Clusters/Michael/#information-for-points-of-contact","title":"Information for Points of Contact","text":"

Points of Contact have some tools they can use to manage users and allocations, documented at MMM Points of Contact.

"},{"location":"Clusters/Michael/#logging-in","title":"Logging in","text":"

You will be assigned a personal username and your SSH key pair will be used to log in. External users will have a username in the form mmmxxxx (where xxxx is a number) and UCL users will use their central username.

You connect with ssh directly to:

michael.rc.ucl.ac.uk\n
"},{"location":"Clusters/Michael/#ssh-timeouts","title":"SSH timeouts","text":"

Idle ssh sessions will be disconnected after 7 days.

"},{"location":"Clusters/Michael/#using-the-system","title":"Using the system","text":"

Michael is a batch system. The login nodes allow you to manage your files, compile code and submit jobs. Very short (\\<15mins) and non-resource-intensive software tests can be run on the login nodes, but anything more should be submitted as a job.

"},{"location":"Clusters/Michael/#full-user-guide","title":"Full user guide","text":"

Michael has the same user environment as RC Support's other clusters, so the User guide is relevant and is a good starting point for further information about how the environment works. Any variations that Michael has should be listed on this page.

"},{"location":"Clusters/Michael/#submitting-a-job","title":"Submitting a job","text":"

Create a job script for non-interactive use and submit your jobscript using qsub. Jobscripts must begin #!/bin/bash -l in order to run as a login shell and get your login environment and modules.

A job on Michael must also specify what type of job it is (Gold, Free, Test) and the project it is being submitted for. (See Budgets and allocations below).

"},{"location":"Clusters/Michael/#memory-requests","title":"Memory requests","text":"

Note: the memory you request is always per core, not the total amount. If you ask for 128G RAM and 24 cores, that will run on 24 nodes using only one core per node. This allows you to have sparse process placement when you do actually need that much RAM per process.

"},{"location":"Clusters/Michael/#monitoring-a-job","title":"Monitoring a job","text":"

In addition to qstat, nodesforjob $JOB_ID can be useful to see what proportion of cpu/memory/swap is being used on the nodes a certain job is running on.

qexplain $JOB_ID will show you the full error for a job that is in Eqw status.

"},{"location":"Clusters/Michael/#useful-utilities","title":"Useful utilities","text":"

As well as nodesforjob, there are the following utilities which can help you find information about your jobs after they have run.

These utilities live in GitHub at https://github.com/UCL-RITS/go-clustertools and https://github.com/UCL-RITS/rcps-cluster-scripts

"},{"location":"Clusters/Michael/#software","title":"Software","text":"

Michael mounts the RC Systems software stack.

Have a look at Software Guides for specific information on running some applications, including example scripts. The list there is not exhaustive.

Access to software is managed through the use of modules.

Access to licensed software may vary based on your host institution and project.

"},{"location":"Clusters/Michael/#requesting-software-installs","title":"Requesting software installs","text":"

To request software installs, email us at the support address below or open an issue on our GitHub. You can see what software has already been requested in the Github issues and can add a comment if you're also interested in something already requested.

"},{"location":"Clusters/Michael/#installing-your-own-software","title":"Installing your own software","text":"

You may install software in your own space. Please look at Compiling for tips.

"},{"location":"Clusters/Michael/#maintaining-a-piece-of-software-for-a-group","title":"Maintaining a piece of software for a group","text":"

It is possible for people to be given central areas to install software that they wish to make available to everyone or to a select group - generally because they are the developers or if they wish to use multiple versions or developer versions. The people given install access would then be responsible for managing and maintaining these installs.

"},{"location":"Clusters/Michael/#licensed-software","title":"Licensed software","text":"

Reserved application groups exist for software that requires them. The group name will begin with leg or lg. After we add you to one of these groups, the central group change will happen overnight. You can check your groups with the groups command.

Please let us know your username when you ask to be added to a group.

"},{"location":"Clusters/Michael/#suggested-job-sizes-on-original-michael","title":"Suggested job sizes on original Michael","text":"

The target job sizes for original Michael K-type nodes are 48-120 cores (2-5 nodes). Jobs larger than this may have a longer queue time and are better suited to ARCHER, and single node jobs may be more suited to your local facilities.

"},{"location":"Clusters/Michael/#maximum-job-resources-on-original-michael","title":"Maximum job resources on original Michael","text":"Cores Max wallclock 864 48hrs

On Michael, interactive sessions using qrsh have the same wallclock limit as other jobs.

The K-type nodes in Michael are 24 cores, 128GB RAM. The default maximum jobsize is 864 cores, to remain within the 36-node 1:1 nonblocking interconnect zones.

Jobs on Michael do not share nodes. This means that if you request less than 24 cores, your job is still taking up an entire node and no other jobs can run on it, but some of the cores are idle. Whenever possible, request a number of cores that is a multiple of 24 for full usage of your nodes.

There is a superqueue for use in exceptional circumstances that will allow access to a larger number of cores outside the nonblocking interconnect zones, going across the 3:1 interconnect between blocks. A third of each CU is accessible this way, roughly approximating a 1:1 connection. Access to the superqueue for larger jobs must be applied for: contact the support address below for details.

Some normal multi-node jobs will use the superqueue - this is to make it easier for larger jobs to be scheduled, as otherwise they can have very long waits if every CU is half full.

"},{"location":"Clusters/Michael/#2020-michael-expansion","title":"2020 Michael expansion","text":"

At the end of March 2020, Michael was expanded to include a new set of nodes. The old Michael nodes are the K-type nodes, while the new ones are the A-type nodes. The node name will look like node-a14a-001 or node-k10a-001.

The Michael expansion consists of 208 compute nodes each with two 20-core Intel Xeon Gold 6248 2.5GHz processors, 192 gigabytes of 2933MHz DDR4 RAM, 1TB disk, and an Intel OmniPath network. Expansion nodes have two Hyperthreads available.

These are arranged in two 32-node CUs (a and b) and four 36-node CUs (c to f). Jobs are restricted to running either within a CU (all nodes connected to the same switch) or across CUs using only the bottom third of nodes attached to each switch. This approximates 1:1 blocking on a cluster that does not have it.

"},{"location":"Clusters/Michael/#maximum-job-resources-on-michael-expansion","title":"Maximum job resources on Michael expansion","text":"

Please consider that Michael's A-type nodes have 40 physical cores - 2 nodes is 80 cores. Jobs do not share nodes, so although asking for 41 cores is possible, it means you are wasting the other 39 cores on your second node!

Cores Max. Duration 2800 48h

These are numbers of physical cores: multiply by two for virtual cores with hyperthreads.

"},{"location":"Clusters/Michael/#hyperthreading","title":"Hyperthreading","text":"

The A-type nodes have hyperthreading enabled and you can choose on a per-job basis whether you want to use it.

Hyperthreading lets you use two virtual cores instead of one physical core - some programs can take advantage of this.

If you do not ask for hyperthreading, your job only uses one thread per core as normal.

The -l threads= request is not a true/false setting, instead you are telling the scheduler you want one slot to block one virtual cpu instead of the normal situation where it blocks two. If you have a script with a threads request and want to override it on the command line or set it back to normal, the usual case is -l threads=2. (Setting threads to 0 does not disable hyperthreading!)

# request hyperthreading in this job\n#$ -l threads=1\n\n# request the number of virtual cores\n#$ -pe mpi 160\n\n# request 2G RAM per virtual core\n#$ -l mem=2G\n\n# set number of OpenMP threads being used per MPI process\nexport OMP_NUM_THREADS=2\n

This job would be using 80 physical cores, using 80 MPI processes each of which would create two threads (on Hyperthreads).

Note that memory requests are now per virtual core with hyperthreading enabled. If you asked for #$ -l mem=4Gon a node with 80 virtual cores and 192G RAM then you are requiring 320G RAM in total which will not fit on that node and so you would be given a sparse process layout across more nodes to meet this requirement.

# request hyperthreading in this job\n#$ -l threads=1\n\n# request the number of virtual cores\n#$ -pe mpi 160\n\n# request 2G RAM per virtual core\n#$ -l mem=2G\n\n# set number of OpenMP threads being used per MPI process\n# (a whole node's worth)\nexport OMP_NUM_THREADS=80\n

This job would still be using 80 physical cores, but would use one MPI process per node which would create 80 threads on the node (on Hyperthreads).

"},{"location":"Clusters/Michael/#choosing-node-types","title":"Choosing node types","text":"

Given the difference in core count on the original and expansion Michael nodes, we strongly suggest you always specify which type of node you intend your job to run on, to avoid unintentionally wasting cores if your total number does not cleanly fit on that node size.

The old nodes are K-type while the new nodes with hyperthreading are A-type. Jobs never run across a mix of node types - it will be all K nodes or all A nodes.

To specify node type in your jobscript, add either:

# run on original 24-core nodes\n#$ -ac allow=K\n

or

# run on expansion 40-core hyperthread-enabled nodes\n#$ -ac allow=A\n
"},{"location":"Clusters/Michael/#queue-names","title":"Queue names","text":"

On Michael, users do not submit directly to queues - the scheduler assigns your job to one based on the resources it requested. The queues have somewhat unorthodox names as they are only used internally, and do not directly map to particular job types.

"},{"location":"Clusters/Michael/#preventing-a-job-from-running-cross-cu","title":"Preventing a job from running cross-CU","text":"

If your job must run within a single CU, you can request the parallel environment as -pe wss instead of -pe mpi (wss standing for 'wants single switch'). This will increase your queue times. It is suggested you only do this for benchmarking or if performance is being greatly affected by running in the superqueue.

back to top

"},{"location":"Clusters/Michael/#disk-quotas","title":"Disk quotas","text":"

You have one per-user quota, with a default amount of 250GB - this is the total across home and Scratch.

If you go over quota, you will no longer be able to create new files and your jobs will fail as they cannot write.

Quota increases may be granted without further approval, depending on size and how full the filesystem is. Otherwise they may need to go to the Thomas User Group for approval.

back to top

"},{"location":"Clusters/Michael/#budgets-and-allocations","title":"Budgets and allocations","text":"

We have enabled Gold for allocation management. Jobs that are run under a project budget have higher priority than free non-budgeted jobs. All jobs need to specify what project they belong to, whether they are paid or free.

To see the name of your project(s) and how much allocation that budget has, run the command budgets.

$ budgets  \nProject\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Machines\u00a0Balance\n--------\u00a0\u00a0\u00a0\u00a0\u00a0--------\u00a0--------\nFaraday_Test\u00a0ANY\u00a0\u00a0\u00a0\u00a0\u00a0\u00a022781.89\n
"},{"location":"Clusters/Michael/#submitting-a-job-under-a-project","title":"Submitting a job under a project","text":"

To submit a paid job that will take Gold from a particular project budget, add this to your jobscript:

#$\u00a0-P\u00a0Gold\n#$\u00a0-A\u00a0MyProject\n

To submit a free job that will not use up any Gold, use this instead:

#$\u00a0-P\u00a0Free\n#$\u00a0-A\u00a0MyProject\n

You can also submit testing jobs that will not use up any Gold, and will have higher priority than normal free jobs, but are limited to 2 nodes (48 cores) and 1 hour of walltime:

#$\u00a0-P\u00a0Test\n#$\u00a0-A\u00a0MyProject\n
"},{"location":"Clusters/Michael/#troubleshooting-unable-to-verify-membership-in-policyjsv-project","title":"Troubleshooting: Unable to verify membership in policyjsv project","text":"
Unable\u00a0to\u00a0run\u00a0job:\u00a0Rejected\u00a0by\u00a0policyjsv\nUnable\u00a0to\u00a0verify\u00a0membership\u00a0of\u00a0`<username>`\u00a0in\u00a0the\u00a0policyjsv\u00a0project\n

You asked for a Free job but didn't specify #$ -A MyProject in your jobscript.

"},{"location":"Clusters/Michael/#troubleshooting-unable-to-verify-membership-in-project-uninitialized-value","title":"Troubleshooting: Unable to verify membership in project / Uninitialized value","text":"
Unable to run job: Rejected by policyjsv  \nReason:Unable to verify sufficient material worth to submit this job:  \nUnable to verify membership of mmmxxxx in the UCL_Example project\n

This error from qsub can mean that you aren't in the project you are trying to submit to, but also happens when the Gold daemon is not running. 

Use of uninitialized value in print at /opt/gold/bin/mybalance line 60, <GBALANCE> line 1.\nFailed sending message: (Unable to connect to socket (Connection refused)).\n

If you also get this error from the budgets command, then the Gold daemon is definitely not running and you should contact rc-support.

"},{"location":"Clusters/Michael/#gold-charging","title":"Gold charging","text":"

When you submit a job, it will reserve the total number of core hours that the job script is asking for. When the job ends, the Gold will move from 'reserved' into charged. If the job doesn't run for the full time it asked for, the unused reserved portion will be refunded after the job ends. You cannot submit a job that you do not have the budget to run.

"},{"location":"Clusters/Michael/#gold-costs-of-a-type-nodes","title":"Gold costs of A-type nodes","text":"

The A-type nodes have twice the peak theoretical performance of the K-type nodes. A 24-core job lasting an hour costs 24 Gold on the K-type nodes. A 40-physical-core job lasting one hour costs 80 Gold on the A-type nodes. An 80-virtual-core job on the A-type nodes also costs 80 Gold.

"},{"location":"Clusters/Michael/#troubleshooting-unable-to-verify-sufficient-material-worth","title":"Troubleshooting: Unable to verify sufficient material worth","text":"
Unable\u00a0to\u00a0run\u00a0job:\u00a0Rejected\u00a0by\u00a0policyjsv\nReason:Unable\u00a0to\u00a0verify\u00a0sufficient\u00a0material\u00a0worth\u00a0to\u00a0submit\u00a0this\u00a0job:\nInsufficient\u00a0balance\u00a0to\u00a0reserve\u00a0job\n

This means you don't have enough Gold to cover the cores*wallclock time cost of the job you are trying to submit. You need to wait for queued jobs to finish and return unused Gold to your project, or submit a smaller/shorter job. Note that array jobs have to cover the whole cost of all the tasks at submit time.

"},{"location":"Clusters/Michael/#job-deletion","title":"Job deletion","text":"

If you qdel a submitted Gold job, the reserved Gold will be made available again. This is done by a cron job that runs every 15 minutes, so you may not see it back instantly.

"},{"location":"Clusters/Michael/#reporting-gold-usage","title":"Reporting Gold usage","text":"

There are a few commands that everyone can run that report Gold usage for their entire project, broken down by user. See Reporting from Gold.

"},{"location":"Clusters/Michael/#support","title":"Support","text":"

Email rc-support@ucl.ac.uk with any support queries. It will be helpful to include Michael in the subject along with some descriptive text about the type of problem, and you should mention your username in the body.

"},{"location":"Clusters/Myriad/","title":"Myriad","text":"

Myriad is designed for high I/O, high throughput jobs that will run within a single node rather than multi-node parallel jobs.

"},{"location":"Clusters/Myriad/#accounts","title":"Accounts","text":"

Myriad accounts can be applied for via the Research Computing sign up process.

As Myriad is our most general-purpose system, everyone who signs up for a Research Computing account is given access to Myriad.

"},{"location":"Clusters/Myriad/#logging-in","title":"Logging in","text":"

You will use your UCL username and password to ssh in to Myriad.

ssh\u00a0uccaxxx@myriad.rc.ucl.ac.uk\n

If using PuTTY, put myriad.rc.ucl.ac.uk as the hostname and your seven-character username (with no @ after) as the username when logging in, eg. uccaxxx. When entering your password in PuTTY no characters or bulletpoints will show on screen - this is normal.

If you are outside the UCL firewall you will need to follow the instructions for Logging in from outside the UCL firewall.

"},{"location":"Clusters/Myriad/#logging-in-to-a-specific-node","title":"Logging in to a specific node","text":"

You can access a specific Myriad login node with: 

ssh\u00a0uccaxxx@login12.myriad.rc.ucl.ac.uk\nssh\u00a0uccaxxx@login13.myriad.rc.ucl.ac.uk\n

The main address will redirect you on to either one of them.

"},{"location":"Clusters/Myriad/#copying-data-onto-myriad","title":"Copying data onto Myriad","text":"

You will need to use an SCP or SFTP client to copy data onto Myriad. Please refer to the page on How do I transfer data onto the system?

"},{"location":"Clusters/Myriad/#quotas","title":"Quotas","text":"

The default quotas on Myriad are 150GB for home and 1TB for Scratch.

These are hard quotas: once you reach them, you will no longer be able to write more data. Keep an eye on them, as this will cause jobs to fail if they cannot create their .o or .e files at the start, or their output files partway through.

You can check both quotas on Myriad by running: 

lquota\n

which will give you output similar to this:

     Storage        Used        Quota   % Used   Path\n        home  721.68 MiB   150.00 GiB       0%   /home/uccaxxx\n     scratch   52.09 MiB     1.00 TiB       0%   /scratch/scratch/uccaxxx\n

You can apply for quota increases using the form at Additional Resource Requests.

Here are some tips for managing your quota and finding where space is being used.

"},{"location":"Clusters/Myriad/#job-sizes","title":"Job sizes","text":"Cores Max wallclock 1 72hrs 2 to 36 48hrs

Interactive jobs run with qrsh have the same maximum wallclock time as other jobs.

"},{"location":"Clusters/Myriad/#node-types","title":"Node types","text":"

Myriad contains three main node types: standard compute nodes, high memory nodes and GPU nodes. As new nodes as added over time with slightly newer processor variants, new letters are added.

Type Cores per node RAM per node tmpfs Nodes H,D 36 192GB 1500G 342 I,B 36 1.5TB 1500G 17 J 36 + 2 P100 GPUs 192GB 1500G 2 E,F 36 + 2 V100 GPUs 192GB 1500G 19 L 36 + 4 A100 GPUs 192GB 1500G 6

You can tell the type of a node by its name: type H nodes are named node-h00a-001 etc.

Here are the processors each node type has:

(If you ever need to check this, you can include cat /proc/cpuinfo in your jobscript so you get it in your job's .o file for the exact node your job ran on. You will get an entry for every core).

"},{"location":"Clusters/Myriad/#gpus","title":"GPUs","text":"

Myriad has four types of GPU nodes: E, F, J and L.

You can include nvidia-smi in your jobscript to get information about the GPU your job ran on.

"},{"location":"Clusters/Myriad/#compute-capability","title":"Compute Capability","text":"

Compute Capability is how NVIDIA categorises its generations of GPU architectures. When code is compiled, it targets one or multiple of these and so it may only be able to run on GPUs of a specific Compute Capability.

If you get an error like this:

CUDA runtime implicit initialization on GPU:0 failed. Status: device kernel image is invalid\n

then the software you are running does not support the Compute Capability of the GPU you tried to run it on, and you probably need a newer version.

"},{"location":"Clusters/Myriad/#requesting-multiple-and-specific-types-of-gpu","title":"Requesting multiple and specific types of GPU","text":"

You can request a number of GPUs by adding them as a resource request to your jobscript: 

# For 1 GPU\n#$\u00a0-l\u00a0gpu=1\n\n# For 2 GPUs\n#$ -l gpu=2\n\n# For 4 GPUs\n#$ -l gpu=4\n

If you ask for one or two GPUs your job can run on any type of GPU since it can fit on any of the nodetypes. If you ask for four, it can only be a node that has four. If you need to specify one node type over the others because you need a particular Compute Capability, add a request for that type of node to your jobscript:

# request a V100 node only\n#$ -ac allow=EF\n\n# request an A100 node only\n#$ -ac allow=L\n

The GPU nodes page has some sample code for running GPU jobs if you need a test example.

"},{"location":"Clusters/Myriad/#tensorflow","title":"Tensorflow","text":"

Tensorflow is installed: type module avail tensorflow to see the available versions.

Modules to load for the non-MKL GPU version: 

module\u00a0load\u00a0python3/3.7\nmodule\u00a0load\u00a0cuda/10.0.130/gnu-4.9.2\nmodule\u00a0load\u00a0cudnn/7.4.2.24/cuda-10.0\nmodule\u00a0load\u00a0tensorflow/2.0.0/gpu-py37\n

Modules to load the most recent version we have installed with GPU support (2.11.0):

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load python/3.9.6-gnu-10.2.0\nmodule load cuda/11.2.0/gnu-10.2.0\nmodule load cudnn/8.1.0.77/cuda-11.2\nmodule load tensorflow/2.11.0/gpu\n
"},{"location":"Clusters/Myriad/#pytorch","title":"PyTorch","text":"

PyTorch is installed: type module avail pytorch to see the versions available.

Modules to load the most recent release we have installed (May 2022) are:

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load python3/3.9-gnu-10.2.0\nmodule load cuda/11.3.1/gnu-10.2.0\nmodule load cudnn/8.2.1.32/cuda-11.3\nmodule load pytorch/1.11.0/gpu\n

If you want the CPU only version then use:

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load python3/3.9-gnu-10.2.0\nmodule load pytorch/1.11.0/cpu\n
"},{"location":"Clusters/Thomas/","title":"MMM Thomas","text":"

Thomas is the UK National Tier 2 High Performance Computing Hub in Materials and Molecular Modelling.

Thomas final retirement

The Thomas cluster has been retired. Thomas' filesystem was getting too old and beginning to fail, so we had to retire Thomas. Notable dates are:

"},{"location":"Clusters/Thomas/#original-thomas-retirement-dates","title":"Original Thomas retirement dates:","text":"

The Thomas cluster is being retired. Notable dates are:

Thomas has now been retired as an MMM Hub machine. A portion of Thomas is running for UCL users only until the end of 2021.

7 June 2021: Thomas is now inside the UCL firewall, use VPN, or SSH via ssh-gateway.ucl.ac.uk to access.

"},{"location":"Clusters/Thomas/#thomas-technical-specs","title":"Thomas technical specs","text":"

Thomas originally consisted of 720 Lenovo Intel x86-64 nodes, giving 17.2k cores in total, with Intel OmniPath interconnect (1:1 nonblocking in 36 node blocks, 3:1 between blocks and across the system).

Each node has the following specs:

"},{"location":"Clusters/Thomas/#applying-for-an-account","title":"Applying for an account","text":"

UCL users can contact David Scanlon or Scott Woodley for access to the post-retirement portion of Thomas.

Thomas accounts belong to you as an individual and are applied for through your own institution's Point of Contact. You will need to supply an SSH public key, which is the only method used to log in.

"},{"location":"Clusters/Thomas/#creating-an-ssh-key-pair","title":"Creating an ssh key pair","text":"

An ssh key consists of a public and a private part, typically named id_rsa and id_rsa.pub by default. The public part is what we need. You must not share your private key with anyone else. You can copy it onto multiple machines belonging to you so you can log in from all of them (or you can have a separate pair for each machine).

"},{"location":"Clusters/Thomas/#creating-an-ssh-key-in-linuxunixmac-os-x","title":"Creating an ssh key in Linux/Unix/Mac OS X","text":"
ssh-keygen\u00a0-t\u00a0rsa\n

The defaults should give you a reasonable key. If you prefer to use ed25519 instead, and/or longer keys, you can. You can also tell it to create one with a different name, so it doesn't overwrite any existing key.

You will be asked to add a passphrase for your key. A blank passphrase is not recommended; if you use one please make sure that no one else ever has access to your local computer account. How often you are asked for a passphrase depends on how long your local ssh agent keeps it.

You may need to run ssh-add to add the key to your agent so you can use it. If you aren't sure what keys your agent can see, running ssh-add -L will show all the public parts of the keys it is aware of.

"},{"location":"Clusters/Thomas/#creating-an-ssh-key-in-windows","title":"Creating an ssh key in Windows","text":"

Have a look at Key-Based SSH Logins With PuTTY which has step-by-step instructions. You can choose whether to use Pageant or not to manage your key. You can again pick RSA, ED25519, ECDSA etc but do not pick SSH-1 as that is a very old and insecure key type. As above, DSA is no longer accepted. The key must be at least 2048-bit.

If you are using Windows 10, then you probably have OpenSSH installed and could instead run ssh-keygen in a terminal per the Linux instructions and use the ssh command to log in instead of PuTTY.

"},{"location":"Clusters/Thomas/#information-for-points-of-contact","title":"Information for Points of Contact","text":"

Points of Contact have some tools they can use to manage users and allocations, documented at MMM Points of Contact.

"},{"location":"Clusters/Thomas/#logging-in","title":"Logging in","text":"

You will be assigned a personal username and your SSH key pair will be used to log in. External users will have a username in the form mmmxxxx (where xxxx is a number) and UCL users will use their central username.

You ssh directly to:

thomas.rc.ucl.ac.uk\n

From inside the UCL VPN you can ssh directly into the above but from outside you can do the following:

ssh -o ProxyJump=<UCL_user_ID>@ssh-gateway.ucl.ac.uk <thomas_user_ID>@thomas.rc.ucl.ac.uk\n
"},{"location":"Clusters/Thomas/#ssh-timeouts","title":"SSH timeouts","text":"

Idle ssh sessions will be disconnected after 7 days.

"},{"location":"Clusters/Thomas/#using-the-system","title":"Using the system","text":"

Thomas is a batch system. The login nodes allow you to manage your files, compile code and submit jobs. Very short (\\<15mins) and non-resource-intensive software tests can be run on the login nodes, but anything more should be submitted as a job.

"},{"location":"Clusters/Thomas/#full-user-guide","title":"Full user guide","text":"

Thomas has the same user environment as RC Support's other clusters, so the User guide is relevant and is a good starting point for further information about how the environment works. Any variations that Thomas has should be listed on this page.

"},{"location":"Clusters/Thomas/#submitting-a-job","title":"Submitting a job","text":"

Create a jobscript for non-interactive use and submit your jobscript using qsub. Jobscripts must begin #!/bin/bash -l in order to run as a login shell and get your login environment and modules.

Jobs no longer need to specify what type of job it is (Gold, Free, Test) or the project it is being submitted for. (See Budgets and allocations below.)

"},{"location":"Clusters/Thomas/#memory-requests","title":"Memory requests","text":"

Note: the memory you request is always per core, not the total amount. If you ask for 128GB RAM and 24 cores, that will run on 24 nodes using only one core per node. This allows you to have sparse process placement when you do actually need that much RAM per process.

"},{"location":"Clusters/Thomas/#monitoring-a-job","title":"Monitoring a job","text":"

In addition to qstat, nodesforjob $JOB_ID can be useful to see what proportion of cpu/memory/swap is being used on the nodes a certain job is running on.

qexplain $JOB_ID will show you the full error for a job that is in Eqw status.

"},{"location":"Clusters/Thomas/#useful-utilities","title":"Useful utilities","text":"

As well as nodesforjob, there are the following utilities which can help you find information about your jobs after they have run.

These utilities live in GitHub at https://github.com/UCL-RITS/go-clustertools and https://github.com/UCL-RITS/rcps-cluster-scripts

"},{"location":"Clusters/Thomas/#software","title":"Software","text":"

Thomas mounts the RC Systems software stack.

Have a look at Software Guides for specific information on running some applications, including example scripts. The list there is not exhaustive.

Access to software is managed through the use of modules.

Access to licensed software may vary based on your host institution and project.

"},{"location":"Clusters/Thomas/#requesting-software-installs","title":"Requesting software installs","text":"

To request software installs, email us at the support address below or open an issue on our GitHub. You can see what software has already been requested in the Github issues and can add a comment if you're also interested in something already requested.

"},{"location":"Clusters/Thomas/#installing-your-own-software","title":"Installing your own software","text":"

You may install software in your own space. Please look at Compiling for tips.

"},{"location":"Clusters/Thomas/#maintaining-a-piece-of-software-for-a-group","title":"Maintaining a piece of software for a group","text":"

It is possible for people to be given central areas to install software that they wish to make available to everyone or to a select group - generally because they are the developers or if they wish to use multiple versions or developer versions. The people given install access would then be responsible for managing and maintaining these installs.

"},{"location":"Clusters/Thomas/#licensed-software","title":"Licensed software","text":"

Reserved application groups exist for software that requires them. The group name will begin with leg or lg. After we add you to one of these groups, the central group change will happen overnight. You can check your groups with the groups command.

Please let us know your username when you ask to be added to a group.

"},{"location":"Clusters/Thomas/#suggested-job-sizes","title":"Suggested job sizes","text":"

The target job sizes for Thomas are 48-120 cores (2-5 nodes). Jobs larger than this may have a longer queue time and are better suited to ARCHER, and single node jobs may be more suited to your local facilities.

"},{"location":"Clusters/Thomas/#maximum-job-resources","title":"Maximum job resources","text":"Cores Max wallclock 864 48hrs

On Thomas, interactive sessions using qrsh have the same wallclock limit as other jobs.

Nodes in Thomas are 24 cores, 128G RAM. The default maximum jobsize is 864 cores, to remain within the 36-node 1:1 nonblocking interconnect zones.

Jobs on Thomas do not share nodes. This means that if you request less than 24 cores, your job is still taking up an entire node and no other jobs can run on it, but some of the cores are idle. Whenever possible, request a number of cores that is a multiple of 24 for full usage of your nodes.

There is a superqueue for use in exceptional circumstances that will allow access to a larger number of cores outside the nonblocking interconnect zones, going across the 3:1 interconnect between blocks. A third of each CU is accessible this way, roughly approximating a 1:1 connection. Access to the superqueue for larger jobs must be applied for: contact the support address below for details.

Some normal multi-node jobs will use the superqueue - this is to make it easier for larger jobs to be scheduled, as otherwise they can have very long waits if every CU is half full.

"},{"location":"Clusters/Thomas/#queue-names","title":"Queue names","text":"

On Thomas, users do not submit directly to queues - the scheduler assigns your job to one based on the resources it requested. The queues have somewhat unorthodox names as they are only used internally, and do not directly map to particular job types.

"},{"location":"Clusters/Thomas/#preventing-a-job-from-running-cross-cu","title":"Preventing a job from running cross-CU","text":"

If your job must run within a single CU, you can request the parallel environment as -pe wss instead of -pe mpi (wss standing for 'wants single switch'). This will increase your queue times. It is suggested you only do this for benchmarking or if performance is being greatly affected by running in the superqueue.

back to top

"},{"location":"Clusters/Thomas/#node-types","title":"Node types","text":"

Thomas has one type of node.

Type Cores per node RAM per node tmpfs Nodes K 24 128G 100G 705

Here are the processors each node type has:

(If you ever need to check this, you can include cat /proc/cpuinfo in your jobscript so you get it in your job's .o file for the exact node your job ran on. You will get an entry for every core).

"},{"location":"Clusters/Thomas/#disk-quotas","title":"Disk quotas","text":"

Quotas were enabled on Thomas on 29 May 2019. You have one per-user quota, with a default amount of 250GB - this is the total across home and Scratch.

If you go over quota, you will no longer be able to create new files and your jobs will fail as they cannot write.

Quota increases may be granted without further approval, depending on size and how full the filesystem is. Otherwise they may need to go to the Thomas User Group for approval.

back to top

"},{"location":"Clusters/Thomas/#budgets-and-allocations","title":"Budgets and allocations","text":"

There is no more Gold on Thomas. Jobs should now be submitted without specifying any -P or -A entries in your jobscript.

Info

1 Gold unit is 1 hour of using 1 processor core.

"},{"location":"Clusters/Thomas/#troubleshooting-unable-to-verify-membership-in-project-uninitialized-value","title":"Troubleshooting: Unable to verify membership in project / Uninitialized value","text":"
Unable to run job: Rejected by policyjsv \nReason:Unable to verify sufficient material worth to submit this job: \nUnable to verify membership of mmmxxxx in the UCL_Example project\n

This error from qsub can mean that you aren't in the project you are trying to submit to, but also happens when the Gold daemon is not running.

Remove the -P and -A lines from your jobscript and submit jobs without any project specified.

Use of uninitialized value in print at /opt/gold/bin/mybalance line 60, <GBALANCE> line 1.\nFailed sending message: (Unable to connect to socket (Connection refused)).\n

The Gold database is uncontactable.

Remove the -P and -A lines from your jobscript and submit jobs without any project specified.

"},{"location":"Clusters/Thomas/#the-tier-2-safe","title":"The Tier 2 SAFE","text":"

SAFE is a service administration platform used by ARCHER and various of the Tier 2 sites. As a user, you can use it to do some management of your details and view your usage across all systems that send data to SAFE.

"},{"location":"Clusters/Thomas/#see-your-usage-data-in-safe","title":"See your usage data in SAFE","text":"

If you wish, you can claim your Thomas account as belonging to you in the Tier 2 SAFE. This lets you view some individual usage reports, and if you have other Tier 2 accounts that also use SAFE, you use the same login information.

If you do not claim your account, then SAFE only contains username/job usage information with no information about who owns that username. Points of Contact can use it to allocate Gold to your budgets.

"},{"location":"Clusters/Thomas/#update-your-ssh-key-via-safe","title":"Update your SSH key via SAFE","text":"

SAFE also gives you a second mechanism for changing your SSH key - changing the key you have associated with your SAFE account will automatically create a request for us to add that key to your Thomas account. (We still need to act on it, so it won't update immediately).

"},{"location":"Clusters/Thomas/#support","title":"Support","text":"

Email rc-support@ucl.ac.uk with any support queries. It will be helpful to include Thomas in the subject along with some descriptive text about the type of problem, and you should mention your username in the body.

"},{"location":"Clusters/Thomas/#notional-costs","title":"Notional costs","text":"

EPSRC contributed to the hardware, so there are two numbers to use for notional costs:

Non-grant-funded projects should use the second figure.

"},{"location":"Clusters/Thomas/#acknowledging-the-use-of-thomas-in-publications","title":"Acknowledging the use of Thomas in publications","text":"

All work arising from this facility should be properly acknowledged in presentations and papers with the following text:

\"We are grateful to the UK Materials and Molecular Modelling Hub for computational resources, which is partially funded by EPSRC (EP/P020194/1 and EP/T022213/1)\"

"},{"location":"Clusters/Thomas/#mcc","title":"MCC","text":"

When publishing work that benefited from resources allocated by the MCC: please include the following acknowledgment:

\"Via our membership of the UK's HEC Materials Chemistry Consortium, which is funded by EPSRC (EP/L000202), this work used the UK Materials and Molecular Modelling Hub for computational resources, MMM Hub, which is partially funded by EPSRC (EP/P020194 and EP/T022213)\"

"},{"location":"Clusters/Thomas/#ukcp","title":"UKCP","text":"

When publishing work that benefited from resources allocated by UKCP, please include:

\"We are grateful for computational support from the UK Materials and Molecular Modelling Hub, which is partially funded by EPSRC (EP/P020194 and EP/T022213), for which access was obtained via the UKCP consortium and funded by EPSRC grant ref EP/P022561/1\"

"},{"location":"Clusters/Young/","title":"MMM Hub Young","text":"

Young is the UK National Tier 2 High Performance Computing Hub in Materials and Molecular Modelling, and replacement for Thomas. Young went into pilot on 3 Aug 2020.

"},{"location":"Clusters/Young/#training-resources","title":"Training resources","text":"

The MMM Hub's Events and Training page contains useful information for new users and about specific software.

In particular the \"Software Training\" section:

"},{"location":"Clusters/Young/#mmm-hub-hpe-nvidia-gpu-training-day","title":"MMM Hub: HPE / NVIDIA GPU Training Day","text":"

The MMM Hub: HPE / NVIDIA GPU Training Day took place on 31 March 2022 and recordings of the talks are available at the link below. The day was split into two parts, \"Technology and Partner sessions (UCL, HPE & NVIDIA)\" which gave an overview of the new GPU nodes being added to Young and tools useful for GPU programming, followed by \"Materials community codes, experiences & lessons learned (invited speakers)\" which looked at how CASTEP, VASP, CP2K, GSGW and QMCpack had approached adding GPU support.

"},{"location":"Clusters/Young/#applying-for-an-account","title":"Applying for an account","text":"

Young accounts belong to you as an individual and are applied for through your own institution's Point of Contact. You will need to supply an SSH public key, which is the only method used to log in.

"},{"location":"Clusters/Young/#creating-an-ssh-key-pair","title":"Creating an ssh key pair","text":"

An ssh key consists of a public and a private part, typically named id_rsa and id_rsa.pub by default. The public part is what we need. You must not share your private key with anyone else. You can copy it onto multiple machines belonging to you so you can log in from all of them (or you can have a separate pair for each machine).

"},{"location":"Clusters/Young/#creating-an-ssh-key-in-linuxunixmac-os-x","title":"Creating an ssh key in Linux/Unix/Mac OS X","text":"
ssh-keygen\u00a0-t\u00a0rsa\n

The defaults should give you a reasonable key. If you prefer to use ed25519 instead, and/or longer keys, you can. You can also tell it to create one with a different name, so it doesn't overwrite any existing key.

You will be asked to add a passphrase for your key. A blank passphrase is not recommended; if you use one please make sure that no one else ever has access to your local computer account. How often you are asked for a passphrase depends on how long your local ssh agent keeps it.

You may need to run ssh-add to add the key to your agent so you can use it. If you aren't sure what keys your agent can see, running ssh-add -L will show all the public parts of the keys it is aware of.

"},{"location":"Clusters/Young/#creating-an-ssh-key-in-windows","title":"Creating an ssh key in Windows","text":"

Have a look at Key-Based SSH Logins With PuTTY which has step-by-step instructions. You can choose whether to use Pageant or not to manage your key. You can again pick RSA, ED25519, ECDSA etc but do not pick SSH-1 as that is a very old and insecure key type. As above, DSA is no longer accepted. The key must be at least 2048-bit.

If you are using Windows 10, then you probably have OpenSSH installed and could instead run ssh-keygen in a terminal per the Linux instructions and use the ssh command to log in instead of PuTTY.

"},{"location":"Clusters/Young/#information-for-points-of-contact","title":"Information for Points of Contact","text":"

Points of Contact have some tools they can use to manage users and allocations, documented at MMM Points of Contact.

"},{"location":"Clusters/Young/#logging-in","title":"Logging in","text":"

You will be assigned a personal username and your SSH key pair will be used to log in. External users will have a username in the form mmmxxxx (where xxxx is a number) and UCL users will use their central username.

You ssh directly to:

young.rc.ucl.ac.uk\n

Young has two login nodes and you will be round-robin assigned to one or the other each time. You can also ssh directly into a specific login node, useful if you had a tmux or screen session running there.

# replace 'mmmxxxx' with your username\nssh mmmxxxx@login01.young.rc.ucl.ac.uk\n# or\nssh mmmxxxx@login02.young.rc.ucl.ac.uk\n
"},{"location":"Clusters/Young/#ssh-timeouts","title":"SSH timeouts","text":"

Idle ssh sessions will be disconnected after 7 days.

"},{"location":"Clusters/Young/#using-the-system","title":"Using the system","text":"

Young is a batch system. The login nodes allow you to manage your files, compile code and submit jobs. Very short (\\<15mins) and non-resource-intensive software tests can be run on the login nodes, but anything more should be submitted as a job.

"},{"location":"Clusters/Young/#full-user-guide","title":"Full user guide","text":"

Young has the same user environment as RC Support's other clusters, so the User guide is relevant and is a good starting point for further information about how the environment works. Any variations that Young has should be listed on this page.

"},{"location":"Clusters/Young/#submitting-a-job","title":"Submitting a job","text":"

Create a jobscript for non-interactive use and submit your jobscript using qsub. Jobscripts must begin #!/bin/bash -l in order to run as a login shell and get your login environment and modules.

A job on Young must also specify what type of job it is (Gold, Free, Test) and the project it is being submitted for. (See Budgets and allocations below.)

"},{"location":"Clusters/Young/#memory-requests","title":"Memory requests","text":"

Note: the memory you request is always per core, not the total amount. If you ask for 192GB RAM and 40 cores, that may run on 40 nodes using only one core per node. This allows you to have sparse process placement when you do actually need that much RAM per process.

Young also has high memory nodes, where a job like this may run.

If you want to avoid sparse process placement and your job taking up more nodes than you were expecting, the maximum memory request you can make when using all the cores in a standard node is 4.6G.

"},{"location":"Clusters/Young/#monitoring-a-job","title":"Monitoring a job","text":"

In addition to qstat, nodesforjob $JOB_ID can be useful to see what proportion of cpu/memory/swap is being used on the nodes a certain job is running on.

qexplain $JOB_ID will show you the full error for a job that is in Eqw status.

"},{"location":"Clusters/Young/#useful-utilities","title":"Useful utilities","text":"

As well as nodesforjob, there are the following utilities which can help you find information about your jobs after they have run.

These utilities live in GitHub at https://github.com/UCL-RITS/go-clustertools and https://github.com/UCL-RITS/rcps-cluster-scripts

"},{"location":"Clusters/Young/#known-problems","title":"Known problems","text":""},{"location":"Clusters/Young/#slowness-in-vim","title":"Slowness in vim","text":"

Use of vim has frequent lags which can make editing (or viewing) files in it annoying. This is caused by how often vim autosaves status (every few seconds by default) which causes a lot of metadata accesses. It might include saving things like the current cursor position into .viminfo). When Young's filesystem is being particularly slow, this can make use of vim impossible.

You can turn off autosaves entirely:

:set noswapfile\n

or you can set the autosave frequency in milliseconds:

:set updatetime=\n

You can save these in your .vimrc.

"},{"location":"Clusters/Young/#software","title":"Software","text":"

Young mounts the RC Systems software stack.

Have a look at Software Guides for specific information on running some applications, including example scripts. The list there is not exhaustive.

Access to software is managed through the use of modules.

Access to licensed software may vary based on your host institution and project.

"},{"location":"Clusters/Young/#loading-and-unloading-modules","title":"Loading and unloading modules","text":"

Young has a newer version of modulecmd which tries to manage module dependencies automatically by loading or unloading prerequisites for you whenever possible.

If you get an error like this:

[uccaxxx@login01 ~]$ module unload compilers mpi\nUnloading compilers/intel/2018/update3\n  ERROR: compilers/intel/2018/update3 cannot be unloaded due to a prereq.\n    HINT: Might try \"module unload default-modules/2018\" first.\n\nUnloading mpi/intel/2018/update3/intel\n  ERROR: mpi/intel/2018/update3/intel cannot be unloaded due to a prereq.\n    HINT: Might try \"module unload default-modules/2018\" first.\n

You can use the -f option to force the module change. It will carry it out and warn you about modules it thinks are dependent.

[uccaxxx@login01 ~]$ module unload -f compilers mpi\nUnloading compilers/intel/2018/update3\n  WARNING: Dependent default-modules/2018 is loaded\n\nUnloading mpi/intel/2018/update3/intel\n  WARNING: Dependent default-modules/2018 is loaded\n
"},{"location":"Clusters/Young/#requesting-software-installs","title":"Requesting software installs","text":"

To request software installs, email us at the support address below or open an issue on our GitHub. You can see what software has already been requested in the Github issues and can add a comment if you're also interested in something already requested.

"},{"location":"Clusters/Young/#installing-your-own-software","title":"Installing your own software","text":"

You may install software in your own space. Please look at Compiling for tips.

"},{"location":"Clusters/Young/#maintaining-a-piece-of-software-for-a-group","title":"Maintaining a piece of software for a group","text":"

It is possible for people to be given central areas to install software that they wish to make available to everyone or to a select group - generally because they are the developers or if they wish to use multiple versions or developer versions. The people given install access would then be responsible for managing and maintaining these installs.

"},{"location":"Clusters/Young/#licensed-software","title":"Licensed software","text":"

Reserved application groups exist for software that requires them. The group name will begin with leg or lg. After we add you to one of these groups, the central group change will happen overnight. You can check your groups with the groups command.

Please let us know your username when you ask to be added to a group.

"},{"location":"Clusters/Young/#suggested-job-sizes","title":"Suggested job sizes","text":"

The target job sizes for Young are 2-5 nodes. Jobs larger than this may have a longer queue time and are better suited to ARCHER, and single node jobs may be more suited to your local facilities.

"},{"location":"Clusters/Young/#maximum-job-resources","title":"Maximum job resources","text":"Job type Cores GPUs Max wallclock Gold CPU job, any 5120 0 48hrs Free CPU job, any 5120 0 24hrs Free GPU job, any 320 40 48hrs Free GPU fast interactive 64 8 6hrs

CPU jobs or GPU jobs can be run on Young, and there are different nodes dedicated for each.

These are numbers of physical cores: multiply by two for virtual cores with hyperthreads on the CPU nodes.

On Young, interactive sessions using qrsh have the same wallclock limit as other jobs.

CPU jobs on Young do not share nodes, whereas GPU jobs do. This means that if you request less than 40 cores for a CPU job, your job is still taking up an entire node and no other jobs can run on it, but some of the cores are idle. Whenever possible, request a number of cores that is a multiple of 40 for full usage of your CPU nodes.

There is a superqueue for use in exceptional circumstances that will allow access to a larger number of cores outside the nonblocking interconnect zones, going across the interconnect between blocks. A third of each CU is accessible this way, roughly approximating a 1:1 connection. Access to the superqueue for larger jobs must be applied for: contact the support address below for details.

Some normal multi-node jobs will use the superqueue - this is to make it easier for larger jobs to be scheduled, as otherwise they can have very long waits if every CU is half full.

"},{"location":"Clusters/Young/#preventing-a-job-from-running-cross-cu","title":"Preventing a job from running cross-CU","text":"

If your job must run within a single CU, you can request the parallel environment as -pe wss instead of -pe mpi (wss standing for 'wants single switch'). This will increase your queue times. It is suggested you only do this for benchmarking or if performance is being greatly affected by running in the superqueue.

back to top

"},{"location":"Clusters/Young/#node-types","title":"Node types","text":"

Young has four types of node: standard nodes, big memory nodes, really big memory nodes and GPU nodes. Note those last two have different processors and number of CPU cores per node.

Type Cores per node RAM per node tmpfs Nodes Memory request necessary GPU C 40 192G None 576 Any None Y 40 1.5T None 3 mpi: mem >=19G, smp: >186G total None Z 36 3.0T None 3 mpi: mem >=42G, smp: >1530G total None X 64 1T 200G 6 Any 8 x Nvidia 40G A100

These are numbers of physical cores: multiply by two for virtual cores with hyperthreading.

The 'memory request necessary' column shows what memory requests a job needs to make to be eligible for that node type. For MPI jobs it looks at the memory per slot requested. For SMP jobs they will go on the node that their total memory request (slots * mem) fits on.

Here are the processors each node type has:

(If you ever need to check this, you can include cat /proc/cpuinfo in your jobscript so you get it in your job's .o file for the exact node your job ran on. You will get an entry for every core).

"},{"location":"Clusters/Young/#gpu-nodes","title":"GPU nodes","text":"

Now available for general use, for Free jobs only. There will be separate GPU Gold budgets in future.

How to use the GPU nodes.

"},{"location":"Clusters/Young/#restricting-to-one-node-type","title":"Restricting to one node type","text":"

The scheduler will schedule your job on the relevant nodetype based on the resources you request, but if you really need to specify the nodetype yourself, use:

# Only run on Z-type nodes\n#$ -ac allow=Z\n
"},{"location":"Clusters/Young/#hyperthreading","title":"Hyperthreading","text":"

Young has hyperthreading enabled and you can choose on a per-job basis whether you want to use it.

Hyperthreading lets you use two virtual cores instead of one physical core - some programs can take advantage of this.

If you do not ask for hyperthreading, your job only uses one thread per core as normal.

The -l threads= request is not a true/false setting, instead you are telling the scheduler you want one slot to block one virtual cpu instead of the normal situation where it blocks two. If you have a script with a threads request and want to override it on the command line or set it back to normal, the usual case is -l threads=2. (Setting threads to 0 does not disable hyperthreading!)

# request hyperthreading in this job\n#$ -l threads=1\n\n# request the number of virtual cores\n#$ -pe mpi 160\n\n# request 2G RAM per virtual core\n#$ -l mem=2G\n\n# set number of OpenMP threads being used per MPI process\nexport OMP_NUM_THREADS=2\n

This job would be using 80 physical cores, using 80 MPI processes each of which would create two threads (on Hyperthreads).

Note that memory requests are now per virtual core with hyperthreading enabled. If you asked for #$ -l mem=4Gon a node with 80 virtual cores and 192G RAM then you are requiring 320G RAM in total which will not fit on that node and so you would be given a sparse process layout across more nodes to meet this requirement.

# request hyperthreading in this job\n#$ -l threads=1\n\n# request the number of virtual cores\n#$ -pe mpi 160\n\n# request 2G RAM per virtual core\n#$ -l mem=2G\n\n# set number of OpenMP threads being used per MPI process\n# (a whole node's worth)\nexport OMP_NUM_THREADS=80\n

This job would still be using 80 physical cores, but would use one MPI process per node which would create 80 threads on the node (on Hyperthreads).

"},{"location":"Clusters/Young/#diskless-nodes","title":"Diskless nodes","text":"

Young nodes are diskless (have no local hard drives) - there is no $TMPDIR available, so you should not request -l tmpfs=10G in your jobscripts or your job will be rejected at submit time.

If you need temporary space, you should use somewhere in your Scratch.

"},{"location":"Clusters/Young/#disk-quotas","title":"Disk quotas","text":"

You have one per-user quota, with a default amount of 250GB - this is the total across home and Scratch.

If you go over quota, you will no longer be able to create new files and your jobs will fail as they cannot write.

Quota increases may be granted without further approval, depending on size and how full the filesystem is. Otherwise they may need to go to the MMM Hub User Group for approval.

back to top

"},{"location":"Clusters/Young/#budgets-and-allocations","title":"Budgets and allocations","text":"

We have enabled Gold for allocation management. Jobs that are run under a project budget have higher priority than free non-budgeted jobs. All jobs need to specify what project they belong to, whether they are paid or free.

To see the name of your project(s) and how much allocation that budget has, run the command budgets.

budgets  \nProject\u00a0\u00a0Machines\u00a0Balance\u00a0  \n--------\u00a0--------\u00a0--------  \nUCL_Test\u00a0ANY\u00a0\u00a0\u00a0\u00a0\u00a0\u00a022781.89\n

Pilot users temporarily have access to a project for their institution, eg. Imperial_pilot. These will be deactivated after the pilot and no longer show up.

Info

1 Gold unit is 1 hour of using 1 virtual processor core (= 0.5 physical core).

Since Young has hyperthreading, a job asking for 40 physical cores and one asking for 80 virtual cores with hyperthreading on both cost the same amount: 80 Gold.

"},{"location":"Clusters/Young/#subprojects","title":"Subprojects","text":"

You might be in a subproject that does not itself have an allocation, but instead takes allocation from a different project:

Project\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Machines\u00a0Balance\n--------\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0--------\u00a0--------\nUCL_physM\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0ANY\u00a0\u00a0\u00a0474999.70  \nUCL_physM_Bowler\u00a0ANY\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a00.00\n

In this case, you submit jobs using the subproject (UCL_physM_Bowler here) even though it says it has 0 budget and it takes Gold from the superproject.

"},{"location":"Clusters/Young/#submitting-a-job-under-a-project","title":"Submitting a job under a project","text":"

To submit a paid job that will take Gold from a particular project budget, add this to your jobscript:

#$\u00a0-P\u00a0Gold\n#$\u00a0-A\u00a0MyProject\n

To submit a free job that will not use up any Gold, use this instead:

#$\u00a0-P\u00a0Free\n#$\u00a0-A\u00a0MyProject\n

You can also submit testing jobs that will not use up any Gold, and will have higher priority than normal free jobs, but are limited to 2 nodes (80 cores) and 1 hour of walltime:

#$\u00a0-P\u00a0Test\n#$\u00a0-A\u00a0MyProject\n
"},{"location":"Clusters/Young/#troubleshooting-unable-to-verify-membership-in-policyjsv-project","title":"Troubleshooting: Unable to verify membership in policyjsv project","text":"
Unable\u00a0to\u00a0run\u00a0job:\u00a0Rejected\u00a0by\u00a0policyjsv\nUnable\u00a0to\u00a0verify\u00a0membership\u00a0of\u00a0`<username>`\u00a0in\u00a0the\u00a0policyjsv\u00a0project\n

You asked for a Free job but didn't specify #$ -A MyProject in your jobscript.

"},{"location":"Clusters/Young/#troubleshooting-unable-to-verify-membership-in-project-uninitialized-value","title":"Troubleshooting: Unable to verify membership in project / Uninitialized value","text":"
Unable to run job: Rejected by policyjsv \nReason:Unable to verify sufficient material worth to submit this job: \nUnable to verify membership of mmmxxxx in the UCL_Example project\n

This error from qsub can mean that you aren't in the project you are trying to submit to, but also happens when the Gold daemon is not running. 

Use of uninitialized value in print at /opt/gold/bin/mybalance line 60, <GBALANCE> line 1.\nFailed sending message: (Unable to connect to socket (Connection refused)).\n

If you also get this error from the budgets command, then the Gold daemon is definitely not running and you should contact rc-support.

"},{"location":"Clusters/Young/#gold-charging","title":"Gold charging","text":"

When you submit a job, it will reserve the total number of core hours that the job script is asking for. When the job ends, the Gold will move from 'reserved' into charged. If the job doesn't run for the full time it asked for, the unused reserved portion will be refunded after the job ends. You cannot submit a job that you do not have the budget to run.

"},{"location":"Clusters/Young/#troubleshooting-unable-to-verify-sufficient-material-worth","title":"Troubleshooting: Unable to verify sufficient material worth","text":"
Unable\u00a0to\u00a0run\u00a0job:\u00a0Rejected\u00a0by\u00a0policyjsv\nReason:Unable\u00a0to\u00a0verify\u00a0sufficient\u00a0material\u00a0worth\u00a0to\u00a0submit\u00a0this\u00a0job:\nInsufficient\u00a0balance\u00a0to\u00a0reserve\u00a0job\n

This means you don't have enough Gold to cover the cores \u2a09 wallclock time cost of the job you are trying to submit. You need to wait for queued jobs to finish and return unused Gold to your project, or submit a smaller/shorter job. Note that array jobs have to cover the whole cost of all the tasks at submit time.

"},{"location":"Clusters/Young/#job-deletion","title":"Job deletion","text":"

If you qdel a submitted Gold job, the reserved Gold will be made available again. This is done by a cron job that runs every 15 minutes, so you may not see it back instantly.

"},{"location":"Clusters/Young/#reporting-gold-usage","title":"Reporting Gold usage","text":"

There are a few commands that everyone can run that report Gold usage for their entire project, broken down by user. See Reporting from Gold.

Specifically, gstatement can show you a summary of who in your budget used what amount during a given time period:

# summarise per user usage of the MyProject budget from 00:00 on these dates\ngstatement -p MyProject -s 2023-01-01 -e 2023-04-01 --summarize\n
"},{"location":"Clusters/Young/#requests-for-longer-wallclock-time","title":"Requests for longer wallclock time","text":"

You can apply for access to a 96-hour queue for Gold jobs only using this form:

The request must be sent to your local MMM Hub point of contact (PoC) who will evaluate the request and approve or reject it, as appropriate. The list of PoCs may be found on the MMM Hub website.

Please complete the form and send it to your PoC.

The request must include a clear and compelling justification of why the usual 48-hour wall time is insufficient, how a 96-hour wall time would alleviate the problem and what it will enable you to achieve scientifically. Justifications that are solely based on simplifying your workflow and job management will not be approved - clear reasoning must be provided for why it is either very difficult or impossible to work within a 48-hour wall time.

"},{"location":"Clusters/Young/#support","title":"Support","text":"

Email rc-support@ucl.ac.uk with any support queries. It will be helpful to include Young in the subject along with some descriptive text about the type of problem, and you should mention your username in the body.

"},{"location":"Clusters/Young/#acknowledging-the-use-of-young-in-publications","title":"Acknowledging the use of Young in publications","text":"

All work arising from this facility should be properly acknowledged in presentations and papers with the following text:

\"We are grateful to the UK Materials and Molecular Modelling Hub for computational resources, which is partially funded by EPSRC (EP/T022213/1, EP/W032260/1 and EP/P020194/1)\"

"},{"location":"Clusters/Young/#mcc","title":"MCC","text":"

When publishing work that benefited from resources allocated by the MCC: please include the following acknowledgment:

\"Via our membership of the UK's HEC Materials Chemistry Consortium, which is funded by EPSRC (EP/L000202), this work used the UK Materials and Molecular Modelling Hub for computational resources, MMM Hub, which is partially funded by EPSRC (EP/T022213/1, EP/W032260/1 and EP/P020194/1)\"

"},{"location":"Clusters/Young/#ukcp","title":"UKCP","text":"

When publishing work that benefited from resources allocated by UKCP, please include:

\"We are grateful for computational support from the UK Materials and Molecular Modelling Hub, which is partially funded by EPSRC (EP/T022213/1, EP/W032260/1 and EP/P020194/1), for which access was obtained via the UKCP consortium and funded by EPSRC grant ref EP/P022561/1\"

"},{"location":"Installed_Software_Lists/module-packages/","title":"General Software Lists","text":"

Our clusters have a wide range of software installed, available by using the modules system.

The module files are organised by name, version, variant (where applicable) and, if relevant, the compiler version used to build the software. If no compiler version is given, either no compiler was required, or only the base system compiler (/usr/bin/gcc) and libraries were used.

When we install applications, we try to install them on all of our clusters, but sometimes licence restrictions prevent it. If something seems to be missing, it may be because we are not able to provide it. Please contact us for more information if this is hindering your work.

The lists below were last updated at 15:05:45 (+0100) on 09 Sep 2023, and are generated from the software installed on the Myriad cluster.

"},{"location":"Installed_Software_Lists/module-packages/#bundles","title":"Bundles","text":"

Some applications or tools depend on a lot of other modules, or have some awkward requirements. For these, we sometimes make a \"bundle\" module in this section, that loads all the dependencies.

For Python and R in particular, we also have recommended bundles that load the module for a recent version of Python or R, along with a collection of packages for it that have been requested by users, and the modules those packages require.

The lists of Python and R packages installed for those bundles are on separate pages:

We'll sometimes include /new and /old versions of these bundles, if we've recently made a version switch or are intending to make one soon. We send out emails to the user lists about version changes, so if you use these bundles, you should look out for those.

Module Description beta-modules This module adds the beta module space to your environment. bioperl/recommended Loads all the modules needed to use BioPerl. blic-modules Adds Cancer Biology supported modules to your environment. brunel-modules Adds Brunel licensed software module space to module avail. cancerit/20190218 adds UCL set of cancerit packages to your environment variables cancerit/recommended adds UCL recommended set of cancerit packages to your environment variables chemistry-modules Adds Chemistry Department supported modules to your environment. climate-tools/recommended Adds set of default applications to the environment for climate science users. deep_earth Sets up VASP, Gnuplot etc for Earth Sciences default-modules-aristotle Adds default Aristotle modules to your environment. default-modules/2015 Adds default modules to your environment. default-modules/2017 Adds default modules to your environment. default-modules/2018 Adds default modules to your environment. economics-modules Adds Economics Department modules to your environment. farr-modules Adds FARR supported modules to your environment. farr/recommended Adds set of default applications to the environment for FARR users. gmt/new Adds set of default modules to the environment for GMT users. gmt/old Adds set of default modules to the environment for gmt users. gmt/recommended Adds set of default modules to the environment for gmt users. imperial-modules Adds Imperial College licensed software module space to module avail. naglib/mark27-intel-2019 adds the NAG Library Mark 27 and required modules to your environment. octave/recommended Octave is an open source competitor to Matlab. personal-modules Adds personal modules to your environment. physics-modules Adds Pysics Department supported modules to your environment. pypy3/3.5-compat Adds UCL recommended set of Pypy3 python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/pypy-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/pypy-3.list python2/recommended Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-2.list python3/3.4 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list python3/3.5 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list python3/3.6 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list python3/3.7 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list python3/3.8 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list python3/3.9 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list python3/3.11 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list python3/recommended Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list r/new adds UCL recommended set of R packages for R 4.2.0 to your environment variables r/old adds UCL recommended set of R packages to your environment for R 3.6.0 r/r-3.5.1_bc-3.7 adds UCL recommended set of R packages to your environment variables r/r-3.6.0_bc-3.9 adds UCL recommended set of R packages to your environment variables r/recommended adds UCL recommended set of R packages for R 4.2.0 to your environment variables rsd-modules Adds Research Software Development supported modules to your environment. thermo-modules Adds modules for Molecular Thermodynamics to your environment. torch-deps Loads the dependencies for Torch and makes a quick-install alias. workaround-modules This module adds the workarounds module space to your environment."},{"location":"Installed_Software_Lists/module-packages/#applications","title":"Applications","text":"Module Description abaqus/2017 Adds Abaqus 2017 to your environment. abaqus/2017-intelmpi Adds Abaqus 2017 to your environment. abinit/9.6.2/intel-2018-update3 adds ABINIT Version 9.6.2 compiled using Intel 2018 to your environment. adf/2014.10 Adds ADF 2014.10 to your environment. afni/22.2.05 Adds AFNI to your environment. afni/23.0.02 Adds AFNI to your environment. afni/20151030 Adds AFNI to your environment. afni/20181011 Adds AFNI to your environment. amber/14/mpi/intel-2015-update2 Adds AMBER 14 to your environment amber/14/openmp/intel-2015-update2 Adds AMBER 14 to your environment amber/14/serial/intel-2015-update2 Adds AMBER 14 to your environment amber/16/mpi/gnu-4.9.2 Adds AMBER 16 to your environment amber/16/mpi/intel-2015-update2 Adds AMBER 16 to your environment amber/16/openmp/gnu-4.9.2 Adds AMBER 16 to your environment amber/16/openmp/intel-2015-update2 Adds AMBER 16 to your environment amber/16/serial/gnu-4.9.2 Adds AMBER 16 to your environment amber/16/serial/intel-2015-update2 Adds AMBER 16 to your environment ampliconarchitect/1.3r1 Amplicon Architect is a tool for finding amplicons in Illumina reads. ams/2023.101 Adds the Amsterdam Modeling Suite Version 2023.101 to your environment. ansys/17.2 Adds Ansys CFX/Fluent etc to your environment ansys/18.0 Adds Ansys CFX/Fluent etc to your environment ansys/19.1 Adds Ansys CFX/Fluent, EM etc to your environment ansys/2019.r3 Adds Ansys CFX/Fluent, EM etc to your environment ansys/2021.r2 Adds Ansys CFX/Fluent, EM etc to your environment ants/2.1.0 Adds ANTs 2.1.0 (Advanced Normalization Tools) to your environment. ANTs is popularly considered a state-of-the-art medical image registration and segmentation toolkit. approxwf/gnu-4.9.2 Adds ApproxWF to your environment. arrayfire/3.5.0/gnu-4.9.2 Adds ArrayFire 3.5.0 to your environment. asp/2.6.2 Adds NASA Ames Stereo Pipeline (ASP) 6.2.2 to your environment. autodock/4.2.6 Adds AutoDock and AutoGrid 4.2.6 to your environment. AutoDock is a suite of automated docking tools. It is designed to predict how small molecules, such as substrates or drug candidates, bind to a receptor of known 3D structure. bamtools/2.4.0/gnu-4.9.2 Adds BamTools 2.4.0 to your environment. BamTools provides both a programmer's API and an end-user's toolkit for handling BAM files. bcftools/1.2/gnu-4.9.2 Adds BCFtools 1.2 to your environment. Reading/writing BCF2/VCF/gVCF files and calling/filtering/summarising SNP and short indel sequence variants bcftools/1.3.1/gnu-4.9.2 Adds BCFtools 1.3.1 to your environment. Reading/writing BCF2/VCF/gVCF files and calling/filtering/summarising SNP and short indel sequence variants bcftools/1.11/gnu-4.9.2 Tools for reading/writing BCF2/VCF/gVCF files and calling/filtering/summarising SNP and short indel sequence variants. bcftools/2.1/gnu-4.9.2 Adds BCFtools 1.2 to your environment. Reading/writing BCF2/VCF/gVCF files and calling/filtering/summarising SNP and short indel sequence variants bcl2fastq/1.8.4 Adds bcl2fastq 1.8.4 to your environment. bcl2fastq2/2.19.1 Adds bcl2fastq2 2.19.1 to your environment. bcl2fastq2/2.20.0-rpm Adds bcl2fastq2 2.20.0.422 from rpm to your environment. beast/2.3.0 Adds BEAST 2.3.0 with addons to your PATH. bedtools/2.25.0 Adds bedtools 2.25.0 to your environment. The bedtools utilities are a swiss-army knife of tools for a wide-range of genomics analysis tasks. bgen/1.1.4 Adds BGen 1.1.4 to your environment. blast+/2.2.30/intel-2015-update2 This module adds the BLAST+ 2.2.30 package to your environment. blast+/2.13.0/gnu-7.3.0 This module adds the BLAST+ 2.13.0 package to your environment. blast/2.2.26 Adds Blast 2.2.26 to your environment. blender/2.79 Adds Blender Version 2.79 to your environment. boltztrap/1.2.5/intel-2018 Adds boltztrap 1.2.5 to your environment. bowtie/1.1.2 Adds Bowtie 1.1.2 to your environment. bowtie2/2.2.5 Adds Bowtie2 2.2.5 to your environment. bwa/0.6.2/gnu-4.9.2 Adds BWA 0.7.12 to your environment. BWA is a software package for mapping DNA sequences against a large reference genome, such as the human genome. bwa/0.7.12/gnu-4.9.2 Adds BWA 0.7.12 to your environment. BWA is a software package for mapping DNA sequences against a large reference genome, such as the human genome. caffe/1.0/cpu Adds Caffe 1.0 for CPU to your environment. caffe/1.0/cudnn Adds Caffe 1.0 for CUDA+CudNN to your environment. caffe/1.0/gpu Adds Caffe 1.0 for CUDA to your environment. cancerit/20190218-python-2.7.12/gnu-4.9.2 Adds CancerIT program versions as of 20190218 to your environment. The CancerIT Suite is a collection of linked bioinformatics tools. cancerit/gnu-4.9.2 Adds the cancer it suite to your environment. castep/17.2/intel-2017 Adds castep 17.2 to your environment. CASTEP is a program that uses density functional theory to calculate the properties of materials from first principles. castep/17.21/intel-2017 Adds castep 17.21 to your environment. CASTEP is a program that uses density functional theory to calculate the properties of materials from first principles. castep/19.1.1/intel-2019 CASTEP is a program that uses density functional theory to calculate the properties of materials from first principles. cctools/5.4.1/gnu-4.9.2 Adds cctools 5.4.1 to your environment. cctools/7.0.11/gnu-4.9.2 Adds cctools 7.0.11 to your environment. cdo/2.0.6/gnu-10.2.0 This module adds the CDO 2.0.6 package to your environment. cellranger/5.0.1 Adds Cell Ranger 5.0.1 to your environment. Cell Ranger is a set of analysis pipelines that process Chromium single-cell RNA-seq output to align reads, generate feature-barcode matrices and perform clustering and gene expression analysis. cellranger/6.0.1 Adds Cell Ranger 6.0.1 to your environment. Cell Ranger is a set of analysis pipelines that process Chromium single-cell RNA-seq output to align reads, generate feature-barcode matrices and perform clustering and gene expression analysis. cesm/1.0.6/intel-2015-update2 Adds CESM 1.0.6 to your environment. cesm/1.2.2/intel-2015-update2 Adds CESM 1.2.2 to your environment. cfd-ace/2014.1 Adds CFD-ACE+ to your execution path. Only on Kathleen and Myriad. cfd-ace/2018.0 Adds CFD-ACE+ to your execution path. Only on Kathleen and Myriad. chemshell/3.7.1/mpi/gulp4.5 This is a modulefile for ChemShell 3.7.1, MPI+GULP version. Can be used to run other packages if you load a module for those. chemshell/3.7.1/standalone This is a modulefile for ChemShell 3.7.1, standalone serial version. Can be used to run GULP and other packages if you load a module for those. clustal-omega/1.2.1 Adds Clustal Omega 1.2.1 to your environment. clustal-w/2.1 Adds Clustal W 2.1 to your environment. cmg/2017.101 Adds CMG Reservoir Simulation Software Version 2017.101 to your environment. cmg/2018.101 Adds CMG Reservoir Simulation Software Version 2018.101 to your environment. cmg/2019.101 Adds CMG Reservoir Simulation Software Version 2019.101 to your environment. collectl/4.0.2 [collectl/4.0.2] collectl is a tool for tracking and monitoring various node usage statistics. compucell3d/3.7.4 Adds CompuCell3D to your environment comsol/6.0 COMSOL is a general-purpose macroscopic physics simulation package. comsol/6.1 COMSOL is a general-purpose macroscopic physics simulation package. comsol/6.1-eee COMSOL is a general-purpose macroscopic physics simulation package. Installed for EEE Dept comsol/52 Adds the COMSOL 52 binaries to your environment. COMSOL Multiphysics\u00ae is a general-purpose software platform, based on advanced numerical methods, for modeling and simulating physics-based problems. Module must be loaded once from a login node prior to running jobs. comsol/52a Adds the COMSOL 52a binaries to your environment. COMSOL Multiphysics\u00ae is a general-purpose software platform, based on advanced numerical methods, for modeling and simulating physics-based problems. Module must be loaded once from a login node prior to running jobs. comsol/53a Adds COMSOL Multiphysics Version 53a to your environment. comsol/56 Adds COMSOL Multiphysics Version 56 to your environment. cosi-corr/oct14 Adds COSI-Corr Version Oct14 for use with ENVI 5.5.2/5.5.3 to your environment. covid-19-spatial-sim/0.8.0/intel-2020 SpatialSim COVID-19 pandemic modelling tool from Imperial College. covid-19-spatial-sim/0.9.0/gnu-4.9.2 SpatialSim COVID-19 pandemic modelling tool from Imperial College. covid-19-spatial-sim/0.13.0/gnu-4.9.2 SpatialSim COVID-19 pandemic modelling tool from Imperial College. covid-19-spatial-sim/0.14.0/gnu-4.9.2 SpatialSim COVID-19 pandemic modelling tool from Imperial College. covid-19-spatial-sim/0.14.0/intel-2020 SpatialSim COVID-19 pandemic modelling tool from Imperial College. covid-19-spatial-sim/0.15.0/gnu-4.9.2 SpatialSim COVID-19 pandemic modelling tool from Imperial College. covid-19-spatial-sim/0.15.0/intel-2020 SpatialSim COVID-19 pandemic modelling tool from Imperial College. cp2k/4.1/ompi/gnu-4.9.2 Adds CP2K to your environment. cp2k/5.1/ompi-plumed/gnu-4.9.2 Adds CP2K to your environment. cp2k/5.1/ompi/gnu-4.9.2 Adds CP2K to your environment. cp2k/6.1/ompi/gnu-4.9.2 Adds CP2K to your environment. cp2k/7.1/ompi/gnu-4.9.2 Adds CP2K to your environment. cpmd/4.1/intel-2017 Adds CPMD 4.1 to your environment. crystal14/v1.0.3 Adds Crystal14 v1.0.3 to your environment. crystal14/v1.0.4 Adds Crystal14 v1.0.4 to your environment. crystal14/v1.0.4_2017 Adds Crystal14 v1.0.4 to your environment. crystal17/v1.0.1 Adds Crystal17 v1.0.1 to your environment. crystal17/v1.0.2/intel-2017 The CRYSTAL program computes the electronic structure of periodic systems within Hartree Fock, density functional or various hybrid approximations. cuba/4.2/gnu-4.9.2 adds Cuba Numerical Integration Package Version 4.2 to your environment. cufflinks/2.2.1 Adds Cufflinks 2.2.1 to your environment. curl/7.47.1/gnu-4.9.2 Adds curl 7.47.1 to your environment. curl/7.86.0/gnu-4.9.2 Adds curl 7.86.0 to your environment. dakota/6.12.0-bindist Dakota is a package for performing parametric analysis with other software packages. datamash/1.4 This is a module with no description string. deeptools/3.0.2 Adds deeptools to your environment. delly/0.7.8-bindist Delly is an integrated structural variant (SV) prediction method that can discover, genotype and visualize deletions, tandem duplications, inversions and translocations at single-nucleotide resolution in short-read massively parallel sequencing data. dftbplus/17.1/intel-2017 DFTB+ is a quantum mechanical simulation software package, based on the Density Functional Tight Binding (DFTB) method. dftbplus/18.2/intel-2018 DFTB+ is a software package for carrying out fast quantum mechanical atomistic calculations based on the Density Functional Tight Binding method. dftbplus/19.1/intel-2018 DFTB+ is a software package for carrying out fast quantum mechanical atomistic calculations based on the Density Functional Tight Binding method. dftbplus/dev/d07f92e/intel-2017 DFTB+ is a quantum mechanical simulation software package, based on the Density Functional Tight Binding (DFTB) method. dl_monte/2.07/intel-2018 DL_MONTE is an open-source program for applying Monte Carlo molecular simulation to a wide range of systems. dl_poly/4.07/intel-2015-update2 Adds DL_POLY 4.07 to your environment dl_poly/4.08-plumed-2.3.1/intel-2017 Adds dl_poly 4.08 to your environment. DL_POLY is a general purpose classical molecular dynamics (MD) simulation software developed at Daresbury Laboratory. This version has been linked against the PLUMED metadynamics library. dl_poly/4.08/intel-2015-update2 Adds DL_POLY 4.08 to your environment. dl_poly/4.09/intel-2018 Adds DL_POLY 4.09 to your environment. dl_poly/4.10.0/intel-2018 Adds DL_POLY 4.10.0 to your environment. DL_POLY is a general purpose classical molecular dynamics (MD) simulation software. dl_poly/5.0.0/intel-2018 DL_POLY is a general-purpose classical molecular dynamics (MD) simulation software package. dl_poly/classic/1.9/intel-2015-update2 Adds DL_POLY Classic 1.9 to your environment dock/6.9-impi/intel-2018 The DOCK suite of programs is designed to find favorable orientations of a ligand in a receptor. This is the Intel MPI build, intended for high-performance parallel runs. dock/6.9-reference/gnu-4.9.2 The DOCK suite of programs is designed to find favorable orientations of a ligand in a receptor. This is a reference build intended to be close to the version of the software the developers test with: a serial build using the GNU compilers. dos2unix/7.3 Adds dos2unix 7.3 to your environment. Text format converters dos2unix, unix2dos, mac2unix, unix2mac. dssp/3.0.0/gnu-4.9.2 Adds dssp 3.0.0 to your environment. DSSP calculates DSSP entries from Protein Databank (PDB) entries. dymola/2020.1-1 Dymola is a commercial modeling and simulation environment based on the open Modelica modeling language. ea-utils/822 Adds ea-utils to your environment. easylausanne/55c7bf0 Adds Easy Lausanne to your environment. eigensoft/6.1.1/gnu-4.9.2 Adds EIGENSOFT 6.1.1 to your environment. Population genetics methods and EIGENSTRAT stratification correction method. elk/4.0.15/intel-2018 Adds Elk 4.0.15 to your environment. Binary is elk. elk/4.0.15/intel-2018+wa Adds Elk 4.0.15 to your environment. Binary is elk. elk/4.3.6/intel-2017 Adds Elk 4.3.6 to your environment. Binary is elk. elk/4.3.6/intel-2017+wa Adds Elk 4.3.6 to your environment. Binary is elk. elk/5.2.14/intel-2018 Elk is an all-electron full-potential linearised augmented-planewave (FP-LAPW) code. elk/6.8.4/intel-2018 An all-electron full-potential linearised augmented-planewave (FP-LAPW) code. energyplus/8.9.0-bindist EnergyPlus\u2122 is a whole building energy simulation program that engineers, architects, and researchers use to model both energy consumption\u2014for heating, cooling, ventilation, lighting and plug and process loads\u2014and water use in buildings. energyplus/9.1.0-bindist EnergyPlus\u2122 is a whole building energy simulation program that engineers, architects, and researchers use to model both energy consumption\u2014for heating, cooling, ventilation, lighting and plug and process loads\u2014and water use in buildings. envi/5.5.2 Adds ENVI 5.5.2 with IDL 8.7.2 to your environment. envi/5.5.3 Adds ENVI 5.5.3 with IDL 8.7.3 to your environment. epacts/3.3.0/gnu-4.9.2 Adds EPACTS 3.3.0 to your environment. examl/8dcf2cc/gnu-4.9.2 Adds ExaML to your environment. exonerate/2.2.0 Adds Exonerate to your environment. fasta/36.3.8d/gnu-4.9.2 Adds the cancer it suite to your environment. fastqc/0.11.5 Adds FastQC 0.11.5 to your environment. A quality control application for high throughput sequence data. fastqc/0.11.8 Adds FastQC 0.11.8 to your environment. A quality control application for high throughput sequence data. ffmpeg/4.1/gnu-4.9.2 FFmpeg is a framework for encoding, decoding, muxing, demuxing, encoding, transcoding, streaming, filtering, and playing many types of audio and video media. fgbio/0.5.1 Adds fgbio to your environment. fgbio is a command line toolkit for working with genomic and particularly next generation sequencing data. fgbio/0.6.1 Adds fgbio to your environment. fgbio is a command line toolkit for working with genomic and particularly next generation sequencing data. figtree/1.4.2 Adds Figtree 1.4.2. foldx/4 Adds FoldX Suite 4.0 to your environment. foldx/5 Adds FoldX Suite 5.0 to your environment. freesurfer/5.3.0 Adds FreeSurfer 5.3.0 to your environment. FreeSurfer is a set of automated tools for reconstruction of the brain's cortical surface from structural MRI data, and overlay of functional MRI data onto the reconstructed surface. freesurfer/6.0.0 Adds FreeSurfer 6.0.0 to your environment. FreeSurfer is a set of automated tools for reconstruction of the brain's cortical surface from structural MRI data, and overlay of functional MRI data onto the reconstructed surface. freesurfer/7.2.0 Adds FreeSurfer 7.2.0 to your environment. FreeSurfer is a set of automated tools for reconstruction of the brain's cortical surface from structural MRI data, and overlay of functional MRI data onto the reconstructed surface. fsl/5.0.9 Adds FSL 5.0.9 (FMRIB Software Library) to your environment. FSL is a comprehensive library of analysis tools for FMRI, MRI and DTI brain imaging data. fsl/5.0.10 Adds FSL 5.0.10 (FMRIB Software Library) to your environment. FSL is a comprehensive library of analysis tools for FMRI, MRI and DTI brain imaging data. fsl/6.0.0 Adds FSL 6.0.0 (FMRIB Software Library) to your environment. FSL is a comprehensive library of analysis tools for FMRI, MRI and DTI brain imaging data. fsl/6.0.0_cuda Adds FSL 6.0.0 CUDA (FMRIB Software Library) to your environment. FSL is a comprehensive library of analysis tools for FMRI, MRI and DTI brain imaging data. fsl/6.0.4 Adds FSL 6.0.4 CUDA (FMRIB Software Library) to your environment. FSL is a comprehensive library of analysis tools for FMRI, MRI and DTI brain imaging data. gamess/5Dec2014_R1/intel-2015-update2 Adds GAMESS 5Dec2014_R1 to your environment, built for Intel MPI. Uses ~/Scratch/gamess for USERSCR. You can override by exporting GAMESS_USERSCR as another path. gatk/3.4.46 Adds GATK 3.4.46 to your environment. The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyze high-throughput sequencing data. Website: https://www.broadinstitute.org/gatk/index.php gatk/3.8.0 Adds GATK 3.8.0 to your environment. The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyze high-throughput sequencing data. Website: https://www.broadinstitute.org/gatk/index.php gatk/4.0.3.0 Adds GATK 4.0.3.0 to your environment. The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyze high-throughput sequencing data. Website: https://www.broadinstitute.org/gatk/index.php gatk/4.0.8.0 Adds GATK 4.0.8.0 to your environment. The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyze high-throughput sequencing data. Website: https://www.broadinstitute.org/gatk/index.php gatk/4.2.1.0 Adds GATK 4.2.1.0 to your environment. The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyze high-throughput sequencing data. Website: https://www.broadinstitute.org/gatk/index.php gatk/4.2.5.0 Adds GATK 4.2.1.0 to your environment. The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyze high-throughput sequencing data. Website: https://www.broadinstitute.org/gatk/index.php gaussian/g09-c01_linda/pgi-2013.9 Adds Gaussian 09 Revision C01 and GaussView 5 to your environment. gaussian/g09-d01/pgi-2015.4 Adds Gaussian G09-D01 to your environment and also includes Linda and Gaussview 5. gaussian/g09-d01/pgi-2015.7 Adds Gaussian G09-D01 to your environment and also includes Linda and Gaussview 5 gaussian/g16-a03/pgi-2016.5 Adds Gaussian G16-A03 to your environment and also includes Linda and Gaussview 6. gaussian/g16-c01/pgi-2018.10 Adds Gaussian G16-C01 to your environment and also includes Linda and Gaussview 6. gcta/1.93.2beta GCTA: a tool for Genome-wide Complex Trait Analysis. gdal/2.0.0 Adds GDAL 2.0.0 to your environment variables. Works with Python 2. gdal/2.1.1 adds GDAL 2.1.1 with PROJ.4 4.9.1 to your environment variables. Works with Python 2. gdal/2.1.4 adds GDAL 2.1.4 with PROJ.4 6.1.0 to your environment variables. Works with Python 2. gdal/3.0.4/gnu-4.9.2 adds GDAL 3.0.4 with PROJ.4 6.1.0 to your environment variables. Works with Python 2. gephi/0.9.1 Adds Gephi Version 0.9.1 to your environment. gftp/2.9.1b gFTP is a file transfer client that supports a wide range of FTP-like protocols. ghostscript/9.16/gnu-4.9.2 Adds Ghostscript 9.16 to your environment. ghostscript/9.19/gnu-4.9.2 Adds Ghostscript 9.19 to your environment. gmsh/2.12.0-bindist Adds gmsh 2.12.0 to your environment. Gmsh is a free 3D finite element grid generator with a build-in CAD engine and post-processor. gmt/5.1.2 adds GMT 5.1.2 to your environment variables gmt/5.3.1 adds GMT 5.3.1 to your environment variables gmt/5.4.5 adds GMT 5.4.5 to your environment variables gnuplot/5.0.1 Adds gnuplot 5.0.1 to your environment. Gnuplot is a portable command-line driven graphing utility. grace/5.1.25 Adds Grace 5.1.25 to your environment. Grace is a 2D plotting tool. graphicsmagick/1.3.21 adds GraphicsMagick 1.3.21 to your environment variables graphviz/2.38.0/gnu-4.9.2 This module adds the Graphviz 2.38.0 package to your environment. Graphviz is open source graph visualization software. graphviz/2.40.1/gnu-4.9.2 This module adds the Graphviz 2.40.1 package to your environment. Graphviz is open source graph visualization software. groff/1.22.3/gnu-4.9.2 Adds GNU groff Version 1.22.3 to your environment. gromacs/5.0.4/intel-2015-update2 Adds GROMACS 5.0.4 to your environment, built using MKL gromacs/5.0.4/plumed/intel-2015-update2 Adds GROMACS 5.0.4 with Plumed 2.1.2 to your environment. Note: Plumed will always run in double precision even if GROMACS is single-precision, so only use that combination if you need it and are aware of the effects. gromacs/5.1.1/intel-2015-update2 Adds GROMACS 5.1.1 to your environment, built using MKL gromacs/5.1.1/plumed/intel-2015-update2 Adds GROMACS 5.1.1 with Plumed 2.2 to your environment. Note: Plumed will always run in double precision even if GROMACS is single-precision, so only use that combination if you need it and are aware of the effects. gromacs/5.1.3/plumed/intel-2015-update2 GROMACS 5.1.3 molecular dynamics package, built with Intel 2015u2 compilers, PLUMED 2.2.3 patches (including libmatheval), and OpenBLAS 0.2.14. gromacs/2016.3/intel-2017-update1 Adds GROMACS 2016.3 to your environment, built using MKL gromacs/2016.3/plumed/intel-2017-update1 GROMACS 2016.3 molecular dynamics package, built with Intel 2017u1 compilers, PLUMED 2.3.1 patches (including libmatheval), and OpenBLAS 0.2.14. gromacs/2016.4/plumed/intel-2017 GROMACS 2016.4 molecular dynamics package, built with Intel 2017u4 compilers, PLUMED 2.4.1 patches (including libmatheval) with hrex, and OpenBLAS 0.2.14. gromacs/2018.2/intel-2018 Adds gromacs 2018 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2018.3/intel-2018 Adds gromacs 2018 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2018.3/plumed/intel-2018 GROMACS 2018.3 molecular dynamics package, built with Intel 2018u3 compilers, PLUMED 2.4.3 patches (including libmatheval). gromacs/2018/intel-2017 Adds gromacs 2018 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2019.3/cuda-10 Adds gromacs 2019 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2019.3/intel-2018 Adds gromacs 2019 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2019.3/plumed/intel-2018 GROMACS 2019.3 molecular dynamics package, built with Intel 2018u3 compilers, PLUMED 2.5.2 patches (including libmatheval). gromacs/2021.3/plumed/gnu-10.2.0 GROMACS 2021.3 molecular dynamics package, built with GNU 10.2.0 compilers, PLUMED 2.7.2 patches (including libmatheval). gulp/4.5/intel-2018 Adds GULP 4.5 to your environment. Built with FoX and without plumed. GULP is a materials simulation code. gulp/5.1.1/intel-2018 Adds GULP 5.1.1 to your environment. Built with FoX and without plumed. GULP is a materials simulation code. gurobi/7.5.1 Adds Gurobi 7.5.1 to your environment. gurobi/8.1.1 Adds Gurobi 8.1.1 to your environment. gurobi/9.1.2 Adds Gurobi 9.1.2 to your environment. h5utils/1.12.1 Adds h5utils 1.12.1 to your environment. h5utils is a set of utilities for visualization and conversion of scientific data in HDF5 format. hammock/1.0.5 Loads the dependencies for Hammock 1.0.5 to your environment and makes a quick-install alias, do-hammock-install. Run as java -Xmx2g -jar $HAMMOCKPATH/Hammock.jar mode param1 param2 -d outputpath. Will use Scratch for temporary files. hhsuite/3.0-beta.1/gnu-4.9.2 Adds hhsuite 3.0-beta.1 to your environment. hisat2/2.2.1/gnu-4.9.2 Adds HISAT2 Version 2.2.1 to your environment. hmmer/3.1b2 Adds HMMER 3.1b2 to your environment. hmri/0.4.0/spm12.jan2020 Adds the hMRI Toolbox for use with SPM12 to your environment hoomd-blue/2.4.2 Adds HOOMD-blue to your environment. hopspack/2.0.2/gnu-4.9.2 Adds HOPSPACK 2.0.2 to your environment hopspack/2.0.2/intel-2017 Adds HOPSPACK 2.0.2 to your environment icommands/4.1.7 [icommands/4.1.7] The iRODS iCommands are the command-line clients to an iRODS system. idl/8.4.1 Adds IDL 8.4.1 to your environment. idl/8.7.3 Adds IDL 8.7.3 to your environment. idl/8.8.3 Adds IDL 8.8.3 to your environment. illustrate/20190807 adds Illustrate to your environment variables impute2/2.3.2 adds Impute2 V2.3.2 to your environment. inetutils/1.9.4 GNU inetutils is a package of utilities for performing a range of network tasks including FTP and telnet clients. intltool/0.51.0 Adds intltool 0.51.0 to your environment. intltool is a set of tools to centralize translation of many different file formats using GNU gettext-compatible PO files. iva/0.11.6 Adds IVA 0.11.6 to your environment. iva/1.0.0 Adds IVA 1.0.0 to your environment. jags/3.4.0/gnu.4.9.2-atlas Adds JAGS 3.4.0 to your environment. jags/3.4.0/gnu.4.9.2-openblas Adds JAGS 3.4.0 to your environment. jags/4.2.0/gnu.4.9.2-openblas Adds JAGS 4.2.0 to your environment. jq/1.5/gnu-4.9.2 adds jq for GCC 4.9.2 to your environment. jq/1.6-bindist jq is a lightweight and flexible command-line JSON processor. kallisto/v0.42.5 Adds Kallisto v0.42.5 to your environment. kallisto/v0.46.1 Adds Kallisto v0.46.1 to your environment. keras/2.2.4 Adds Keras to your environment. kmc/2.1.1/gnu-4.9.2 Adds KMC 2.1.1 to your environment. KMC is a disk-based program for counting k-mers from FASTQ/FASTA files. knitro/12.0.0/gnu-4.9.2 Adds Knitro solver 12.0.0 to your environment. knitro/12.4.0/gnu-4.9.2 Adds Knitro solver 12.4.0 to your environment. knitro/13.1.0/gnu-4.9.2 Adds Knitro solver 13.1.0 to your environment. lammps/3Mar20/plumed-colvars/intel-2018 Adds LAMMPS 3Mar20 to your environment. LAMMPS is a GPL molecular dynamics code which shows exceptional scaling on a wide variety of machines. Binary is lmp_mpi or lmp_default. This version was built with packages kspace, manybody, molecule, rigid, lib-linalg, user-colvars and user-plumed. lammps/7Aug19/basic/intel-2018 Adds LAMMPS 7Aug19 to your environment. Binary is lmp_default. lammps/7Aug19/gpu/intel-2018 Adds LAMMPS 7Aug19 to your environment. Binary is lmp_default. lammps/7Aug19/userintel/intel-2018 Adds LAMMPS 7Aug19 to your environment. Binary is lmp_default. lammps/8Dec15/intel-2015-update2 Adds LAMMPS 8Dec15 to your environment. Binary is lmp_default. lammps/10Feb15/intel-2015-update2 Adds LAMMPS 10Feb15 to your environment. Binary is lmp_default. lammps/13Apr17/intel-2017 Adds LAMMPS 13Apr17 to your environment. Binary is lmp_default. lammps/16Mar18/basic/intel-2018 Adds LAMMPS 16Mar18 to your environment. Binary is lmp_default. lammps/16Mar18/gpu/intel-2018 Adds LAMMPS 16Mar18 to your environment. Binary is lmp_default. lammps/16Mar18/intel-2017 Adds LAMMPS 16Mar18 to your environment. Binary is lmp_default. lammps/16Mar18/userintel/intel-2018 Adds LAMMPS 16Mar18 to your environment. Binary is lmp_default. lynx/2.8.9 Adds Lynx Version 2.8.9 to your environment. mathematica/10.1.0 Adds Mathematica 10.1.0 to your environment. mathematica/10.2.0 Adds Mathematica 10.2.0 to your environment. mathematica/10.4.0 Adds Mathematica 10.4.0 to your environment. mathematica/11.0.1 Adds Mathematica 11.0.1 to your environment. mathematica/11.2.0 Adds Mathematica 11.2 to your environment. mathematica/11.3.0 Adds Mathematica 11.3 to your environment. mathematica/12.2.0 Adds Mathematica 12.2 to your environment. mathematica/13.1.0 Adds Mathematica 13.1 to your environment. matlab/full/r2015a/8.5 Adds Matlab R2015a for SPM to your environment. matlab/full/r2015b/8.6 Adds Matlab R2015b to your environment. matlab/full/r2016b/9.1 Adds Matlab R2016b to your environment. matlab/full/r2017a/9.2 Adds Matlab R2017a to your environment. matlab/full/r2018a/9.4 Adds Matlab R2018a to your environment. matlab/full/r2018b/9.5 Adds Matlab R2018b to your environment. matlab/full/r2019b/9.7 Adds Matlab R2019b to your environment. matlab/full/r2021a/9.10 Adds Matlab R2021a to your environment. matlab/full/r2023a/9.14 Adds Matlab R2023a to your environment. mcl/14-137 Adds MCL 14-137 your environment. meep/1.3-ompi/gnu-4.9.2 Adds meep 1.3-ompi to your environment. meep/1.3/gnu-4.9.2 Adds meep 1.3 to your environment. meep/1.11.0-ompi/gnu-4.9.2 Adds meep 1.11.0-ompi to your environment. MEEP is a package for electromagnetics simulation via the finite-diffe rence time-domain (FDTD) method. meme/4.10.1_4 Adds MEME Suite 4.10.1_4 to your environment. The MEME Suite: Motif-based sequence analysis tools. This install is for the command-line tools and connects to their website for further analysis. Web: http://meme-suite.org mgltools/1.5.6 Adds MGLTools 1.5.6 to your environment. Applications for visualization and analysis of molecular structures. Contains AutoDockTools (ADT), Python Molecular Viewer (PMV) and Vision. mirdeep/2.0.0.7 Adds mirdeep 2.0.0.7 to your environment. molden/5.2.2 Adds Molden 5.2.2 to your environment. molpro/2012.1.25/gnu-4.9.2 Adds Molpro to your environment molpro/2015.1.3 Adds Molpro 2015.1.3 binary (no Infiniband support) to your environment. molpro/2015.1.5/intel-2015-update2 Adds Molpro 2015.1.5 built from source with MPI to your environment. molpro/2020.1/openmp Adds Molpro 2020.1 mpi binary to your environment. mosek/8.1.0.83 The MOSEK Optimization Suite is a software package for solving large optimization problems with many constraints and variables. mosek/9.1.12 Adds Mosek 9.1.12 to your environment. mothur/1.41.3-bindist Mothur is an expandable, multi-purpose bioinformatics tool aimed at microbial ecology. mpb/1.5-ompi/gnu-4.9.2 Adds mpb 1.5 to your environment. mpb/1.5/gnu-4.9.2 Adds mpb 1.5 to your environment. mpb/1.9.0-hdf5-ompi/gnu-4.9.2 Adds serial mpb 1.9.0 to your environment. Built with HDF5-ompi for use by parallel MEEP. mrbayes/3.2.5/mpi/intel-2015-update2 Adds MrBayes 3.2.5 to your environment mrbayes/3.2.5/serial/intel-2015-update2 Adds MrBayes 3.2.5 to your environment mrtrix/0.3.12/nogui Adds MRtrix3 0.3.12 to your environment. MRtrix3 provides a set of tools to perform analysis of diffusion MRI data, based around the concept of spherical deconvolution and probabilistic tractography. Note: mrview and shview cannot be run over a remote X11 connection so are not usable. mrtrix/0.3.16/gnu-4.9.2/nogui MRtrix provides a set of tools to perform various advanced diffusion MRI analyses, including constrained spherical deconvolution (CSD), probabilistic tractography, track-density imaging, and apparent fibre density. mrtrix/3.0.4/gnu-10.2.0/nogui Adds MRtrix 3.0.4 to your environment. mrtrix/3.0rc3/gnu-4.9.2/nogui Adds MRtrix 3.0RC3 to your environment. mstor/2013/gnu-4.9.2 MSTor is a program for calculating partition functions, free energies, enthalpies, entropies, and heat capacities of complex molecules including torsional anharmonicity. mumax/3.9.3 Adds Mumax 3.9.3 to your environment. mumax/3.10-bindist mumax3 is a GPU-accelerated micromagnetic simulation program. mummer/3.23/gnu-4.9.2 Adds MUMmer 3.23 to your environment. MUMmer is a system for rapidly aligning entire genomes, whether in complete or draft form. muscle/3.8.31 Adds MUSCLE 3.8.31 to your environment. mutect/1.1.7 Adds MuTect 1.1.7 to your environment. MuTect is a GATK-based variant caller specialized for somatic/cancer variants. namd/2.10/intel-2015-update2 Adds NAMD 2.10 to your environment namd/2.11/intel-2015-update2 Adds NAMD 2.11 to your environment namd/2.12/intel-2015-update2 Adds NAMD 2.12 to your environment namd/2.12/intel-2017-update1 Adds NAMD 2.12 to your environment namd/2.12/intel-2018-update3 Adds NAMD 2.12 to your environment namd/2.13/intel-2018-update3 Adds NAMD 2.13 to your environment namd/2.13/plumed/intel-2018-update3 Adds NAMD 2.13 to your environment namd/2.14/multicore-gpu NAMD is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. This is NAMD's Linux-x86_64-multicore-CUDA binary. namd/2.14/ofi-smp-gpu/intel-2019 Adds NAMD 2.14 OFI-SMP-GPU version to your environment. This version should be run using charmrun. namd/2.14/ofi-smp/intel-2019 Adds NAMD 2.14 OFI-SMP version to your environment. This version should be run using charmrun. namd/2.14/ofi/intel-2019 Adds NAMD 2.14 OFI version to your environment. This version should be run using charmrun. ncdu/2.2.1-bindist Ncdu is a disk usage analyzer with an ncurses interface. nco/4.5.0 Adds nco to your environment. nektar++/4.3.5-impi/intel-2017-update1 Adds Nektar++ Version 4.3.5 to your environment nektar++/4.3.5-ompi/gnu-4.9.2 Adds Nektar++ Version 4.3.5 to your environment ngsutils/0.5.9 Adds a set of python scripts for handling various NGS tasks to your environment. nighres/1.1.0b Adds Nighres to your environment. nonmem/7.3.0/gnu-4.9.2 Adds NONMEM 7.3.0 using GCC Fortran 4.9.2 to your environment. nonmem/7.3.0/intel-2015-update2 Adds NONMEM 7.3.0 using Intel Fortran 2015 to your environment. novocraft/3.04.06 Adds novocraft 3.04.06 to your environment. Novocraft is a set of tools for bioinformatics, including Novoalign for short-read mapping. nwchem/6.5-r26243/atlas/intel-2015-update2 Adds NWChem 6.5 revision 26243 to your PATH, creates symlink to global .nwchemrc. You may need to alter/remove any old ~/.nwchemrc. Built with Python 2.7 interface and ATLAS. Global .nwchemrc: /shared/ucl/apps/nwchem/6.5-r26243-atlas/intel-2015-update2.nwchemrc nwchem/6.5-r26243/intel-2015-update2 Adds NWChem 6.5 revision 26243 to your PATH, creates symlink to global .nwchemrc. You may need to alter/remove any old ~/.nwchemrc. Built with Python 2.7 interface and MKL with ScaLAPACK. Global .nwchemrc: /shared/ucl/apps/nwchem/6.5-r26243/intel-2015-update2/.nwchemrc nwchem/6.6-r27746/intel-2015-update2 Adds NWChem 6.6 revision 27746 patched 2016-01-20 to your PATH, creates symlink to global .nwchemrc. You may need to alter/remove any old ~/.nwchemrc. Built with Python 2.7 interface and MKL with ScaLAPACK. Global .nwchemrc: /shared/ucl/apps/nwchem/6.6-r27746/intel-2015-update2/.nwchemrc nwchem/6.6-r27746/intel-2017 Adds NWChem 6.6 revision 27746 patched 2016-01-20 to your PATH, creates symlink to global .nwchemrc. You may need to alter/remove any old ~/.nwchemrc. Built with Python 2.7 interface and MKL with ScaLAPACK. Global .nwchemrc: /shared/ucl/apps/nwchem/6.6-r27746/intel-2017/.nwchemrc nwchem/6.8-47-gdf6c956/intel-2017 Adds NWChem 6.8 47-gdf6c956 to your PATH, creates symlink to global .nwchemrc. You may need to alter/remove any old ~/.nwchemrc. Built with Python 2.7 interface and MKL with ScaLAPACK. Global .nwchemrc: /shared/ucl/apps/nwchem/6.8-47-gdf6c956/intel-2017/.nwchemrc oasislmf/1.2.4 Oasis LMF 1.2.4 oasislmf/1.26.3 Oasis LMF 1.26.3 oasislmf/ktools/3.0.3/gnu-4.9.2 OasisLMF ktools package built with the GNU compilers oasislmf/ktools/3.4.1/gnu-4.9.2 OasisLMF ktools package built with the GNU compilers oasislmf/ktools/3.9.5/gnu-4.9.2 OasisLMF ktools package built with the GNU compilers oasislmf/ktools/f92a41f/gnu-4.9.2 OasisLMF ktools package built with the GNU compilers octave/4.4.1 Octave is an open source competitor to Matlab which is mostly compatible with Matlab. octopus/4.1.2-impi/intel-2015-update2 Adds octopus 4.1.2 to your environment. octopus/4.1.2/intel-2015-update2 Adds octopus 4.1.2 to your environment. octopus/5.0.1-ompi/gnu-4.9.2 Adds octopus 5.0.1 to your environment. octopus/5.0.1/gnu-4.9.2 Adds octopus 5.0.1 to your environment. octopus/6.0-ompi/gnu-4.9.2 Adds octopus 6.0 to your environment. octopus/6.0/gnu-4.9.2 Adds octopus 6.0 to your environment. openbabel/2.4.1/gnu-4.9.2 OpenBabel is a library and command-line tool for manipulating and converting between various chemistry file formats. opencv/2.4.13/gnu-4.9.2 Adds OpenCV 2.4.13 to your environment. Open Source Computer Vision Library. opencv/3.4.1/gnu-4.9.2 Adds OpenCV 3.4.1 to your environment. Open Source Computer Vision Library. openfoam/2.3.1/intel-2015-update2 Adds OpenFOAM 2.3.1 to your environment openfoam/2.4.0/intel-2017-update1 Adds OpenFOAM 2.4.0 to your environment openfoam/7.20200120/gnu-7.3.0 Adds OpenFOAM 7 to your environment openfoamplus/v1706/gnu-4.9.2 Adds OpenFOAMplus v1706 to your environment openfoamplus/v1906/gnu-7.3.0 Adds OpenFOAMplus v1906 to your environment openfoamplus/v1906/gnu-7.3.0-64 Adds OpenFOAMplus v1906 with 64 bit labels to your environment openfoamplus/v2112/gnu-7.3.0-64 Adds OpenFOAMplus v2112 with 64 bit labels to your environment openmm/7.3.1/cuda-10 Adds OpenMM to your environment. openmm/7.3.1/gnu-4.9.2 Adds OpenMM to your environment. openmx/3.8.3 Adds OpenMX 3.8.3 to your environment. optimet/1.0.1/gnu-4.9.2 Adds Optimet to your environment. orca/4.2.1-bindist/gnu-4.9.2 ORCA is an ab initio, DFT, and semiempirical SCF-MO package. orca/5.0.4-sbindist ORCA is an ab initio, DFT, and semiempirical SCF-MO package. This installation is the official statically-linked binary distribution. p7zip/15.09/gnu-4.9.2 Adds p7zip 15.09 to your environment. To expand 7z files: 7za x archive.7z p7zip/16.02/gnu-4.9.2 p7zip is a port of the command-line version of the 7-zip file compression tool to UNIX-like systems. pandoc/1.19.2.1 Adds pandoc Version 1.19.2.1 to your environment. parallel/20181122 GNU parallel is a shell tool for executing jobs in parallel using one or more computers. paraview/5.3.0 This module adds the ParaView 5.3.0 binaries to your environment. ParaView is an open-source, multi-platform data analysis and visualization application. paraview/5.10.1 This module adds the ParaView 5.10.1 binaries to your environment. ParaView is an open-source, multi-platform data analysis and visualization application. parmed/3.2.0 Adds ParmEd to your environment. petsc/3.12.1/gnu-4.9.2 Adds Petsc 3.12.1 to your environment phon/1.39/gnu-4.9.2 Adds Phon 1.3.9 with addons to your PATH. phon/1.43/gnu-4.9.2 Adds Phon 1.43 with addons to your PATH. picard-tools/1.136 Adds Picard Tools 1.136 to your environment. If using the java -jar command, you should pass TMP_DIR=$TMPDIR to Picard. picard-tools/2.18.9 Adds Picard Tools to your environment. If using the java -jar command, you should pass TMP_DIR=$TMPDIR to Picard. platypus/3e72641 Adds Platypus to your environment. plink/1.07 Adds Plink 1.07 with addons to your PATH. plink/1.90b3.40 Adds PLINK 1.90b3.40 to your environment. A comprehensive update to the PLINK association analysis toolset. plink/2.0alpha-git Adds PLINK 2.0 alpha to your environment. A comprehensive update to the PLINK association analysis toolset. plumed/2.1.2/intel-2015-update2 Adds PLUMED 2.1.2 to your environment, built using OpenBLAS plumed/2.2.3/intel-2015-update2 Adds PLUMED 2.2.3 to your environment, built using OpenBLAS and libmatheval plumed/2.2/intel-2015-update2 Adds PLUMED 2.2 to your environment, built using OpenBLAS plumed/2.3.1/intel-2017-update1 Adds PLUMED 2.3.1 to your environment, built using OpenBLAS and libmatheval plumed/2.4.1/gnu-4.9.2 Adds PLUMED 2.4.1 to your environment, built using OpenBLAS and libmatheval plumed/2.4.1/intel-2017-update4 Adds PLUMED 2.4.1 to your environment, built using OpenBLAS and libmatheval plumed/2.4.3/intel-2018 Adds PLUMED 2.4.3 to your environment, built using MKL and libmatheval plumed/2.5.2/intel-2018 Adds PLUMED 2.5.2 to your environment, built using MKL and libmatheval plumed/2.6.0/intel-2018 Adds PLUMED 2.6.0 to your environment, built using MKL and libmatheval plumed/2.7.2/gnu-10.2.0 Adds PLUMED 2.7.2 to your environment, built using GCC, OpenBLAS and libmatheval. PLUMED is a plugin that works with a large number of molecular dynamics codes. plumed/2.7.2/intel-2020 Adds PLUMED 2.7.2 to your environment, built using MKL and libmatheval. PLUMED is a plugin that works with a large number of molecular dynamics codes. poppler/22.10.0/gnu-9.2.0 Adds Poppler 22.10.0 to your environment. postgres+postgis/9.5.3+2.2.2/gnu-4.9.2 Adds postgres+postgis 9.5.3+2.2.2 to your environment. PostgreSQL is a relational database, and PostGIS is a geographical information enhancement for PostgreSQL. postgresql/9.5.3/gnu-4.9.2 Adds postgresql 9.5.3 to your environment. PostgreSQL is a relational database. primer3/2.3.6 This module adds the primer3 package to your environment. probabel/0.4.5/gnu-4.9.2 Adds ProbABEL to your environment. proj.4/4.9.1 Adds the PROJ.4 Cartographic Projections library to your environment. proj.4/5.2.0 Adds the PROJ.4 Cartographic Projections library to your environment. proj.4/6.0.0 Adds the PROJ.4 Cartographic Projections library to your environment. proj.4/6.1.0 Adds the PROJ.4 Cartographic Projections library to your environment. proovread/2.13.11-8Jan2016-f6a856a Adds proovread 2.13.11-8Jan2016-f6a856a to your environment. f6a856a is the commit for this version. pymol/1.7.7.2 Adds PyMol to your environment. pymol/1.8.2.1 Adds PyMol to your environment. pyrosetta/release-73 Adds PyRosetta to your environment. pytorch/1.2.0/cpu Adds PyTorch 1.2.0 to your environment. pytorch/1.2.0/gpu Adds PyTorch 1.2.0 to your environment. pytorch/1.11.0/cpu Adds PyTorch 1.11.0 to your environment. pytorch/1.11.0/gpu Adds PyTorch 1.11.0 to your environment. qctool/2/beta/ba5eaa44a62f This module adds qctool v2 beta to your environment. quantum-espresso/5.2.0-impi/intel-2015-update2 Adds quantum-espresso 5.2.0 to your environment. quantum-espresso/6.1-impi/intel2017 Adds quantum-espresso 6.1 to your environment. quantum-espresso/6.3-impi/thermo_pw-1.0.9/intel-2018 Adds quantum-espresso 6.3 + thermo_pw 1.0.9 to your environment. quantum-espresso/6.4.1-impi/intel-2018 Adds quantum-espresso 6.4.1 to your environment. quantum-espresso/6.5-impi/intel-2018 Adds quantum-espresso 6.5 to your environment. quantum-espresso/6.5-impi/thermo_pw-1.2.1/intel-2018 Adds quantum-espresso 6.5 + thermo_pw 1.2.1 to your environment. quantum-espresso/7.0-impi/intel-2018 Adds quantum-espresso 7.0 to your environment. r/3.2.0-atlas/gnu-4.9.2 Adds R 3.2.0 and Bioconductor 3.2 to your environment. r/3.2.2-openblas/gnu-4.9.2 Adds R 3.2.2 and Bioconductor 3.2 to your environment. r/3.3.0-openblas/gnu-4.9.2 Adds R 3.3.0 and Bioconductor 3.3 to your environment. r/3.3.2-openblas/gnu-4.9.2 Adds R 3.3.2 and Bioconductor 3.4 to your environment. r/3.4.0-openblas/gnu-4.9.2 Adds R 3.4.0 and Bioconductor 3.5 to your environment. r/3.4.2-openblas/gnu-4.9.2 Adds R 3.4.2 and Bioconductor 3.6 to your environment. r/3.5.0-openblas/gnu-4.9.2 Adds R 3.5.0 and Bioconductor 3.7 to your environment. r/3.5.1-openblas/gnu-4.9.2 Adds R 3.5.1 and Bioconductor 3.7 to your environment. r/3.5.3-openblas/gnu-4.9.2 Adds R 3.5.3 and Bioconductor 3.8 to your environment. r/3.6.0-openblas/gnu-4.9.2 Adds R 3.6.0 and Bioconductor 3.9 to your environment. r/4.2.0-openblas/gnu-10.2.0 Adds R 4.2.0 and Bioconductor 3.15 to your environment. r/4.2.2-openblas/gnu-10.2.0 Adds R 4.2.2 and Bioconductor 3.16 to your environment. r/4.2.3-openblas/gnu-10.2.0 Adds R 4.2.3 and Bioconductor 3.16 to your environment. randfold/2.0/gnu-4.9.2 Adds randfold 2.0 to your environment. rclone/1.51.0 RClone is a command-line program intended to download and upload files from and to various storage services and providers. rclone/1.61.0 RClone is a command-line program intended to download and upload files from and to various storage services and providers. repast-hpc/2.1/gnu-4.9.2 Adds Repast HPC 2.1 compiled with GCC 4.9.2 and OpenMPI to your environment. root/5.34.30/gnu-4.9.2 Adds ROOT 5.34.30 to your environment. root/5.34.30/gnu-4.9.2-fftw-3.3.6 Adds ROOT 5.34.30 to your environment. root/5.34.36/gnu-4.9.2-fftw-3.3.6 Adds ROOT 5.34.36 to your environment. root/5.34.36/gnu-4.9.2-fftw-3.3.6-gsl-2.4 Adds ROOT 5.34.36 to your environment. root/6.04.00/gnu-4.9.2 Adds ROOT 6.04.00 to your environment. rosetta/2015.31.58019 Adds Rosetta 2015.31.58019 to your environment. rosetta/2015.31.58019-mpi Adds Rosetta 2015.31.58019 with MPI to your environment. rosetta/2018.48.60516 Adds Rosetta 2018.48.60516 serial version to your environment. rosetta/2018.48.60516-mpi Adds Rosetta 2018.48.60516 MPI version to your environment. rsem/1.2.31 Adds RSEM 1.2.31 to your environment. rvtests/2.1.0 Rvtests: Rare variant test software for next generation sequencing data sac/101.6a Adds SAC 101.6a to your environment. sac/102.0 Adds SAC 102.0 to your environment. salmon/1.9.0 Adds Salmon 1.9.0 to your environment. sambamba/0.6.7-bindist A tool for extracting information from SAM/BAM files. samblaster/0.1.24/gnu-4.9.2 samblaster is a program for marking duplicates in read-id grouped paired-end SAM files. samsrf/5.84/matlab.r2019b Adds the SamSrf Matlab toolbox to your environment samtools/0.1.19 This module adds the Samtools 0.1.19 package to your environment. samtools/1.2/gnu-4.9.2 Adds SAMtools 1.2 to your environment. Reading/writing/editing/indexing/viewing SAM/BAM/CRAM format. samtools/1.3.1/gnu-4.9.2 Adds SAMtools 1.3.1 to your environment. Reading/writing/editing/indexing/viewing SAM/BAM/CRAM format. samtools/1.9/gnu-4.9.2 Adds SAMtools 1.9 to your environment. Reading/writing/editing/indexing/viewing SAM/BAM/CRAM format. samtools/1.11/gnu-4.9.2 Tools for reading/writing/editing/indexing/viewing SAM/BAM/CRAM formatted data. sas/9.4-M6/64 Adds SAS 9.4 (9.04.01M6) 64 bit to your environment sas/9.4-m7/64 Adds SAS 9.4 (9.04.01M7) 64 bit to your environment sas/9.4/64 Adds SAS 9.4 64 bit to your environment sc/7.16 Adds sc 7.16 to your environment. siesta/4.0.1/intel-2017 Adds SIESTA 4.0.1 to your environment. siesta/5.0-dev/intel-2018 Adds SIESTA 5.0 Development Version to your environment. skewer/0.2.2 Adds skewer 0.2.2 to your environment. slim/4.0.1/gnu-4.9.2 This module adds SLiM 4.0.1 evolutionary simulation framework to your environment. smalt/0.7.6/gnu-4.9.2 Adds SMALT 0.7.6 to your environment. SMALT aligns DNA sequencing reads with a reference genome. Compiled with bambamc support for SAM/BAM input and BAM output. snpAD/0.3.4-bindist snpAD is an ancient DNA aware genotype caller. snptest/2.5.4-beta3 Adds SNPtest 2.5.4-beta3 to your environment. sod/3.2.7 Adds SOD 3.2.7 to your environment. SOD is a program that automates tedious data selection, downloading, and routine processing tasks in seismology. sod/3.2.10 Adds SOD 3.2.10 to your environment. SOD is a program that automates tedious data selection, downloading, and routine processing tasks in seismology. spaceranger/1.2.2 Adds Space Ranger 1.2.2 to your environment. Space Ranger is a set of analysis pipelines that process Visium spatial RNA-seq output and brightfield and fluorescence microscope images. spm/8/r6313/matlab.r2015a Adds SPM8 to your environment spm/12/jan2020/matlab.r2019b Adds SPM12 to your environment spm/12/r6470/matlab.r2015a Adds SPM12 to your environment spss/25 Adds SPSS 25 to your environment spss/26 Adds SPSS 26 to your environment sqlite/3.31.1/gnu-9.2.0 Adds SQLite Version 3.31.1 to your environment. sqlite/3.41.2 SQLite is a C-language library that implements a small, fast, self-contained, high-reliability, full-featured, SQL database engine. stacks/2.54/gnu-4.9.2 Stacks is a software pipeline for building loci from short-read sequences, such as those generated on the Illumina platform. Stacks was developed to work with restriction enzyme-based data, such as RAD-seq, for the purpose of building genetic maps and conducting population genomics and phylogeography. star-ccm+/9.06.011 Adds STAR-CCM+ and STAR-View to your environment. star-ccm+/11.04.010-R8 Adds STAR-CCM+ and STAR-View to your environment. star-ccm+/12.04.010 Adds STAR-CCM+ and STAR-View to your environment. star-ccm+/13.02.011 Adds STAR-CCM+ and STAR-View to your environment. star-ccm+/13.06.012 Adds STAR-CCM+ and STAR-View to your environment. star-ccm+/14.06.013 Adds STAR-CCM+ and STAR-View to your environment. star-cd/4.22.058 Adds STAR-CD 4.22.058 to your environment. star-cd/4.26.011 Adds STAR-CD 4.26.011 using Intel 2016 compiler suite to your environment. STAR-CD is a code for performing CFD simulations. It is designed for modelling fluid flow, heat transfer, mass transfer and chemical reactions. star-cd/4.26.022 Adds STAR-CD 4.26.022 using Intel 2016 compiler suite to your environment. STAR-CD is a code for performing CFD simulations. It is designed for modelling fluid flow, heat transfer, mass transfer and chemical reactions. star-cd/4.28.050 Adds STAR-CD 4.28.050 using Intel 2016 compiler suite to your environment. STAR-CD is a code for performing CFD simulations. It is designed for modelling fluid flow, heat transfer, mass transfer and chemical reactions. star/2.5.2a Adds STAR 2.5.2a to your environment. star/2.7.3a Adds STAR 2.7.3a to your environment. star/2.7.10b/sbindist The Spliced Transcripts Alignment to a Reference (STAR) package for RNA sequence alignment. This installation uses the statically-linked binaries by default. stata/14 Adds Stata/MP 14 to your environment. stata/15 Adds Stata/MP 15 to your environment. supermagic/1.2/intel-2017 Adds supermagic 1.2 to your environment. Supermagic is a simple MPI sanity test. taup/2.1.2 adds TauP 2.1.2 to your environment variables tensorflow/1.4.1/cpu Adds Tensorflow 1.4.1 to your environment. tensorflow/1.4.1/gpu Adds Tensorflow 1.4.1 to your environment. tensorflow/1.4.1/mkl Adds Tensorflow 1.4.1 to your environment. tensorflow/1.8.0/cpu Adds Tensorflow 1.8.0 to your environment. tensorflow/1.8.0/gpu Adds Tensorflow 1.8.0 to your environment. tensorflow/1.8.0/mkl Adds Tensorflow 1.8.0 to your environment. tensorflow/1.12.0/cpu Adds Tensorflow 1.12.0 to your environment. tensorflow/1.12.0/gpu Adds Tensorflow 1.12.0 to your environment. tensorflow/1.12.0/mkl Adds Tensorflow 1.12.0 to your environment. tensorflow/1.13.1/cpu Adds Tensorflow 1.13.1 to your environment. tensorflow/1.13.1/gpu Adds Tensorflow 1.13.1 to your environment. tensorflow/1.13.1/mkl Adds Tensorflow 1.13.1 to your environment. tensorflow/1.14.0/cpu Adds Tensorflow 1.14.0 to your environment. tensorflow/1.14.0/gpu Adds Tensorflow 1.14.0 to your environment. tensorflow/1.14.0/mkl Adds Tensorflow 1.14.0 to your environment. tensorflow/2.0.0/gpu-py37 Adds Tensorflow 2.0.0 to your environment. tensorflow/2.0.0/gpu-py37-cudnn75 Adds Tensorflow 2.0.0 to your environment. tensorflow/2.0.0/mkl-py37 Adds Tensorflow 2.0.0 to your environment. tensorflow/2.8.0/python-3.8.6 Adds Tensorflow 2.8.0 to your environment. tensorflow/2.11.0/cpu Adds Tensorflow 2.11.0 to your environment. tensorflow/2.11.0/gpu Adds Tensorflow 2.11.0 to your environment. tephra2/2.0/gnu-4.9.2 Adds Tephra2 version 2.0 to your environment. tephra2/normal/r149 Adds Tephra2 version r149 to your environment. tesseract/3.05.01 Adds Tesseract 3.05.01 to your environment. texinfo/5.2/gnu-4.9.2 Adds GNU texinfo 5.2 to your environment. texinfo/6.6/gnu-4.9.2 Adds GNU texinfo 6.6 to your environment. texlive/2014 Adds TeX Live 2014 to your environment. texlive/2015 Adds TeX Live 2015 to your environment. texlive/2019 Adds TeX Live 2019 to your environment. textract/1.5.0 Adds textract 1.5.0 to your environment. textract extracts text from a wide range of document types. tmux/2.2 This module adds the tmux 2.2 package to your environment. tmux/3.2a This module adds the tmux 3.2a package to your environment. tmux/3.3a This module adds the tmux 3.3a package to your environment. tophat/2.1.0 Adds Tophat 2.1.0 to your environment. tracer/1.6 Adds Tracer 1.6. tractor/3.2.5 Adds TractoR 3.2.5 to your environment. tree/1.7.0 Adds tree 1.7.0 to your environment. This shows your directory structure as a tree. trim_galore/0.4.1 Adds Trim Galore 0.4.1 to your environment. A wrapper tool around Cutadapt and FastQC to consistently apply quality and adapter trimming to FastQ files. trim_galore/0.6.10 Adds Trim Galore 0.6.10 to your environment. A wrapper tool around Cutadapt and FastQC to consistently apply quality and adapter trimming to FastQ files. trimmomatic/0.33 Adds Trimmomatic 0.33 to your environment. A flexible read trimming tool for Illumina NGS data. turbomole/6.4/mpi Adds turbomole 6.4 (using MPI) to your environment. turbomole/6.4/serial Adds turbomole 6.4 (serial) to your environment. turbomole/6.4/smp Adds turbomole 6.4 (using SMP) to your environment. turbomole/6.5/mpi Adds turbomole 6.5 (using MPI) to your environment. turbomole/6.5/serial Adds turbomole 6.5 (serial) to your environment. turbomole/6.5/smp Adds turbomole 6.5 (using SMP) to your environment. turbomole/6.6/mpi Adds turbomole 6.6 (using MPI) to your environment. turbomole/6.6/serial Adds turbomole 6.6 (serial) to your environment. turbomole/6.6/smp Adds turbomole 6.6 (using SMP) to your environment. ubpred/1-bin32dist UbPred is a random forest-based predictor of potential ubiquitination sites in proteins. udunits/2.2.19 Adds udunits to your environment. udunits/2.2.20/gnu-4.9.2 adds the UDUNITS-2 package to your environment. udunits/2.2.26/gnu-4.9.2 adds the UDUNITS-2 package to your environment. varscan/2.3.9 Adds VarScan v2.3.9 to your environment. VarScan is a platform-independent mutation caller for targeted, exome, and whole-genome resequencing data generated on Illumina, SOLiD, Life/PGM, Roche/454, and similar instruments. vasp/5.4.1-05feb16-p2/intel-2015-update2 The VASP Quantum Chemistry package, version 5.4.1-05feb16 with patches 1 and 2. vasp/5.4.1-24jun15-p2-vtst-r160/intel-2015-update2 Adds VASP 5.4.1 built with VTST r160 to your environment. vasp/5.4.1-24jun15-p08072015/intel-2015-update2 The VASP Quantum Chemistry package, version 5.4.1-24jun15 with patch 08072015. vasp/5.4.4-18apr2017-libbeef/intel-2017-update1 Adds VASP 5.4.4 with BEEF-vdW functionals to your environment. vasp/5.4.4-18apr2017-vtst-r178/intel-2017-update1 Adds VASP 5.4.4 built with VTST r178 to your environment. vasp/5.4.4-18apr2017/intel-2017-update1 Adds VASP 5.4.4 to your environment. vasp/5.4.4-18apr2017/intel-2019 Adds VASP 5.4.4 to your environment. Built with Intel 2019. vasp/6.1.2-01dec2020/intel-2019-update5 Adds VASP 6.1.2 to your environment. vasp/6.2.1-19Jan2022/intel-2019-update5 Adds VASP 6.2.1 to your environment. vasp/6.2.1-19Jan2022/nvidia-22.1-gpu Adds VASP 6.2.1 FOR GPUS ONLY to your environment. vasp/6.3.0-24Jan2022/intel-2019-update5 Adds VASP 6.3.0 to your environment. vasp/6.3.0-24Jan2022/nvidia-22.1-gpu Adds VASP 6.3.0 FOR GPUS ONLY to your environment. vcftools/0.1.15/gnu-4.9.2 Adds VCFtools version 0.1.15 to your environment. Tools for working with VCF files. velvet/1.2.10 Adds Velvet 1.2.10 to your environment. vep/95.0 Adds VEP 95.0 to your environment. vesta/3.4.6-bindist VESTA is a 3D visualization program for structural models, volumetric data such as electron/nuclear densities, and crystal morphologies. vg/1.11.0 This is a module with no description string. viennarna/2.1.9/gnu-4.9.2 Adds viennarna 2.1.9 to your environment. vinalc/1.1.2/gnu-4.9.2 Adds VinaLC to your environment. visit/2.9.2 This package adds VisIt 2.9.2 to your environment. VisIt is a distributed, parallel visualization and graphical analysis tool for data defined on two- and three-dimensional (2D and 3D) meshes. Visit will create a ~/.visit directory and a ~/Scratch/visit directory. Jobfiles created by the GUI will go in the latter. Legion hostfile: /shared/ucl/apps/visit/2.9.2/gnu-4.9.2/host_legion.xml Web: https://wci.llnl.gov/simulation/computer-codes/visit/ vmd/1.9.3/GL+CUDA VMD is a molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3-D graphics and built-in scripting. vmd/1.9.3/text-only The binary, text only version of VMD 1.9.3 vmd/1.9.4/a55-gl+cuda Adds VMD 1.9.4 Development Version a55 to your environment. vt/2018-08-01/gnu-4.9.2 [ref:f6d2b5dab73c] A tool set for short variant discovery in genetic sequence data. xmds/2.2.2 Adds XMDS 2.2.2 (GNU/ATLAS/Intel MPI/FFTW toolchain) to your environment. xmds/3.0.0 Adds XMDS 3.0.0 (GNU/ATLAS/Intel MPI/FFTW toolchain) to your environment. xtalopt/r12.1/gnu-4.9.2 Adds XtalOpt r12.1 to your environment. xulrunner/3.6.28/gnu-4.9.2 Adds XULRunner 3.6.28 to your environment. XULRunner is a Mozilla runtime package that can be used to bootstrap XUL+XPCOM applications. This version was built including javaxpcom. xulrunner/10.0.2 Adds the XULRunner 3.6.28 64-bit runtime binaries to your environment. XULRunner is a Mozilla runtime package that can be used to bootstrap XUL+XPCOM applications. yambo/4.1.4/intel-2017 This is a module with no description string."},{"location":"Installed_Software_Lists/module-packages/#libraries","title":"Libraries","text":"

Modules in this section set up your environment to use specific C, C++, or Fortran libraries. This can include being able to use them with other languages, like Python.

Module Description apr-util/1.5.4 adds APR-util 1.5.4 to your environment variables apr-util/1.6.1 adds APR-util 1.6.1 to your environment variables apr/1.5.2 adds APR 1.5.2 to your environment variables apr/1.7.0 adds APR 1.7.0 to your environment variables argtable/2.13 Adds argtable 2.13 to your environment. armadillo/7.400.3/intel-2015-update2 Adds armadillo 7.400.3 to your environment. Armadillo is a linear alebra library for C++, aiming to balance speed and ease of use. arpack-ng/3.4.0/intel-2015-update2 Adds arpack-ng 3.4.0 to your environment. ARPACK-NG is a collection of Fortran77 subroutines designed to solve large scale eigenvalue problems. arpack-ng/3.5.0/gnu-4.9.2-serial Adds arpack-ng 3.5.0 to your environment. ARPACK-NG is a collection of Fortran77 subroutines designed to solve large scale eigenvalue problems. arpack-ng/3.5.0/gnu-4.9.2-threaded Adds arpack-ng 3.5.0 to your environment. ARPACK-NG is a collection of Fortran77 subroutines designed to solve large scale eigenvalue problems. arpack-ng/3.5.0/intel-2017 Adds arpack-ng 3.5.0 to your environment. ARPACK-NG is a collection of Fortran77 subroutines designed to solve large scale eigenvalue problems. atlas/3.10.2/gnu-4.9.2 adds ATLAS 3.10.2 for GCC 4.9.2 compilers to your environment variables atlas/3.10.2/intel-2015-update2 adds ATLAS 3.10.2 for Intel 15 compilers to your environment variables bambamc/0.0.50/gnu-4.9.2 Adds bambamc 0.0.50 to your environment. bambamc is a lightweight C implementation of the read name collation code from the larger libmaus/biobambam C++ project. boost/1.75.0/gnu-4.9.2 Boost is a collection of miscellaneous C++ libraries. This build does not include Python bindings or MPI support, and is multi-threaded. boost/1_54_0/gnu-4.9.2 Adds Boost 1.54.0 with Python libraries to your environment. boost/1_54_0/mpi/gnu-4.9.2 Adds Boost 1.54.0 with Python and MPI libraries to your environment. boost/1_54_0/mpi/gnu-4.9.2-ompi-1.10.1 Adds Boost 1.54.0 with Python and MPI libraries to your environment. boost/1_54_0/mpi/intel-2015-update2 Adds Boost 1.54.0 with Python and MPI libraries to your environment. boost/1_63_0/gnu-4.9.2 Adds Boost 1.63.0 with Python libraries to your environment. boost/1_63_0/mpi/gnu-4.9.2 Adds Boost 1.63.0 with Python and MPI libraries to your environment. boost/1_63_0/mpi/intel-2017-update1 Adds Boost 1.63.0 with Python and Intel MPI libraries to your environment. cernlib/2006-35 Adds the CERN Program library to your environment. 2006-35 EL6 RPM binaries. cernlib/2006/gnu-4.9.2 Adds the CERN Program library to your environment cfitsio/3370/gnu-4.9.2 Adds cfitsio 3370 to your environment. cfitsio/3370/intel-2015-update2 Adds cfitsio 3370 to your environment. cgal/4.9/gnu-4.9.2 Adds CGAL 4.9 with Qt5 to your environment. The Computational Geometry Algorithms Library. clusteringsuite/2.6.6/bindist Adds clusteringsuite 2.6.6 to your environment. Clustering Suite is a set of tools to automatically expose the main performance trends in applications' computation structure. cppunit/1.15.1/gnu-4.9.2 Adds CppUnit 1.15.1 to your environment. CppUnit is a package for writing and running unit tests in C++. cppunit/1.15.1/gnu-10.2.0 Adds CppUnit 1.15.1 built wuth GNU 10.2.0 to your environment. CppUnit is a package for writing and running unit tests in C++. cunit/2.1-3/gnu-4.9.2 Adds cunit 2.1-3 to your environment. CUnit is a package for writing and running unit tests in C. cvmfs/2.2.1/gnu-4.9.2 Adds libcvmfs 2.2.1 to your environment. dyninst/9.3.2/gnu-4.9.2 Adds dyninst 9.3.2 to your environment. DynInst is a library for performing dynamic instrumentation of executables. eigen/3.2.5/gnu-4.9.2 adds Eigen for GCC 4.9.2 compilers to your environment variables eigen/3.3.9/gnu-4.9.2 Adds Eigen 3.3.9 to your environment. Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms. eigen/3.4.0/gnu-10.2.0 Adds Eigen 3.4.0 to your environment. Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms. elfutils/0.170/gnu-4.9.2 Adds elfutils 0.170 to your environment. Elfutils provides utilities for manipulating binary ELF files, and is one possible provider of libelf. fftw/2.1.5/gnu-4.9.2 adds FFTW 2.1.5 for GCC 4.9.2 compilers to your environment variables fftw/2.1.5/intel-2015-update2 adds FFTW 2.1.5 for Intel compilers to your environment variables fftw/3.3.4-impi/gnu-4.9.2 Adds fftw 3.3.4 (built with Intel MPI) to your environment. fftw/3.3.4-impi/intel-2017-update1 Adds fftw 3.3.4 (built with Intel MPI) to your environment. fftw/3.3.4-ompi-1.10.1/gnu-4.9.2 Adds fftw 3.3.4 (built with OpenMPI) to your environment. fftw/3.3.4-ompi/gnu-4.9.2 Adds fftw 3.3.4 (built with OpenMPI) to your environment. fftw/3.3.4-threads/gnu-4.9.2 adds FFTW 3.3.4 for GCC 4.9.2 compilers to your environment variables fftw/3.3.4/gnu-4.9.2 adds FFTW 3.3.4 for GCC 4.9.2 compilers to your environment variables fftw/3.3.4/intel-2015-update2 adds FFTW 3.3.4 for Intel compilers to your environment variables fftw/3.3.6-pl2/gnu-4.9.2 Adds FFTW 3.3.6 pl2 for GCC 4.9.2 compilers to your environment variables. Includes single and double precision only on Legion, plus long-double and quad on Grace/Thomas. Includes OpenMP and POSIX threads libraries. fftw/3.3.6-pl2/intel-2017 Adds FFTW 3.3.6 pl2 for Intel compilers to your environment variables. Includes single and double precision versions on Legion, plus long-double on Grace/Thomas. Includes OpenMP and POSIX threads libraries. fftw/3.3.8-impi/intel-2018 Adds fftw fftw (built with Intel MPI) to your environment. fftw/3.3.8-ompi/gnu-4.9.2 Adds fftw 3.3.8 (built with OpenMPI) to your environment. fftw/3.3.10/nvidia-22.1 Adds fftw 3.3.10 to your environment. forge/1.0.0/gnu-4.9.2 Adds forge 1.0.0 to your environment. freeimage/3.17.0/gnu-4.9.2 Adds FreeImage 3.17.0 to your environment. freetype/2.8.1/gnu-4.9.2 Adds freetype 2.8.1 to your environment. FreeType is a freely available software library to render fonts. ga/5.7-8BInts/intel-2018 Global Arrays (GA) is a library that provides a Partitioned Global Address Space (PGAS) programming model. This version has been compiled with 8-byte integers in the Fortran code. ga/5.7/intel-2018 Global Arrays (GA) is a library that provides a Partitioned Global Address Space (PGAS) programming model. gcc-libs/4.9.2 adds GCC 4.9.2 runtime to your evironment. geos/3.5.0/gnu-4.9.2 Adds geos 3.5.0 to your environment. GEOS (Geometry Engine, Open Source) is a library for performing various spatial operations, especially for boolean operations on GIS data. Note this version does not include the SWIG, Python, Ruby, or PHP bindings. gflags/2.2.1 Adds Google gflags 2.2.1 to your environment. giflib/5.1.1 Adds giflib 5.1.1 to your environment. A library and utilities for processing gifs. glbinding/2.1.2/gnu-4.9.2 Adds glbinding 2.1.2 to your environment. glew/1.13.0/gnu-4.9.2 Adds GLEW The OpenGL Extension Wrangler Library 1.11.0 to your environment. glew/2.1.0/gnu-4.9.2 Adds GLEW -- The OpenGL Extension Wrangler Library -- 2.1.0 to your environment. glfw/3.2.1/gnu-4.9.2 Adds GLFW 3.2.1 to your environment. glog/0.3.5 Adds Google glog 0.3.5 to your environment. glpk/4.60/gnu-4.9.2 Adds the GNU Linear Programming Kit Version 4.60 for GCC 4.9.2 to your environment. gsl/1.16/gnu-4.9.2 adds GSL 1.16 for GCC 4.9.2 to your environment. gsl/1.16/intel-2015-update2 Adds gsl 1.16 to your environment. gsl/2.4/gnu-4.9.2 adds GSL 2.4 for GCC 4.9.2 to your environment. gsl/2.4/intel-2017 adds GSL 2.4 for Intel 2017 to your environment. gstreamer/1.12.0 GStreamer is a library for constructing graphs of media-handling components, including codecs for various audio and video formats. gulp/4.5/libgulp/intel-2018 Adds GULP 4.5 library version to your environment. Built libgulp only, without FoX, for programs such as ChemShell to link. GULP is a materials simulation code. h5py/2.10.0-ompi/gnu-4.9.2 Adds h5py 2.10.0-ompi for Python 3.7 to your environment. harminv/1.4.1/gnu-4.9.2 Adds harminv 1.4.1 to your environment. harminv/1.4/gnu-4.9.2 Adds harminv 1.4 to your environment. hdf/5-1.8.15-p1-impi/intel-2015-update2 Adds hdf5 1.8.5-p1 (built with Fortran and IntelMPI options) to your environment. hdf/5-1.8.15-p1-ompi/gnu-4.9.2 Adds hdf5 1.8.5-p1 (built with Fortran and OpenMPI options) to your environment. hdf/5-1.8.15/gnu-4.9.2 adds HDF5 1.8.15 (Serial) for GCC 4.9.2 to your environment. hdf/5-1.8.15/intel-2015-update2 adds HDF5 1.8.15 (Serial) for Intel 2015 to your environment. hdf/5-1.10.2-impi/intel-2018 adds HDF5 1.10.2 (Parallel) for Intel 2018 to your environment. hdf/5-1.10.2/intel-2018 adds HDF5 1.10.2 (Serial) for Intel 2018 to your environment. hdf/5-1.10.5-ompi/gnu-4.9.2 Adds hdf 5-1.10.5-ompi to your environment. Built with OpenMPI and GNU. hdf/5-1.10.5/gnu-4.9.2 Adds hdf 5-1.10.5 to your environment. Serial version built with GNU. htslib/1.2.1 This module adds the HTSlib 1.2.1 package to your environment. HTSlib is an implementation of a unified C library for accessing common file formats, such as SAM, CRAM and VCF, used for high-throughput sequencing data. htslib/1.3.1 This module adds the HTSlib 1.3.1 package to your environment. HTSlib is an implementation of a unified C library for accessing common file formats, such as SAM, CRAM and VCF, used for high-throughput sequencing data. htslib/1.7 This module adds the HTSlib 1.7 package to your environment. HTSlib is an implementation of a unified C library for accessing common file formats, such as SAM, CRAM and VCF, used for high-throughput sequencing data. htslib/1.11/gnu-4.9.2 A C library for reading/writing high-throughput sequencing data. hwloc/1.11.12 The Portable Hardware Locality (hwloc) software package provides a portable abstraction (across OS, versions, architectures, ...) of the hierarchical topology of modern architectures, including NUMA memory nodes, sockets, shared caches, cores and simultaneous multithreading. This installation includes the optional libnuma dependency. hypre/2.11.2/openmpi-3.0.0/intel-2017 Adds HYPRE 2.11.2 to your environment. hypre/2.11.2/openmpi-3.1.1/intel-2018 Adds HYPRE 2.11.2 to your environment. ipopt/3.14.2/intel-2018 Ipopt (Interior Point OPTimizer, pronounced eye-pea-Opt) is a software package for large-scale nonlinear optimization. jansson/2.11 This is a module with no description string. json-c/0.12/gnu-4.9.2 Adds json-c 0.12 to your environment. JSON-C is a library for converting between JSON-formatted strings and C representations of the equivalent objects. lapack/3.8.0/gnu-4.9.2 LAPACK is a reference library of routines for Linear Algebra. It is not recommended for use, as its ABI is replicated in the much higher-performance libraries OpenBLAS, MKL, or ATLAS instead. leptonica/1.74.4 Adds Leptonica 1.74.4 to your environment. leveldb/1.20 Adds Google leveldb 1.20 to your environment. libbdwgc/7.4.2/gnu-4.9.2 Adds libbdwgc (a garbage-collector library) to your environment. libbeef/0.1.3/intel-2018 Library for Bayesian error estimation functionals for use in density functional theory codes: libbeef 0.1.3 commit 2822afe libctl/3.2.2/gnu.4.9.2 Adds libctl (built using Intel compilers) to your environment. libctl/4.3.0/gnu-4.9.2 Adds libctl 4.3.0 to your environment. libdwarf/20170709/gnu-4.9.2 Adds libdwarf 20170709 to your environment. libdwarf is a library for interacting with debugging info in the DWARF 2, 3, and 5 formats. libelf/0.8.13/gnu-4.9.2 Adds libelf 0.8.13 to your environment. libetsfio/1.0.4/gnu-4.9.2 Adds libetsfio 1.0.4 to your environment. libetsfio/1.0.4/intel-2015-update2 Adds libetsfio 1.0.4 to your environment. libflac/1.3.1/gnu-4.9.2 Adds libflac 1.3.1 to your environment. libFLAC is the Xiph library for handling their lossless audio codec. libgd/2.1.1/gnu-4.9.2 Adds libgd 2.1.1 to your environment. libgd/2.1.1/intel-2015-update2 Adds libgd 2.1.1 to your environment. libgdsii/0.21/gnu-4.9.2 Adds libgdsii 0.21 to your environment. C++ library and command-line utility for reading GDSII geometry files. libint/1.1.4/gnu-4.9.2 Adds libint 1.1.4 to your environment. Libint is required for CP2K. libmatheval/1.1.11 Adds libmatheval 1.1.11 to your environment. GNU libmatheval is a library (callable from C and Fortran) to parse and evaluate symbolic expressions input as text. libsodium/1.0.6/gnu-4.9.2 Adds libsodium 1.0.6 to your environment. libsodium is a crypto library primarily used by ZeroMQ. libsox/14.4.2/gnu-4.9.2 Adds libsox 14.4.2 to your environment. SoX is a library for reading, writing, and converting a variety of sound file formats. If you require support for a file format that is not installed, contact rc-support and the library can be rebuilt. libuuid/1.0.3/gnu-4.9.2 Adds a static libuuid 1.0.3 to your environment. libxc/2.1.2/intel-2015-update2 Adds libxc 2.1.2 to your environment. libxc/2.2.2/gnu-4.9.2 Adds libxc 2.2.2 to your environment. libxc/2.2.2/intel-2015-update2 Adds libxc 2.2.2 to your environment. libxc/3.0.0/gnu-4.9.2 Adds libxc 3.0.0 to your environment. libxc/3.0.0/intel-2015-update2 Adds libxc 3.0.0 to your environment. libxc/4.2.3/intel-2018 libxc is a library of routines implementing a range of exchange-correlation functionals for density-functional theory calculations. libxml2/2.9.4/gnu-4.9.2 Adds libxml2 2.9.4 to your environment. Libxml2 is an XML C parser and toolkit. Includes Python (2.7.9) bindings. llvm/3.3 This module adds the LLVM 3.3 package to your environment. The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. The LLVM Core libraries provide a modern source- and target-independent optimizer, along with code generation support for many popular CPUs. llvm/3.9.1 This module adds the LLVM 3.9.1 package to your environment. The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. The LLVM Core libraries provide a modern source- and target-independent optimizer, along with code generation support for many popular CPUs. llvm/6.0.1 This module adds the LLVM 6.0.1 package to your environment. The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. The LLVM Core libraries provide a modern source- and target-independent optimizer, along with code generation support for many popular CPUs. llvm/8.0.0/gnu-4.9.2 The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. This installation includes clang, a C compiler based on LLVM. lmdb/0.9.22 Adds LMDB 0.9.22 to your environment. lz4/1.8.3 This is a module with no description string. magma/2.4.0 This is a module with no description string. med/4.0.0/gnu-4.9.2 Adds med 4.0.0 to your environment. Allows reading and writing of MED format files. mesa/6.5/gnu-4.9.2 Adds mesa 6.5 to your environment. Mesa is an open-source implementation of the OpenGL specification - a system for rendering interactive 3D graphics. This is an old version installed to satisfy a particular dependency: please do not use for new builds. mesa/10.6.3 Adds Mesa 10.6.3 to your environment. Mesa is an open-source implementation of the OpenGL specification. (Built for offscreen rendering: OSMesa, Xlib GLX, no Gallium, no EGL, no llvm, no DRI). mesa/10.6.9/gnu-4.9.2 Adds Mesa 10.6.9 to your environment. Mesa is an open-source implementation of the OpenGL specification. (Built for offscreen rendering: OSMesa, Xlib GLX, no Gallium, no EGL, no llvm, no DRI). mesa/13.0.6/gnu-4.9.2 Adds Mesa 13.0.6 to your environment. Mesa is an open-source implementation of the OpenGL specification. (Built options: Gallium, LLVM, no EGL, no DRI, no GLX). The default driver is llvmpipe. You can use \"export GALLIUM_DRIVER\" to explicitly choose llvmpipe, softpipe, or swr metis/5.1.0/gnu-4.9.2 Adds metis 5.1.0 to your environment. METIS is a set of serial programs for partitioning graphs, partitioning finite element meshes, and producing fill reducing orderings for sparse matrices. metis/5.1.0/intel-2015-update2 Adds metis 5.1.0 to your environment. METIS is a set of serial programs for partitioning graphs, partitioning finite element meshes, and producing fill reducing orderings for sparse matrices. metis/5.1.0/intel-2018 Adds metis 5.1.0 to your environment. METIS is a set of serial programs for partitioning graphs, partitioning finite element meshes, and producing fill reducing orderings for sparse matrices. mpi/intel/2013/update1/intel adds Intel MPI 4.1.3.048 to your environment variables mpi/intel/2015/update3/gnu-4.9.2 adds Intel MPI to your environment variables mpi/intel/2015/update3/intel adds Intel MPI to your environment variables mpi/intel/2017/update1/gnu-4.9.2 adds Intel MPI to your environment variables mpi/intel/2017/update1/intel adds Intel MPI to your environment variables mpi/intel/2017/update2/gnu-4.9.2 adds Intel MPI to your environment variables configured to use GCC 4.9.2 mpi/intel/2017/update2/intel adds Intel MPI to your environment variables mpi/intel/2017/update3/gnu-4.9.2 [Intel MPI/2017.3.196] This is Intel's MPI implementation, version 2017.3.196, which is bundled with compiler package version 2017.Update4. This module sets up the compiler wrappers to use GCC 4.9.2 underneath. mpi/intel/2017/update3/intel [Intel MPI/2017.3.196] This is Intel's MPI implementation, version 2017.3.196, which is bundled with compiler package version 2017.Update4. mpi/intel/2018/update3/intel [Intel MPI/2018.3.222] This is Intel's MPI implementation, version 2018.3.222, which is bundled with compiler package version 2018.Update3. mpi/intel/2019/update4/intel [Intel MPI/2019.4.243] This is Intel's MPI implementation, version 2019.4.243, which is bundled with compiler package version 2019.Update4. mpi/intel/2019/update5/intel [Intel MPI/2019.5.281] This is Intel's MPI implementation, version 2019.5.281, which is bundled with compiler package version 2019.Update5. mpi/intel/2019/update6/intel [Intel MPI/2019.6.166] This is Intel's MPI implementation, version 2019.6.166, which is bundled with compiler package version 2020. mpi/intel/2021.6.0/intel [Intel MPI/2021.6.0] This is Intel's MPI implementation, version 2021.6.0, which is bundled with compiler package version 2022.2. mpi/openmpi/1.8.4/gnu-4.9.2 adds OpenMPI 1.8.4 for GCC 4.9.2 compilers to your environment variables mpi/openmpi/1.8.4/intel-2015-update2 adds OpenMPI 1.8.4 for Intel 2015 update 2 compilers to your environment variables mpi/openmpi/1.10.1/gnu-4.9.2 adds OpenMPI 1.10.1 for GCC 4.9.2 compilers to your environment variables mpi/openmpi/1.10.1/intel-2015-update2 adds OpenMPI 1.10.1 for Intel 2015 update 2 compilers to your environment variables mpi/openmpi/2.0.2/gnu-4.9.2 Adds openmpi 2.0.2 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/2.0.2/intel-2017 Adds openmpi 2.0.2 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/2.1.2/gnu-4.9.2 Adds openmpi 2.1.2 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/2.1.2/intel-2017 Adds openmpi 2.1.2 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.0.0/gnu-4.9.2 Adds openmpi 3.0.0 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.0.0/intel-2017 Adds openmpi 3.0.0 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.1.1/gnu-4.9.2 Adds openmpi 3.1.1 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.1.1/intel-2018 Adds openmpi 3.1.1 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.1.4/gnu-4.9.2 Adds openmpi 3.1.4 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.1.4/intel-2018 Adds openmpi 3.1.4 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.1.6/gnu-4.9.2 Adds openmpi 3.1.6 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/4.0.3/gnu-4.9.2 Adds openmpi 4.0.3 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/4.1.1/gnu-4.9.2 Adds openmpi 4.1.1 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. Note: we set OMPI_MCA_btl=vader on OmniPath clusters for this version. As a result it will work multi-node but be slower than other transports. mpi4py/2.0.0/python2 Adds Python2 mpi4py 2.0.0 to your environment. MPI for Python. mpi4py/2.0.0/python3 Adds Python3 mpi4py 2.0.0 to your environment. MPI for Python. mpi4py/3.0.0/python3 Adds Python3 mpi4py 3.0.0 to your environment. MPI for Python. mpi4py/3.0.2/gnu-4.9.2 Adds mpi4py 3.0.2 for Python 3.7 to your environment. mpi4py/3.1.4/gnu-4.9.2 Adds mpi4py 3.1.4 for Python 3.9 to your environment. multinest/3.10b/gnu-4.9.2 Adds MultiNest 3.10b to your environment. mumps-thirdparty/3.0.0/intel-2018 The COIN-OR Tools project ThirdParty-Mumps can be used to download the MUMPS code and build a MUMPS library that is recognized by Ipopt. This version of ThirdParty-Mumps retrieves and builds MUMPS 5.4.0. mumps/5.2.1/intel-2018 Adds mumps 5.2.1 to your environment. Sequential version built with Intel and METIS. mysql-connector-python/2.0.4/python-3.5.2 Adds mysql-connector-python 2.0.4 to your environment. This is Oracle's python-only MySQL connector for Python3 mysql-connector-python/2.0.4/python-3.6.3 Adds mysql-connector-python 2.0.4 to your environment. This is Oracle's python-only MySQL connector for Python3 mysql-connector-python/2.0.4/python-3.7.4 Adds mysql-connector-python 2.0.4 to your environment. This is Oracle's python-only MySQL connector for Python3 mysql-connector-python/2.0.4/python-3.8.0 Adds mysql-connector-python 2.0.4 to your environment. This is Oracle's python-only MySQL connector for Python3 mysql-connector-python/8.0.22/python-3.8.6 Adds mysql-connector-python 8.0.22 to your environment. This is Oracle's python-only MySQL connector for Python3 mysql-connector-python/8.0.22/python-3.9.0 Adds mysql-connector-python 8.0.22 to your environment. This is Oracle's python-only MySQL connector for Python3 mysql-connector-python/8.0.22/python-3.9.6 Adds mysql-connector-python 8.0.22 to your environment. This is Oracle's python-only MySQL connector for Python3 mysql-connector-python/8.0.28/python-3.9.10 Adds mysql-connector-python 8.0.28 to your environment. This is Oracle's python-only MySQL connector for Python3 nag/fortran/mark22/gnu-4.9.2 Adds NAG Fortran Library Mark 22 for GCC to your environment. nag/fortran/mark24/gnu-4.9.2 Adds NAG Fortran Library Mark 24 for GCC to your environment. nag/fortran/mark24/nag-6.0.1044 Adds NAG Fortran Library Mark 24 for NAG Fortran to your environment. nag/fortran/mark25/intel-2015-update2 Adds NAG Fortran Library Mark 25 for Intel 2015 to your environment. nag/fortran/mark26/gnu-4.9.2 Adds NAG Fortran Library Mark 26 for GCC to your environment. nag/fortran/mark26/intel-2017 Adds NAG Fortran Library Mark 26 for Intel 2017 to your environment. nag/fortran/mark26/nag-6.1.6106 Adds NAG Fortran Library Mark 26 for NAG Fortran to your environment. nag/fortran/mark26/nag-6.2.6223 Adds NAG Fortran Library Mark 26 for NAG Fortran to your environment. nag/mark27/intel-2019 Adds NAG Library Mark 27 for Intel 2019 to your environment. netcdf-c++/4.2/gnu-4.9.2 adds NetCDF C++ 4.2 for GCC to your environment. netcdf-c++/4.2/intel-2015-update2 adds NetCDF C++ 4.2 for Intel 2015 to your environment. netcdf-c++4/4.2/gnu-4.9.2 adds NetCDF C++ 4.2 for GCC to your environment. netcdf-c++4/4.2/intel-2015-update2 adds NetCDF C++ 4.2 for Intel 2015 to your environment. netcdf-fortran/4.4.1/gnu-4.9.2 adds NetCDF 4.4.1 for GCC to your environment. netcdf-fortran/4.4.1/intel-2015-update2 adds NetCDF 4.4.1 for Intel 2015 to your environment. netcdf-fortran/4.5.4/intel-2018-update3 adds NetCDF Fortran 4.5.4 for Intel 2018 to your environment. netcdf-fortran/4.6.1/gnu-10.2.0 adds NetCDF 4.6.1 for GCC to your environment. netcdf/4.3.3.1/gnu-4.9.2 adds NetCDF 4.3.3.1 for GCC 4.9.2 to your environment. netcdf/4.3.3.1/intel-2015-update2 adds NetCDF 4.3.3.1 for Intel 2015 to your environment. netcdf/4.9.0/intel-2018-update3 adds NetCDF 4.9.0 for Intel 2018 to your environment. netcdf/4.9.2/gnu-10.2.0 adds NetCDF 4.9.2 for GCC 10.2.0 to your environment. numactl/2.0.12 numactl provides NUMA policy support, as well as tools and a library to display NUMA allocation statistics and debugging information. openblas/0.2.14-threads/gnu-4.9.2 adds OpenBLAS 0.2.14 for GCC 4.9.2 compilers to your environment variables openblas/0.2.14/gnu-4.9.2 adds OpenBLAS 0.2.14 for GCC 4.9.2 compilers to your environment variables openblas/0.2.14/intel-2015-update2 adds OpenBLAS 0.2.14 for Intel 2015 update 2compilers to your environment variables openblas/0.3.2-native-threads/gnu-4.9.2 This is a module with no description string. openblas/0.3.2-openmp/gnu-4.9.2 This is a module with no description string. openblas/0.3.2-serial/gnu-4.9.2 This is a module with no description string. openblas/0.3.7-native-threads/gnu-4.9.2 This is a module with no description string. openblas/0.3.7-openmp/gnu-4.9.2 This is a module with no description string. openblas/0.3.7-serial/gnu-4.9.2 This is a module with no description string. openjpeg/2.4.0/gnu-4.9.2 OpenJPEG is an open-source JPEG 2000 codec written in C language. openslide/3.4.1/gnu-4.9.2 adds OpenSlide library 3.4.1 to your environment variables openssl/1.1.1t OpenSSL is a widely-used toolkit for general-purpose cryptography and secure communication. openssl/1.1.1u OpenSSL is a widely-used toolkit for general-purpose cryptography and secure communication. papi/5.5.1/gnu-4.9.2 Adds PAPI 5.5.1 to your environment. PAPI is a library for working with performance counters, often used in profiling applications. parmetis/4.0.3/intel-2015-update2 Adds parmetis 4.0.3 to your environment. ParMETIS is an MPI-based parallel library that implements a variety of algorithms for partitioning unstructured graphs, meshes, and for computing fill-reducing orderings of sparse matrices. pcre2/10.21/gnu-4.9.2 Adds pcre2 10.21 to your environment. PCRE (Perl-compatible regular expressions) is a C library implementing regular expression pattern-matching using the same semantics as Perl 5. pgplot/5.2.2/intel-2017 Adds PGPlot 5.2.2 to your environment. pgplot/5.2.2/intel-2018 Adds PGPlot 5.2.2 to your environment. pillow-simd/6.0.0.post0/python-3.7.4 Adds Pillow-SIMD to your environment. protobuf/3.5.1/gnu-4.9.2 adds Google Protocol Buffers for GCC 4.9.2 to your environment. protobuf/12-2017/gnu-4.9.2 adds Google Protocol Buffers for GCC 4.9.2 to your environment. psm2/11.2.185/gnu-4.9.2 Adds psm2 11.2.185 to your environment. The PSM2 messaging API, libpsm2. A low-level user-level communications interface for the Intel(R) OPA. family of products. pstreams/1.0.1/gnu-4.9.2 Adds pstreams 1.0.1 to your environment. PStreams is a C++ wrapper for process control and streaming using popen and pclose. pygsl/2.1.1-python3.6/gnu-4.9.2 Adds pygsl 2.1.1 to your environment. PyGSL provides Python bindings for the GNU Scientific Library. pyngl/1.4.0 Adds PyNGL to your environment. pynio/1.4.1 Adds PyNIO to your environment. quip/18c5440-threads/gnu-4.9.2 Adds QUIP to your environment. QUIP is required for CP2K. quip/18c5440/gnu-4.9.2 Adds QUIP to your environment. QUIP is required for CP2K. qutip/4.1.0/python-2.7.12 Adds qutip to your environment. scalapack/2.0.2/gnu-4.9.2/openblas Adds ScaLAPACK 2.0.2 to your environment, built with GCC, OpenBLAS and OpenMPI. Static and shared libraries. scalapack/2.0.2/gnu-4.9.2/openblas-0.3.2 Adds ScaLAPACK 2.0.2 to your environment, built with GCC, OpenBLAS and OpenMPI. Static and shared libraries. scalapack/2.0.2/gnu-4.9.2/openblas-0.3.7 Adds ScaLAPACK 2.0.2 to your environment, built with GCC, OpenBLAS and OpenMPI. Static and shared libraries. snappy/1.1.7 Adds Google snappy 1.1.7 to your environment. spark/3.1.1-bin-hadoop2.7 Apache Spark is an analytics engine for data processing. sparskit2/2009.11.18/gnu-4.9.2 Adds sparskit2 2009.11.18 to your environment. sparskit2/2009.11.18/intel-2015-update2 Adds sparskit2 2009.11.18 to your environment. spectral/3.4.0/bindist Adds spectral 3.4.0 to your environment. Spectral is a set of tools for performing spectral analysis on traces produced by the BSC profiling toolkit. spglib/1.7.4/gnu-4.9.2 Adds spglib 1.7.4 to your environment. Spglib is a library for finding and handling crystal symmetries written in C. squid/1.9g/gnu-4.9.2 Adds squid 1.9g to your environment. suitesparse/4.5.5/gnu-4.9.2-serial Adds suitesparse 4.5.5 to your environment. SuiteSparse is a suite of sparse matrix algorithms. suitesparse/4.5.5/gnu-4.9.2-threaded Adds suitesparse 4.5.5 to your environment. SuiteSparse is a suite of sparse matrix algorithms. suitesparse/4.5.5/intel-2017-update1 Adds suitesparse 4.5.5 to your environment. SuiteSparse is a suite of sparse matrix algorithms. superlu-dist/5.1.0/intel-2015-update2 Adds superlu-dist 5.1.0 to your environment. SuperLU_DIST is the distributed-memory parallel version of SuperLU, a general purpose library for the direct solution of large, sparse, nonsymmetric systems of linear equations. superlu/5.2.1/intel-2015-update2 Adds superlu 5.2.1 to your environment. SuperLU is a general purpose library for the direct solution of large, sparse, nonsymmetric systems of linear equations. szip/2.1 Adds szip to your environment. ucx/1.8.0/gnu-4.9.2 Adds ucx 1.8.0 to your environment. Unified Communication X (UCX) provides an optimized communication layer for Message Passing (MPI), PGAS/OpenSHMEM libraries and RPC/data-centric applications. ucx/1.9.0/gnu-4.9.2 Adds ucx 1.9.0 to your environment. Unified Communication X (UCX) provides an optimized communication layer for Message Passing (MPI), PGAS/OpenSHMEM libraries and RPC/data-centric applications. unixodbc/2.3.7 Unix ODBC driver vtk/5.10.1/gnu-4.9.2 adds VTK 5.10.1 for GCC 4.9.2 to your environment. vtk/6.2.0/gnu-4.9.2 adds VTK 6.2.0 for GCC 4.9.2 to your environment. wavpack/5.1.0/gnu-4.9.2 WavPack is a completely open audio compression format providing lossless, high-quality lossy, and a unique hybrid compression mode. webkitgtk/2.2.4-1 Adds the webkitgtk-1 with webkitgtk-devel library to your environment. 2.2.4-1 EL7 RPM binaries. webkitgtk/2.4.9-1 Adds the webkitgtk with webkitgtk-devel library to your environment. 2.4.9-1 EL7 RPM binaries. zeromq/4.1.4/gnu-4.9.2 Adds zeromq 4.1.4 to your environment. ZeroMQ is a distributed messaging library that supports many message-passing patterns and methods."},{"location":"Installed_Software_Lists/module-packages/#development-tools","title":"Development Tools","text":"

This section is for modules for programs that are used in software development, profiling, or troubleshooting.

It also contains language interpreters, like Python, Ruby, and Java.

Module Description armforge/20.1.2 ArmForge is a collection of parallel profiling and debugging tools for x86_64 and ARM processors. autoconf/2.69 Adds GNU Autoconf Version 2.69 to your environment. autogen/5.18.12/gnu-4.9.2 AutoGen is a tool designed to simplify the creation and maintenance of programs that contain large amounts of repetitious text. automake/1.16.1 Adds GNU Automake Version 1.16.1 to your environment. bazel/0.7.0 Adds bazek to your environment. bazel/0.14.1/gnu-4.9.2 Adds bazek to your environment. bazel/0.21.0/gnu-4.9.2 Adds bazek to your environment. bazel/0.24.0/gnu-4.9.2 Adds bazek to your environment. bazel/0.24.1/gnu-4.9.2 Adds bazek to your environment. bazel/0.26.1/gnu-4.9.2 Adds bazek to your environment. bazel/0.27.1/gnu-4.9.2 Adds bazek to your environment. bazel/4.2.1 Adds bazek to your environment. binutils/2.29.1/gnu-4.9.2 Adds binutils 2.29.1 to your environment. The GNU binutils are a collection of tools for working with binary files and assembling and disassembling machine instructions. bison/3.0.4/gnu-4.9.2 Adds Bison 3.0.4 to your environment. Bison is a general-purpose parser generator. chicken/4.13.0 adds Chicken 4.13.0 to your environment variables clojure/1.10.0.411 This is a module with no description string. clojure/1.10.3.1069 This is a module with no description string. cmake/3.2.1 adds Cmake 3.2.1 compilers to your environment variables cmake/3.7.2 adds Cmake 3.7.2 compilers to your environment variables cmake/3.13.3 adds Cmake 3.13.3 compilers to your environment variables cmake/3.19.1 adds Cmake 3.19.1 to your environment variables cmake/3.21.1 adds Cmake 3.21.1 to your environment variables cmake/3.27.3 CMake is an open-source, cross-platform family of tools designed to build, test and package software. cmdstan/2.24.1/gnu-4.9.2 Adds CmdStan 2.24.1 to your environment. cuda/7.5.18/gnu-4.9.2 Adds cuda 7.5.18 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/8.0.61-patch2/gnu-4.9.2 Adds cuda 8.0.61 patch2 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/9.0.176-patch4/gnu-4.9.2 Adds cuda 9.0.176 patch4 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/10.0.130/gnu-4.9.2 Adds cuda 10.0.130 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/10.1.243/gnu-4.9.2 Adds cuda 10.1.243 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cudnn/5.1/cuda-7.5 Adds cuDNN to your environment. cudnn/5.1/cuda-8.0 Adds cuDNN to your environment. cudnn/6.0/cuda-7.5 Adds cuDNN to your environment. cudnn/6.0/cuda-8.0 Adds cuDNN to your environment. cudnn/7.0.4/cuda-8.0 Adds cuDNN to your environment. cudnn/7.1.4/cuda-9.0 Adds cuDNN to your environment. cudnn/7.4.2.24/cuda-9.0 Adds cuDNN to your environment. cudnn/7.4.2.24/cuda-10.0 Adds cuDNN to your environment. cudnn/7.5.0.56/cuda-10.0 Adds cuDNN to your environment. cudnn/7.5.0.56/cuda-10.1 Adds cuDNN to your environment. cudnn/7.6.5.32/cuda-10.0 Adds cuDNN to your environment. cudnn/7.6.5.32/cuda-10.1 Adds cuDNN to your environment. ddt/6.0.4 This module adds DDT 6.0.4 with MIC support to your environment. depot_tools/788d9e0d adds depot_tools to your environment variables depot_tools/c03a9cf adds depot_tools to your environment variables dimemas/5.3.3/bindist Adds dimemas 5.3.3 to your environment. Dimemas is an abstracted network simulator for message-passing programs. dotnet-sdk/7.0.203 Microsoft .NET SDK doxygen/1.8.14 This is a module with no description string. emacs/24.5 Adds Emacs 24.5 to your environment. An extensible text editor. emacs/26.3 Adds Emacs 26.3 to your environment. An extensible text editor. emacs/28.1 Adds Emacs 28.1 to your environment. An extensible text editor. extrae/3.5.2/intel-2017 Adds extrae 3.5.2 to your environment. Extrae is an instrumentation framework to generate execution traces of the most used parallel runtimes. f2c/2013-09-26/gnu-4.9.2 Adds f2c 2013-09-26 to your environment. f2c is a source-to-source translator from Fortran 77 to C. It is not standards-compliant and is not recommended for use under any circumstances. flex/2.5.39 adds Flex 2.4.39 to your environment variables git/2.3.5 adds Git 2.3.5 to your environment variables git/2.10.2 adds Git 2.10.2 to your environment variables git/2.19.1 adds Git 2.19.1 to your environment variables git/2.32.0 adds Git 2.32.0 to your environment variables git/2.41.0-lfs-3.3.0 Git is a free and open source distributed version control system. This installation includes git-lfs pre-configured. gperf/3.0.4/gnu-4.9.2 Adds gperf 3.0.4 to your environment. GNU gperf is a perfect hash function generator. guile/2.0.11/gnu-4.9.2 Adds guile 2.0.11 to your environment. haskellplatform/2014.2.0.0 adds Haskell Platform to your environment variables htop/1.0.3/gnu-4.9.2 Adds htop 1.0.3 to your environment. htop/3.2.2 htop is a cross-platform interactive process viewer. java/1.8.0_45 adds Oracle JDK 1.8.0_45 compilers to your environment variables java/1.8.0_92 adds Oracle JDK 1.8.0_92 compilers to your environment variables java/openjdk-8/8u212/hotspot adds Oracle JDK 8 compilers to your environment variables java/openjdk-8/8u212/openj9 adds Oracle JDK 8 compilers to your environment variables java/openjdk-11/11.0.1 adds Oracle JDK 11.0.1 compilers to your environment variables java/openjdk-11/11.0.3u7/hotspot adds Oracle JDK 11.0.3 compilers to your environment variables java/openjdk-11/11.0.3u7/openj9 adds Oracle JDK 11.0.3 compilers to your environment variables java/semeru-8/8u322-b06 adds IBM Semeru 8u322-b06 compilers to your environment variables java/semeru-11/11.0.14.1+1 adds IBM Semeru 11.0.14.1+1 compilers to your environment variables java/semeru-17/17.0.2+8 adds IBM Semeru 17.0.2+8 compilers to your environment variables java/temurin-8/8u322-b06 adds Adoptium Temurin 8u322-b06 compilers to your environment variables java/temurin-11/11.0.14+9 adds Adoptium Temurin 11.0.14+9 compilers to your environment variables java/temurin-11/11.0.14.1+1 adds Adoptium Temurin 11.0.14.1+1 compilers to your environment variables java/temurin-17/17.0.2+8 adds Adoptium Temurin 17.0.2+8 compilers to your environment variables julia/0.3.10 adds Julia 0.3.10 to your environment variables julia/0.4.0 adds Julia 0.4.0 to your environment variables julia/0.4.7 adds Julia 0.4.7 to your environment variables julia/0.5.0 adds Julia 0.5.0 to your environment variables julia/0.6.0 adds Julia 0.6.0 to your environment variables julia/0.7.0 adds Julia 0.7.0 to your environment variables julia/1.0.0 adds Julia 1.0.0 to your environment variables julia/1.1.0 adds Julia 1.1.0 to your environment variables julia/1.2.0 adds Julia 1.2.0 to your environment variables julia/1.3.1 adds Julia 1.3.1 to your environment variables julia/1.5.0 adds Julia 1.5.0 to your environment variables julia/1.6.0 adds Julia 1.6.0 to your environment variables julia/1.6.2 adds Julia 1.6.2 to your environment variables julia/1.7.0 adds Julia 1.7.0 to your environment variables julia/1.7.1 adds Julia 1.7.1 to your environment variables julia/1.8.5 adds Julia 1.8.5 to your environment variables julia/1.9.0 adds Julia 1.9.0 to your environment variables julia/1.9.1 adds Julia 1.9.1 to your environment variables julia/1.9.2 adds Julia 1.9.2 to your environment variables julia/1.9.3 adds Julia 1.9.3 to your environment variables libtool/2.4.6 Adds libtool 2.4.6 to your environment. GNU libtool is a generic library support script. Libtool hides the complexity of using shared libraries behind a consistent, portable interface. libxkbcommon/1.5.0 Adds xkbcommon to your environment. ltrace/0.7.3/gnu-4.9.2 Adds ltrace 0.7.3 to your environment. lua/5.3.1 This module adds the Lua 5.3.1 package to your environment. Lua is a powerful, fast, lightweight, embeddable scripting language. mc/4.8.14 This module adds Midnight Commander 4.8.14 to your environment. mono/3.12.1 adds Mono 3.12.1 compilers to your environment variables mono/5.20.1.27/gnu-4.9.2 This is a module with no description string. nano/2.4.2 Adds nano 2.4.2 to your environment. A simple text editor. nano/4.9 The nano text editor. nano/6.1 The nano text editor. nasm/2.13.01 The Netwide Assembler, NASM, is an 80x86 and x86-64 assembler. ncl/6.0.0 adds NCL 6.0.0 to your environment variables ncl/6.3.0 adds NCL 6.3.0 to your environment variables nedit/5.6-aug15 Adds the NEdit GUI text editor to your environment. netlogo/6.1.0 adds NetLogo tooklit compilers to your environment variables ninja/1.11.1 Adds Ninja Version 1.11.1 to your environment. paraver/4.6.4.rc1/bindist Adds paraver 4.6.4.rc1 to your environment. Paraver is a trace visualizer for post-mortem trace analysis. perl/5.16.0 This module adds adds Perl 5.16.0 to your environment. perl/5.22.0 This module adds adds Perl 5.22.0 to your environment. perlbrew/0.73 This module adds the Perlbrew 0.73 package to your environment. Use Perlbrew to manage your own Perls and Perl modules pigz/2.4 pigz is a fully functional replacement for gzip that exploits multiple processors and multiple cores when compressing data. pkg-config/0.29.2 Adds pkg-config 0.29.2 to your environment variables. pycuda/2017.1/python2 Adds Python2 PyCuda to your environment. MPI for Python. pycuda/2017.1/python3 Adds Python3 PyCuda to your environment. MPI for Python. pypy3/6.0.0/gnu-4.9.2 Pypy is a JIT-ing interpreter for the Python language. This is the version intended to be compatible with CPython 3.5. python/2.7.9 adds Python 2.7.9 with pip and virtualenv to your environment variables python/2.7.12 adds Python 2.7.12 with pip and virtualenv to your environment variables python/3.4.3 adds Python 3.4.3 with pip and virtualenv to your environment variables python/3.5.2 adds Python 3.5.2 with pip and virtualenv to your environment variables python/3.6.1/gnu-4.9.2 Adds Python 3.6.1 with pip and virtualenv to your environment variables. python/3.6.3 Adds Python 3.6.3 with pip and virtualenv to your environment variables. python/3.7.0 Adds Python 3.7.0 with pip and virtualenv to your environment variables. python/3.7.2 Adds Python 3.7.2 with pip and virtualenv to your environment variables. python/3.7.4 Adds Python 3.7.4 with pip and virtualenv to your environment variables. python/3.8.0 Adds Python 3.8.0 with pip and virtualenv to your environment variables. python/3.8.6 Adds Python 3.8.6 with pip and virtualenv to your environment variables. python/3.9.0 Adds Python 3.9.0 with pip and virtualenv to your environment variables. python/3.9.1 Adds Python 3.9.1 with pip and virtualenv to your environment variables. python/3.9.6 Adds Python 3.9.6 with pip and virtualenv to your environment variables. python/3.9.10 Adds Python 3.9.10 with pip and virtualenv to your environment variables. python/3.11.3 Adds Python 3.11.3 with pip and virtualenv to your environment variables. python/3.11.4 Adds Python 3.11.4 with pip and virtualenv to your environment variables. python/idp3/2019/3.6.8 Adds Intel Distribution for Python to your environment variables. python/miniconda3/4.5.11 Adds Miniconda 4.5.11 to your environment variables. python/miniconda3/4.10.3 Adds Miniconda 4.10.3 to your environment variables. qt/4.8.6/gnu-4.9.2 Adds Qt 4.8.6 to your environment. Qt is a cross-platform development tool. qt/5.4.2/gnu-4.9.2 Adds Qt 5.4.2 to your environment. Qt is a cross-platform development tool. qt/5.12.1/gnu-4.9.2 Adds Qt 5.12.1 to your environment. Qt is a cross-platform development tool. qwt/6.1.4/gnu-4.9.2 Adds Qwt 6.1.4 to your environment. racket/6.8 Adds Racket 6.8 to your enviroment. rappture/20130903 Adds the Rappture toolkit to your environment. rstudio-ide/1.4.1717 RStudio is an integrated development environment (IDE) for R ruby/2.2.2 Ruby 2.2.2 with RubyGems 2.4.8 and libffi 3.2.1 ruse/1.0.1 A command-line utility to periodically measure the memory use of a process and its subprocesses. ruse/2.0 Ruse is a command-line utility that periodically measures the resource use of a process and its subprocesses. sbcl/1.3.19 Adds Steelbank Common LISP 1.3.19 to your environment. sbcl/2.1.6 Adds Steelbank Common LISP 2.1.6 to your environment. sbt/1.6.1 This is a module with no description string. scala/2.13.8 This is a module with no description string. scala/3.1.1 This is a module with no description string. scons/2.3.4 adds scons 2.3.4 to your environment variables shellcheck/0.8.0 Shellcheck is a tool for performing static analysis of shell scripts. strace/4.12 Adds strace 4.12 to your environment. Trace system calls and signals. subversion/1.8.13 adds Subversion 1.8.13 to your environment variables subversion/1.14.1 adds Subversion 1.14.1 to your environment variables swig/3.0.5/gnu-4.9.2 This module adds the SWIG 3.0.5 package to your environment. SWIG is an interface compiler that connects programs written in C and C++ with scripting languages such as Perl, Python, Ruby, and Tcl. swig/3.0.7/gnu-4.9.2 This module adds the SWIG 3.0.7 package to your environment. SWIG is an interface compiler that connects programs written in C and C++ with scripting languages such as Perl, Python, Ruby, and Tcl. tcl/8.6.8 This is a modulefile for Tcl/Tk 8.6.8 v8/3.15 adds v8 to your environment variables v8/5.6 adds v8 to your environment variables valgrind/3.11.0/gnu-4.9.2 Adds valgrind 3.11.0 to your environment. Valgrind is a framework for building dynamic analysis tools. It includes the memgrind and cachegrind tools. xbae/4.60.4 Adds the Xbae Matrix Widget to your environment. xcb-util/0.4.0 Adds xcb-util packages to your environment. xorg-utils/X11R7.7 Adds xorg-utils from X11R7.7 to your environment. Includes util-macros-1.17, makedepend-1.0.5 libXdmcp-1.1.1 and libXScrnSaver-1.2.2 and imake-1.0.7."},{"location":"Installed_Software_Lists/module-packages/#core-modules","title":"Core Modules","text":"

These modules refer to groups of system tools, rather than applications. They're intended to help you use the system, and some are loaded by default.

Module Description gerun adds gerun wrapper to your environment variables lm-utils/1.0 adds utilities to check license manager status to your environment. mrxvt/0.5.4 Adds Mrxvt a multi-tabbed xterm replacement to your environment. ops-tools/1.0.0 Tools for Ops work ops-tools/1.1.0 Tools for Ops work ops-tools/2.0.0 Tools for Ops work pipe-gifts/1.0.0 A tool for transferring files between users on the same node: 'pipe-give' and 'pipe-receive'. pv/1.6.6 Pipe Viewer (pv) is a tool for monitoring the progress of data through a pipeline. rcps-core/1.0.0 adds a core set of applications and libraries to your environment. rlwrap/0.43 adds rlwrap 0.43 to your environment variables screen/4.2.1 adds Screen 4.2.1 to your environment variables screen/4.8.0-ucl1 adds Screen 4.8.0 to your environment variables screen/4.9.0 adds Screen 4.9.0 to your environment variables singularity-env/1.0.0 Sets up the environment and cache directories for the Singularity container runtime. userscripts/1.0.0 Adds userscripts dir to your path. Provides jobhist among other utilities. userscripts/1.1.0 Adds userscripts dir to your path. Provides jobhist among other utilities. userscripts/1.2.0 Adds userscripts dir to your path. Provides jobhist among other utilities. userscripts/1.3.0 Adds userscripts dir to your path. Provides jobhist among other utilities. userscripts/1.4.0 Adds userscripts dir to your path. Provides jobhist among other utilities. GATK Some GATK tools require conda and associated libraries."},{"location":"Installed_Software_Lists/module-packages/#beta-modules","title":"Beta Modules","text":"

This section is for modules we're still trying out. They may or may not work with applications from other sections.

Module Description amber/20/mpi/gnu-10.2.0 Adds AMBER 20 and AmberTools 21 to your environment amber/20/openmp/gnu-10.2.0 Adds AMBER 20 and AmberTools 21 to your environment. OpenMP threaded version. amber/20/serial/gnu-10.2.0 Adds AMBER 20 and AmberTools 21 to your environment armadillo/10.4.0/gnu-10.2.0 Adds Armadillo Version 10.4.0 C++ library to your environment. arpack-ng/3.8.0-threaded/gnu-10.2.0 Adds arpack-ng 3.8.0 to your environment. ARPACK-NG is a collection of Fortran77 subroutines designed to solve large scale eigenvalue problems. atlas-adna/0.9_2021-04-16/gnu-10.2.0 Adds ATLAS Ancient DNA Analysis package to your environment. atlas-adna/0.9_2021-08-13/gnu-10.2.0 Adds ATLAS Ancient DNA Analysis package to your environment. bedtools/2.30.0/gnu-10.2.0 Adds bedtools 2.30.0 to your environment. The bedtools utilities are a swiss-army knife of tools for a wide-range of genomics analysis tasks. binutils/2.36.1/gnu-10.2.0 Adds binutils 2.36.1 to your environment. The GNU binutils are a collection of tools for working with binary files and assembling and disassembling machine instructions. compilers/chapel/1.26.0 Chapel 1.26.0 for GCC compilers/gnu/7.3.0 The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages (libstdc++,...). compilers/gnu/8.3.0 The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages (libstdc++,...). compilers/gnu/9.2.0 The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages (libstdc++,...). compilers/gnu/10.2.0 The GNU Compiler Collection includes front ends for C, C++, Objective-C, and Fortran, as well as libraries for these languages (libstdc++,...). cp2k/8.2/ompi/gnu-10.2.0 Adds CP2K to your environment. cuda/10.1.243/gnu-7.3.0 Adds cuda 10.1.243 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/11.1.1/gnu-10.2.0 Adds cuda 11.1.1 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/11.2.0/gnu-10.2.0 Adds cuda 11.2.0 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/11.3.1/gnu-10.2.0 Adds cuda 11.3.1 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cuda/11.8.0/gnu-10.2.0 Adds cuda 11.8.0 to your environment. This is for compiling CUDA code to run on NVIDIA GPUs. cudnn/8.1.0.77/cuda-11.2 Adds cuDNN 8.1.0 to your environment. cudnn/8.2.1.32/cuda-11.3 Adds cuDNN 8.2.1 to your environment. fftw/3.3.8/gnu-7.3.0 Adds FFTW 3.3.8 for GCC 7.3.0 compilers to your environment variables. Includes single and double precision, plus long-double and quad. Includes OpenMP and POSIX threads libraries. fftw/3.3.8/gnu-9.2.0 Adds FFTW 3.3.8 for GCC 9.2.0 compilers to your environment variables. Includes single and double precision, plus long-double and quad. Includes OpenMP and POSIX threads libraries. fftw/3.3.9/gnu-10.2.0 Adds FFTW 3.3.9 for GCC 10.2.0 compilers to your environment variables. Includes single and double precision, plus long-double and quad. Includes OpenMP and POSIX threads libraries. gcc-libs/7.3.0 Base module for gcc 7.3.0 -- does not set the standard compiler environment variables. The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages (libstdc++,...). gcc-libs/8.3.0 Base module for gcc 8.3.0 -- does not set the standard compiler environment variables. The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages (libstdc++,...). gcc-libs/9.2.0 Base module for gcc 9.2.0 -- does not set the standard compiler environment variables. The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages (libstdc++,...). gcc-libs/10.2.0 Base module for gcc 10.2.0 -- does not set the standard compiler environment variables. The GNU Compiler Collection includes front ends for C, C++, Objective-C, and Fortran, as well as libraries for these languages (libstdc++,...). Patch 95889 for __has_include applied. gdal/3.0.4/gnu-9.2.0 adds GDAL 3.0.4 with PROJ.4 7.0.0 to your environment variables. gdal/3.1.3/gnu-9.2.0 adds GDAL 3.1.3 with PROJ.4 7.0.0 to your environment variables. gdal/3.3.2/gnu-10.2.0 adds GDAL 3.3.2 with PROJ.4 8.1.1 to your environment variables. geos/3.8.1/gnu-9.2.0 Adds geos 3.8.1 to your environment. GEOS (Geometry Engine, Open Source) is a library for performing various spatial operations, especially for boolean operations on GIS data. Note this version does not include the SWIG, Python, Ruby, or PHP bindings. geos/3.9.1/gnu-10.2.0 Adds geos 3.9.1 to your environment. GEOS (Geometry Engine, Open Source) is a library for performing various spatial operations, especially for boolean operations on GIS data. Note this version does not include the SWIG, Python, Ruby, or PHP bindings. gmt/6.0.0/gnu-9.2.0 adds GMT 6.0.0 to your environment variables gmt/6.2.0/gnu-10.2.0 adds GMT 6.2.0 to your environment variables grass/8.0-dev/gnu-10.2.0 Adds GRASS GIS 8.0 Development Version to your environment. grass/8.0dev adds UCL set of modules for GRASS GIS 8.0 Dev to your environment gromacs/2020.1/cuda-10.1 Adds gromacs 2020.1 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2020.1/intel-2020 Adds gromacs 2020.1 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2020.4/cuda-10.1 Adds gromacs 2020.4 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2020.4/intel-2020 Adds gromacs 2020.4 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2021.2/gnu-7.3.0 Adds gromacs 2021.2 to your environment. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gromacs/2021.5/cuda-11.3 Adds gromacs 2021.5 built with CUDA 11.3 to your environment. ThreadMPI and OpenMPI versions included. GROMACS is a package for performing molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. gsl/2.6/gnu-9.2.0 adds GSL 2.6 for GCC 9.2.0 to your environment. gsl/2.7/gnu-10.2.0 adds GSL 2.7 for GCC 10.2.0 to your environment. hdf/5-1.10.5/gnu-9.2.0 Adds hdf 5-1.10.5 to your environment. Serial version built with GNU. hdf/5-1.10.6/gnu-10.2.0 Adds hdf 5-1.10.6 to your environment. Serial version built with GNU 10.2.0. jags/4.3.0/gnu-10.2.0-openblas Adds JAGS 4.3.0 to your environment. jags/4.3.1/gnu-10.2.0-openblas Adds JAGS 4.3.1 to your environment. jags/4.3.2/gnu-10.2.0-openblas Adds JAGS 4.3.2 to your environment. jagurs/gnu-10.2.0-ompi adds JAGURS V0516 MPI version to your environment variables jagurs/mpi/recommended Adds set of default modules to the environment for JAGURS users. lammps/2aug23/basic-fftw/gnu-10.2.0 Adds LAMMPS 2 August 2023 to your environment. Binary is lmp_mpi. lammps/28mar23/basic-fftw/gnu-10.2.0 Adds LAMMPS 28 March 2023 to your environment. Binary is lmp_mpi. lammps/29sep21up2/basic-fftw/gnu-10.2.0 Adds LAMMPS 29Sep2021 Update 2 to your environment. Binary is lmp_mpi. lammps/29sep21up2/basic/gnu-10.2.0 Adds LAMMPS 29Sep2021 Update 2 to your environment. Binary is lmp. lammps/29sep21up2/basic/gnu-10.2.0-aristotle Adds LAMMPS 29Sep2021 Update 2 to your environment. Binary is lmp_aristotle. lammps/29sep21up2/basic/intel-2020 Adds LAMMPS 29Sep2021 Update 2 to your environment. Binary is lmp_mpi or lmp_default. lammps/29sep21up2/gpu/gnu-10.2.0 Adds LAMMPS 29Sep2021 Update 2 to your environment. Binary is lmp. lammps/29sep21up2/gpu/intel-2020 Adds LAMMPS 29Sep2021 Update 2 with gpu package to your environment. Binary is lmp_gpu or lmp_default. lammps/29sep21up2/userintel/intel-2020 Adds LAMMPS 29Sep2021 Update 2 with INTEL package to your environment. Binary is lmp_mpi or lmp_default. libpng/1.6.37/gnu-9.2.0 Adds libpng 1.6.37 to your environment. matlab/full/r2018a/9.4-prefdir-fix Adds Matlab R2018a to your environment. med/4.0.0/gnu-9.2.0 Adds med 4.0.0 to your environment. Allows reading and writing of MED format files. mpi/openmpi/3.1.4/gnu-7.3.0 Adds openmpi 3.1.4 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/3.1.5/gnu-9.2.0 Adds openmpi 3.1.5 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mpi/openmpi/4.0.5/gnu-10.2.0 Adds openmpi 4.0.5 to your environment. The Open MPI Project is an open source Message Passing Interface implementation. mumps/5.2.1/gnu-9.2.0 Adds mumps 5.2.1 to your environment. Sequential (threaded) version built with GNU, OpenBLAS and METIS. namd/2.13/intel-2018-update3/testing Adds NAMD 2.13 to your environment namd/2.13/plumed/intel-2018-update3/testing Adds NAMD 2.13 to your environment netcdf/4.7.4/gnu-9.2.0 adds NetCDF 4.7.4 for GCC 9.2.0 to your environment. netcdf/4.8.1/gnu-10.2.0 adds NetCDF 4.8.1 for GCC 10.2.0 to your environment. openblas/0.3.7-native-threads/gnu-9.2.0 OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version. openblas/0.3.7-openmp/gnu-9.2.0 OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version. openblas/0.3.7-serial/gnu-9.2.0 OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version. openblas/0.3.13-native-threads/gnu-10.2.0 OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version. openblas/0.3.13-openmp/gnu-10.2.0 OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version. openblas/0.3.13-serial/gnu-10.2.0 OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version. pcre2/10.35/gnu-9.2.0 Adds pcre2 10.35 to your environment. PCRE (Perl-compatible regular expressions) is a C library implementing regular expression pattern-matching using the same semantics as Perl 5. pcre2/10.37/gnu-10.2.0 Adds pcre2 10.37 to your environment. PCRE (Perl-compatible regular expressions) is a C library implementing regular expression pattern-matching using the same semantics as Perl 5. proj.4/7.0.0/gnu-9.2.0 Adds the PROJ.4 Cartographic Projections library to your environment. proj.4/8.1.1/gnu-10.2.0 Adds the PROJ.4 Cartographic Projections library to your environment. proj.4/9.2.0/gnu-10.2.0 Adds the PROJ.4 Cartographic Projections library to your environment. protobuf/3.14.0/gnu-9.2.0 adds Google Protocol Buffers for GCC 9.2.0 to your environment. protobuf/3.17.3/gnu-10.2.0 adds Google Protocol Buffers for GCC 10.2.0 to your environment. prsice/2.3.3/gnu-9.2.0 PRSice (pronounced 'precise') is a software package for calculating, applying, evaluating and plotting the results of polygenic risk scores (PRS). python/3.9.6-gnu-10.2.0 Adds Python 3.9.6 compiled with GNU 10.2.0 with pip and virtualenv to your environment variables. python3/3.9-gnu-10.2.0 Adds UCL recommended set of python packages to your environment variables. To see what is included, check https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-shared.list and https://github.com/UCL-RITS/rcps-buildscripts/blob/master/lists/python-3.list quip/c6359e1/gnu-10.2.0 Adds QUIP to your environment built with GNU 10.2.0 for recent versions of LAMMPS r/3.6.3-openblas/gnu-9.2.0 Adds R 3.6.3 and Bioconductor 3.10 to your environment. r/4.0.2-openblas/gnu-9.2.0 Adds R 4.0.2 and Bioconductor 3.11 to your environment. r/4.1.1-openblas/gnu-10.2.0 Adds R 4.1.1 and Bioconductor 3.13 to your environment. r/r-3.6.3_bc-3.10 adds UCL recommended set of R packages to your environment variables r/r-4.0.2_bc-3.11 adds UCL recommended set of R packages for R 4.0.2 to your environment variables r/r-4.1.1_bc-3.13 adds UCL recommended set of R packages for R 4.1.1 to your environment variables r/r-4.2.0_bc-3.15 adds UCL recommended set of R packages for R 4.2.0 to your environment variables r/r-4.2.2_bc-3.16 adds UCL recommended set of R packages for R 4.2.2 to your environment variables r/r-4.2.3_bc-3.16 adds UCL recommended set of R packages for R 4.2.3 to your environment variables rcps-core-beta/gnu-7.3.0 adds a core set of applications and libraries to your environment. rcps-core-beta/gnu-8.3.0 adds a core set of applications and libraries to your environment. rcps-core-beta/gnu-9.2.0 adds a core set of applications and libraries to your environment. rcps-core-beta/gnu-10.2.0 adds a core set of applications and libraries to your environment. sac/102.0-test/gnu-10.2.0 Adds SAC 102.0 to your environment. scalapack/2.1.0/gnu-9.2.0/openblas-0.3.7 ScaLAPACK is a library of high-performance linear algebra routines for parallel distributed memory machines. ScaLAPACK solves dense and banded linear systems, least squares problems, eigenvalue problems, and singular value problems. sqlite/3.36.0/gnu-10.2.0 Adds SQLite Version 3.36.0 to your environment. sra-tools/3.0.6/gnu-10.2.0 adds SRA Tools 3.0.6 for GCC 10.2.0 to your environment. stata/16 Adds Stata/MP 16 to your environment. superlu/5.2.1/gnu-10.2.0 Adds superlu 5.2.1 to your environment. SuperLU is a general purpose library for the direct solution of large, sparse, nonsymmetric systems of linear equations. ucx/1.9.0/gnu-10.2.0 Adds ucx 1.9.0 to your environment. Unified Communication X (UCX) provides an optimized communication layer for Message Passing (MPI), PGAS/OpenSHMEM libraries and RPC/data-centric applications. udunits/2.2.26/gnu-9.2.0 adds the UDUNITS-2 package to your environment. udunits/2.2.28/gnu-10.2.0 adds the UDUNITS-2 package to your environment."},{"location":"Installed_Software_Lists/module-packages/#workaround-modules","title":"Workaround Modules","text":"

Sometimes we'll find a problem that can't be fixed properly, but can be worked-around by doing something that can be loaded as a module. That kind of module goes in this section.

Module Description bazel-compiler-helpers/intel-2018 Adds bazel compiler wrappers to your environment. getcwd-autoretry This module uses LD_PRELOAD to shadow the getcwd function with a version that retries on failure, and is intended to workaround a bug in the Lustre filesystem."},{"location":"Installed_Software_Lists/python-packages/","title":"Python Packages","text":"

We provide a collection of installed Python packages for each minor version of Python, as a bundle module. This page lists the packages for the current recommended Python 3 bundle.

This can be loaded using:

module load python3/recommended\n

The version of Python 3 provided with this bundle is currently Python 3.9.10.

Note that some packages we do not provide this way, because they have complicated non-Python dependencies. These are usually provided using the normal application modules system. This includes TensorFlow.

The following list was last updated at 15:36:07 (+0100) on 09 Sep 2023.

Module Version Description acora 2.3 Fast multi-keyword search engine for text strings appdirs 1.4.4 A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\". args 0.1.0 Command Arguments for Humans. ase 3.22.1 Atomic Simulation Environment astor 0.8.1 Read/rewrite/write Python ASTs astropy 5.0.1 Astronomy and astrophysics core library asttokens 2.0.5 Annotate AST trees with source code positions attrs 21.4.0 Classes Without Boilerplate backcall 0.2.0 Specifications for callback functions passed in to an API biopython 1.79 Freely available tools for computational molecular biology. bitarray 2.3.6 efficient arrays of booleans -- C extension black 22.1.0 The uncompromising code formatter. BoltzTraP2 20.7.1 band-structure interpolator and transport coefficient calculator certifi 2021.10.8 Python package for providing Mozilla's CA Bundle. cffi 1.15.0 Foreign Function Interface for Python calling C code. cftime 1.5.2 Time-handling functionality from netcdf4-python charset-normalizer 2.0.11 The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet. click 8.0.3 Composable command line interface toolkit clint 0.5.1 Python Command Line Interface Tools colorama 0.4.4 Cross-platform colored terminal text. coloredlogs 15.0.1 Colored terminal output for Python's logging module colormath 3.0.0 Color math and conversion library. commonmark 0.9.1 Python parser for the CommonMark Markdown spec cryptography 36.0.1 cryptography is a package which provides cryptographic recipes and primitives to Python developers. cutadapt 3.5 trim adapters from high-throughput sequencing reads cvxopt 1.2.7 Convex optimization package cycler 0.11.0 Composable style cycles Cython 0.29.27 The Cython compiler for writing C extensions for the Python language. cyvcf2 0.30.14 fast vcf parsing with cython + htslib deap 1.3.1 Distributed Evolutionary Algorithms in Python decorator 5.1.1 Decorators for Humans distlib 0.3.4 Distribution utilities dnaio 0.7.1 Read and write FASTA and FASTQ files efficiently ecdsa 0.17.0 ECDSA cryptographic signature library (pure python) emcee 3.1.1 The Python ensemble sampling toolkit for MCMC ephem 4.1.3 Compute positions of the planets and stars executing 0.8.2 Get the currently executing AST node of a frame, and other information filelock 3.4.2 A platform independent file lock. fonttools 4.29.1 Tools to manipulate font files funcparserlib 0.3.6 Recursive descent parsing library based on functional combinators future 0.18.2 Clean single-source support for Python 3 and 2 greenlet 1.1.2 Lightweight in-process concurrent programming hankel 1.1.0 Hankel Transformations using method of Ogata 2005 humanfriendly 10.0 Human friendly output for text interfaces using Python hy 0.20.0 Lisp and Python love each other. idna 3.3 Internationalized Domain Names in Applications (IDNA) imageio 2.15.0 Library for reading and writing a wide range of image, video, scientific, and volumetric data formats. importlib-metadata 4.10.1 Read metadata from Python packages iniconfig 1.1.1 iniconfig: brain-dead simple config-ini parsing ipython 8.0.1 IPython: Productive Interactive Computing isal 0.11.1 Faster zlib and gzip compatible compression and decompression by providing python bindings for the ISA-L library. jedi 0.18.1 An autocompletion tool for Python that can be used for text editors. Jinja2 3.0.3 A very fast and expressive template engine. joblib 1.1.0 Lightweight pipelining with Python functions kiwisolver 1.3.2 A fast implementation of the Cassowary constraint solver ldap3 2.9.1 A strictly RFC 4510 conforming LDAP V3 pure Python client library llvmlite 0.38.0 lightweight wrapper around basic LLVM functionality lxml 4.7.1 Powerful and Pythonic XML processing library combining libxml2/libxslt with the ElementTree API. lzstring 1.0.4 lz-string for python Mako 1.1.6 A super-fast templating language that borrows the best ideas from the existing templating languages. Markdown 3.3.6 Python implementation of Markdown. MarkupSafe 2.0.1 Safely add untrusted strings to HTML/XML markup. matplotlib 3.5.1 Python plotting package matplotlib-inline 0.1.3 Inline Matplotlib backend for Jupyter mpmath 1.2.1 Python library for arbitrary-precision floating-point arithmetic multiqc 1.12 Create aggregate bioinformatics analysis reports across many samples and tools mypy-extensions 0.4.3 Experimental type system extensions for programs checked with the mypy typechecker. netCDF4 1.5.8 Provides an object-oriented python interface to the netCDF version 4 library. networkx 2.6.3 Python package for creating and manipulating graphs and networks nibabel 3.2.2 Access a multitude of neuroimaging data formats ninja 1.10.2.3 Ninja is a small build system with a focus on speed nose 1.3.7 nose extends unittest to make testing easier numba 0.55.1 compiling Python code using LLVM numpy 1.21.5 NumPy is the fundamental package for array computing with Python. obspy 1.2.2 ObsPy - a Python framework for seismological observatories. packaging 21.3 Core utilities for Python packages pandas 1.4.0 Powerful data structures for data analysis, time series, and statistics parso 0.8.3 A Python Parser pathspec 0.9.0 Utility library for gitignore style pattern matching of file paths. pexpect 4.8.0 Pexpect allows easy control of interactive console applications. pickleshare 0.7.5 Tiny 'shelve'-like database with concurrency support Pillow 9.0.1 Python Imaging Library (Fork) pip 22.0.3 The PyPA recommended tool for installing Python packages. platformdirs 2.5.4 A small Python package for determining appropriate platform-specific dirs, e.g. a \"user data dir\". pluggy 1.0.0 plugin and hook calling mechanisms for python Pmw 2.0.1 Python Mega Widgets prompt-toolkit 3.0.27 Library for building powerful interactive command lines in Python ptyprocess 0.7.0 Run a subprocess in a pseudo terminal pure-eval 0.2.2 Safely evaluate AST nodes without side effects py 1.11.0 library with cross-python path, ini-parsing, io, code, log facilities pyasn1 0.4.8 ASN.1 types and codecs pycparser 2.21 C parser in Python pyerfa 2.0.0.1 Python bindings for ERFA pyfastaq 3.17.0 Script to manipulate FASTA and FASTQ files, plus API for developers Pygments 2.11.2 Pygments is a syntax highlighting package written in Python. pyparsing 3.0.7 Python parsing module pysam 0.18.0 pysam pyspglib 1.8.3.1 This is the pyspglib module. pytest 7.0.0 pytest: simple powerful testing with Python python-dateutil 2.8.2 Extensions to the standard Python datetime module python-Levenshtein 0.12.2 Python extension for computing string edit distances and similarities. pytz 2021.3 World timezone definitions, modern and historical PyWavelets 1.2.0 PyWavelets, wavelet transform module PyYAML 6.0 YAML parser and emitter for Python qutip 4.6.3 QuTiP: The Quantum Toolbox in Python regex 2022.1.18 Alternative regular expression module, to replace re. requests 2.27.1 Python HTTP for Humans. rich 11.2.0 Render rich text, tables, progress bars, syntax highlighting, markdown and more to the terminal rply 0.7.8 A pure Python Lex/Yacc that works with RPython scikit-image 0.19.1 Image processing in Python scikit-learn 1.0.2 A set of python modules for machine learning and data mining scipy 1.8.0 SciPy: Scientific Library for Python seaborn 0.11.2 seaborn: statistical data visualization setuptools 60.8.1 Easily download, build, install, upgrade, and uninstall Python packages simplejson 3.17.6 Simple, fast, extensible JSON encoder/decoder for Python six 1.16.0 Python 2 and 3 compatibility utilities spectra 0.0.11 Color scales and color conversion made easy for Python. spglib 1.16.3 This is the spglib module. SQLAlchemy 1.4.31 Database Abstraction Library sshpubkeys 3.3.1 SSH public key parser stack-data 0.1.4 Extract data from python stack frames and tracebacks for informative displays tabulate 0.8.9 Pretty-print tabular data threadpoolctl 3.1.0 threadpoolctl tifffile 2022.2.2 Read and write TIFF files tomli 2.0.1 A lil' TOML parser traitlets 5.1.1 Traitlets Python configuration system typing_extensions 4.0.1 Backported and Experimental Type Hints for Python 3.6+ urllib3 1.26.8 HTTP library with thread-safe connection pooling, file post, and more. virtualenv 20.13.1 Virtual Python Environment builder wcwidth 0.2.5 Measures the displayed width of unicode strings in a terminal weblogo 3.7.9 WebLogo3 : Sequence Logos Redrawn wheel 0.37.1 A built-package format for Python xlrd 2.0.1 Library for developers to extract data from Microsoft Excel (tm) .xls spreadsheet files XlsxWriter 3.0.2 A Python module for creating Excel XLSX files. xlutils 2.0.0 Utilities for working with Excel files that require both xlrd and xlwt xlwt 1.3.0 Library to create spreadsheet files compatible with MS Excel 97/2000/XP/2003 XLS files, on any platform, with Python 2.6, 2.7, 3.3+ xopen 1.4.0 Open compressed files transparently zipp 3.7.0 Backport of pathlib-compatible object wrapper for zip files"},{"location":"Installed_Software_Lists/r-packages/","title":"R Packages","text":"

We provide a collection of installed R packages for each release of R, as a bundle module. This page lists the packages for the current recommended R bundle.

This can be loaded using:

module load r/recommended\n

The version of R provided with this bundle is currently R version 4.2.0 (2022-04-22).

The following list was last updated at: 15:05:23 (+0100) on 09 Sep 2023.

Module Version Description abc 2.2.1 Tools for Approximate Bayesian Computation (ABC) abc.data 1.0 Data Only: Tools for Approximate Bayesian Computation (ABC) abind 1.4-5 Combine Multidimensional Arrays ade4 1.7-19 Analysis of Ecological Data: Exploratory and Euclidean Methods in Environmental Sciences adegenet 2.1.6 Exploratory Analysis of Genetic and Genomic Data ADGofTest 0.3 Anderson-Darling GoF test admisc 0.27 Adrian Dusa's Miscellaneous afex 1.2-1 Analysis of Factorial Experiments affxparser 1.68.1 Affymetrix File Parsing SDK affy 1.74.0 Methods for Affymetrix Oligonucleotide Arrays affydata 1.44.0 Affymetrix Data for Demonstration Purpose affyio 1.66.0 Tools for parsing Affymetrix data files affylmGUI 1.70.0 GUI for limma Package with Affymetrix Microarrays affyPLM 1.72.0 Methods for fitting probe-level models akima 0.6-3.4 Interpolation of Irregularly and Regularly Spaced Data annaffy 1.68.0 Annotation tools for Affymetrix biological metadata annmap 1.38.0 Genome annotation and visualisation package pertaining to Affymetrix arrays and NGS analysis. annotate 1.74.0 Annotation for microarrays AnnotationDbi 1.58.0 Manipulation of SQLite-based annotations in Bioconductor AnnotationFilter 1.20.0 Facilities for Filtering Bioconductor Annotation Resources AnnotationForge 1.38.0 Tools for building SQLite-based annotation data packages AnnotationHub 3.4.0 Client to access AnnotationHub resources ape 5.6-2 Analyses of Phylogenetics and Evolution arm 1.12-2 Data Analysis Using Regression and Multilevel/Hierarchical Models aroma.affymetrix 3.2.0 Analysis of Large Affymetrix Microarray Data Sets aroma.apd 0.6.0 A Probe-Level Data File Format Used by 'aroma.affymetrix' [deprecated] aroma.core 3.2.2 Core Methods and Classes Used by 'aroma.*' Packages Part of the Aroma Framework aroma.light 3.26.0 Light-Weight Methods for Normalization and Visualization of Microarray Data using Only Basic R Data Types arrangements 1.1.9 Fast Generators and Iterators for Permutations, Combinations, Integer Partitions and Compositions arsenal 3.6.3 An Arsenal of 'R' Functions for Large-Scale Statistical Summaries askpass 1.1 Safe Password Entry for R, Git, and SSH assertthat 0.2.1 Easy Pre and Post Assertions backports 1.4.1 Reimplementations of Functions Introduced Since R-3.0.0 bamsignals 1.28.0 Extract read count signals from bam files base 4.2.0 The R Base Package base64 2.0 Base64 Encoder and Decoder base64enc 0.1-3 Tools for base64 encoding BaSTA 1.9.4 Age-Specific Survival Analysis from Incomplete Capture-Recapture/Recovery Data BatchJobs 1.9 Batch Computing with R bayesplot 1.9.0 Plotting for Bayesian Models BBmisc 1.12 Miscellaneous Helper Functions for B. Bischl beachmat 2.12.0 Compiling Bioconductor to Handle Each Matrix Type beadarray 2.46.0 Quality assessment and low-level analysis for Illumina BeadArray data beadarrayExampleData 1.34.0 Example data for the beadarray package BeadDataPackR 1.48.0 Compression of Illumina BeadArray data beanplot 1.3.1 Visualization via Beanplots (Like Boxplot/Stripchart/Violin Plot) benchmarkme 1.0.7 Crowd Sourced System Benchmarks benchmarkmeData 1.0.4 Data Set for the 'benchmarkme' Package bezier 1.1.2 Toolkit for Bezier Curves and Splines BH 1.78.0-0 Boost C++ Header Files BiasedUrn 1.07 Biased Urn Model Distributions binom 1.1-1.1 Binomial Confidence Intervals for Several Parameterizations bio3d 2.4-3 Biological Structure Analysis Biobase 2.56.0 Biobase: Base functions for Bioconductor BiocFileCache 2.4.0 Manage Files Across Sessions BiocGenerics 0.42.0 S4 generic functions used in Bioconductor BiocIO 1.6.0 Standard Input and Output for Bioconductor Packages BiocManager 1.30.18 Access the Bioconductor Project Package Repository BiocParallel 1.30.2 Bioconductor facilities for parallel evaluation BiocVersion 3.15.2 Set the appropriate version of Bioconductor packages biomaRt 2.52.0 Interface to BioMart databases (i.e. Ensembl) Biostrings 2.64.0 Efficient manipulation of biological strings biovizBase 1.44.0 Basic graphic utilities for visualization of genomic data. bit 4.0.4 Classes and Methods for Fast Memory-Efficient Boolean Selections bit64 4.0.5 A S3 Class for Vectors of 64bit Integers bitops 1.0-7 Bitwise Operations blob 1.2.3 A Simple S3 Class for Representing Vectors of Binary Data ('BLOBS') blockmodeling 1.0.5 Generalized and Classical Blockmodeling of Valued Networks boot 1.3-28 Bootstrap Functions (Originally by Angelo Canty for S) BradleyTerry2 1.1-2 Bradley-Terry Models brew 1.0-7 Templating Framework for Report Generation brglm 0.7.2 Bias Reduction in Binomial-Response Generalized Linear Models bridgesampling 1.1-2 Bridge Sampling for Marginal Likelihoods and Bayes Factors brio 1.1.3 Basic R Input Output brms 2.18.0 Bayesian Regression Models using 'Stan' Brobdingnag 1.2-9 Very Large Numbers in R broom 0.8.0 Convert Statistical Objects into Tidy Tibbles BSgenome 1.64.0 Software infrastructure for efficient representation of full genomes and their SNPs BSgenome.Hsapiens.UCSC.hg19 1.4.3 Full genome sequences for Homo sapiens (UCSC version hg19, based on GRCh37.p13) bslib 0.3.1 Custom 'Bootstrap' 'Sass' Themes for 'shiny' and 'rmarkdown' bsseq 1.32.0 Analyze, manage and store bisulfite sequencing data bumphunter 1.38.0 Bump Hunter cachem 1.0.6 Cache R Objects with Automatic Pruning callr 3.7.0 Call R from R car 3.0-13 Companion to Applied Regression carData 3.0-5 Companion to Applied Regression Data Sets caret 6.0-92 Classification and Regression Training Category 2.62.0 Category Analysis caTools 1.18.2 Tools: Moving Window Statistics, GIF, Base64, ROC AUC, etc CDM 8.1-12 Cognitive Diagnosis Modeling cellranger 1.1.0 Translate Spreadsheet Cell Ranges to Rows and Columns ChAMP 2.26.0 Chip Analysis Methylation Pipeline for Illumina HumanMethylation450 and EPIC ChAMPdata 2.28.0 Data Packages for ChAMP package checkmate 2.1.0 Fast and Versatile Argument Checks class 7.3-20 Functions for Classification classInt 0.4-3 Choose Univariate Class Intervals cli 3.6.0 Helpers for Developing Command Line Interfaces clipr 0.8.0 Read and Write from the System Clipboard clue 0.3-60 Cluster Ensembles cluster 2.1.3 \"Finding Groups in Data\": Cluster Analysis Extended Rousseeuw et al. cmprsk 2.2-11 Subdistribution Analysis of Competing Risks coda 0.19-4 Output Analysis and Diagnostics for MCMC codetools 0.2-18 Code Analysis Tools for R colorRamps 2.3.1 Builds Color Tables colorspace 2.0-3 A Toolbox for Manipulating and Assessing Colors and Palettes colourpicker 1.1.1 A Colour Picker Tool for Shiny and for Selecting Colours in Plots combinat 0.0-8 combinatorics utilities commonmark 1.8.0 High Performance CommonMark and Github Markdown Rendering in R compiler 4.2.0 The R Compiler Package copula 1.0-1 Multivariate Dependence with Copulas copynumber 1.36.0 Segmentation of single- and multi-track copy number data by penalized least squares regression. corpcor 1.6.10 Efficient Estimation of Covariance and (Partial) Correlation corrplot 0.92 Visualization of a Correlation Matrix cowplot 1.1.1 Streamlined Plot Theme and Plot Annotations for 'ggplot2' cpp11 0.4.2 A C++11 Interface for R's C Interface crayon 1.5.1 Colored Terminal Output credentials 1.3.2 Tools for Managing SSH and Git Credentials crosstalk 1.2.0 Inter-Widget Interactivity for HTML Widgets crul 1.2.0 HTTP Client curl 4.3.2 A Modern and Flexible Web Client for R data.table 1.14.2 Extension of data.frame datasets 4.2.0 The R Datasets Package DBI 1.1.2 R Database Interface dbplyr 2.1.1 A 'dplyr' Back End for Databases DelayedArray 0.22.0 A unified framework for working transparently with on-disk and in-memory array-like datasets DelayedMatrixStats 1.18.0 Functions that Apply to Rows and Columns of 'DelayedMatrix' Objects deldir 1.0-6 Delaunay Triangulation and Dirichlet (Voronoi) Tessellation dendextend 1.15.2 Extending 'dendrogram' Functionality in R DEoptimR 1.0-11 Differential Evolution Optimization in Pure R Deriv 4.1.3 Symbolic Differentiation desc 1.4.1 Manipulate DESCRIPTION Files DescTools 0.99.47 Tools for Descriptive Statistics DESeq2 1.36.0 Differential gene expression analysis based on the negative binomial distribution devtools 2.4.3 Tools to Make Developing R Packages Easier DEXSeq 1.42.0 Inference of differential exon usage in RNA-Seq dichromat 2.0-0.1 Color Schemes for Dichromats diffobj 0.3.5 Diffs for R Objects digest 0.6.29 Create Compact Hash Digests of R Objects diptest 0.76-0 Hartigan's Dip Test Statistic for Unimodality - Corrected distr 2.8.0 Object Oriented Implementation of Distributions distrEx 2.8.0 Extensions of Package 'distr' distributional 0.3.0 Vectorised Probability Distributions DMRcate 2.10.0 Methylation array and sequencing spatial analysis methods DNAcopy 1.70.0 DNA copy number data analysis docopt 0.7.1 Command-Line Interface Specification Language doMC 1.3.8 Foreach Parallel Adaptor for 'parallel' doMPI 0.2.2 Foreach Parallel Adaptor for the Rmpi Package doParallel 1.0.17 Foreach Parallel Adaptor for the 'parallel' Package doRNG 1.8.2 Generic Reproducible Parallel Backend for 'foreach' Loops dotCall64 1.0-1 Enhanced Foreign Function Interface Supporting Long Vectors dparser 1.3.1-9 Port of 'Dparser' Package dplyr 1.0.9 A Grammar of Data Manipulation DSS 2.44.0 Dispersion shrinkage for sequencing data DT 0.23 A Wrapper of the JavaScript Library 'DataTables' dtplyr 1.2.1 Data Table Back-End for 'dplyr' dygraphs 1.1.1.6 Interface to 'Dygraphs' Interactive Time Series Charting Library dynamicTreeCut 1.63-1 Methods for Detection of Clusters in Hierarchical Clustering Dendrograms DynDoc 1.74.0 Dynamic document tools e1071 1.7-9 Misc Functions of the Department of Statistics, Probability Theory Group (Formerly: E1071), TU Wien easyRNASeq 2.32.0 Count summarization and normalization for RNA-Seq data EBSeq 1.36.0 An R package for gene and isoform differential expression analysis of RNA-seq data edgeR 3.38.1 Empirical Analysis of Digital Gene Expression Data in R effects 4.2-1 Effect Displays for Linear, Generalized Linear, and Other Models ellipse 0.4.2 Functions for Drawing Ellipses and Ellipse-Like Confidence Regions ellipsis 0.3.2 Tools for Working with ... ensembldb 2.20.1 Utilities to create and use Ensembl-based annotation databases Epi 2.46 Statistical Analysis in Epidemiology erer 3.1 Empirical Research in Economics with R estimability 1.3 Tools for Assessing Estimability of Linear Predictions etm 1.1.1 Empirical Transition Matrix evaluate 0.15 Parsing and Evaluation Tools that Provide More Details than the Default evd 2.3-6 Functions for Extreme Value Distributions Exact 3.2 Unconditional Exact Test ExperimentHub 2.4.0 Client to access ExperimentHub resources expm 0.999-6 Matrix Exponential, Log, 'etc' FactoMineR 2.4 Multivariate Exploratory Data Analysis and Data Mining fail 1.3 File Abstraction Interface Layer (FAIL) fansi 1.0.3 ANSI Control Sequence Aware String Functions farver 2.1.0 High Performance Colour Space Manipulation fastcluster 1.2.3 Fast Hierarchical Clustering Routines for R and 'Python' fastICA 1.2-3 FastICA Algorithms to Perform ICA and Projection Pursuit fastmap 1.1.0 Fast Data Structures fastmatch 1.1-3 Fast 'match()' Function FDb.InfiniumMethylation.hg19 2.2.0 Annotation package for Illumina Infinium DNA methylation probes fdrtool 1.2.17 Estimation of (Local) False Discovery Rates and Higher Criticism fields 13.3 Tools for Spatial Data filelock 1.0.2 Portable File Locking flashClust 1.01-2 Implementation of optimal hierarchical clustering flexmix 2.3-17 Flexible Mixture Modeling fontawesome 0.2.2 Easily Work with 'Font Awesome' Icons forcats 0.5.1 Tools for Working with Categorical Variables (Factors) foreach 1.5.2 Provides Foreach Looping Construct foreign 0.8-82 Read Data Stored by 'Minitab', 'S', 'SAS', 'SPSS', 'Stata', 'Systat', 'Weka', 'dBase', ... formatR 1.12 Format R Code Automatically Formula 1.2-4 Extended Model Formulas fpc 2.2-9 Flexible Procedures for Clustering fs 1.5.2 Cross-Platform File System Operations Based on 'libuv' futile.logger 1.4.3 A Logging Utility for R futile.options 1.0.1 Futile Options Management future 1.25.0 Unified Parallel and Distributed Processing in R for Everyone future.apply 1.9.0 Apply Function to Elements in Parallel using Futures gam 1.20.1 Generalized Additive Models gamlss 5.4-3 Generalised Additive Models for Location Scale and Shape gamlss.data 6.0-2 Data for Generalised Additive Models for Location Scale and Shape gamlss.dist 6.0-3 Distributions for Generalized Additive Models for Location Scale and Shape gamlss.mx 6.0-0 Fitting Mixture Distributions with GAMLSS gamlss.nl 4.1-0 Fitting non linear parametric GAMLSS models gargle 1.2.0 Utilities for Working with Google APIs gcrma 2.68.0 Background Adjustment Using Sequence Information gdata 2.18.0.1 Various R Programming Tools for Data Manipulation genefilter 1.78.0 genefilter: methods for filtering genes from high-throughput experiments geneLenDataBase 1.32.0 Lengths of mRNA transcripts for a number of genomes GeneNet 1.2.16 Modeling and Inferring Gene Networks geneplotter 1.74.0 Graphics related functions for Bioconductor generics 0.1.2 Common S3 Generics not Provided by Base R Methods Related to Model Fitting genetics 1.3.8.1.3 Population Genetics GenomeInfoDb 1.32.2 Utilities for manipulating chromosome names, including modifying them to follow a particular naming style GenomeInfoDbData 1.2.8 Species and taxonomy ID look up tables used by GenomeInfoDb genomeIntervals 1.52.0 Operations on genomic intervals GenomicAlignments 1.32.0 Representation and manipulation of short genomic alignments GenomicFeatures 1.48.1 Conveniently import and query gene models GenomicRanges 1.48.0 Representation and manipulation of genomic intervals geojson 0.3.4 Classes for 'GeoJSON' geojsonio 0.9.4 Convert Data from and to 'GeoJSON' or 'TopoJSON' geojsonsf 2.0.2 GeoJSON to Simple Feature Converter geometries 0.2.0 Convert Between R Objects and Geometric Structures GEOquery 2.64.2 Get data from NCBI Gene Expression Omnibus (GEO) gert 1.6.0 Simple Git Client for R getopt 1.20.3 C-Like 'getopt' Behavior GGally 2.1.2 Extension to 'ggplot2' ggforce 0.3.3 Accelerating 'ggplot2' ggplot2 3.3.6 Create Elegant Data Visualisations Using the Grammar of Graphics ggrepel 0.9.1 Automatically Position Non-Overlapping Text Labels with 'ggplot2' ggridges 0.5.3 Ridgeline Plots in 'ggplot2' ggtext 0.1.2 Improved Text Rendering Support for 'ggplot2' gh 1.3.0 'GitHub' 'API' gitcreds 0.1.1 Query 'git' Credentials from 'R' GJRM 0.2-6 Generalised Joint Regression Modelling gld 2.6.6 Estimation and Use of the Generalised (Tukey) Lambda Distribution glmnet 4.1-4 Lasso and Elastic-Net Regularized Generalized Linear Models globals 0.15.0 Identify Global Objects in R Expressions globaltest 5.50.0 Testing Groups of Covariates/Features for Association with a Response Variable, with Applications to Gene Set Testing glue 1.6.2 Interpreted String Literals gmp 0.6-5 Multiple Precision Arithmetic GO.db 3.15.0 A set of annotation maps describing the entire Gene Ontology goftest 1.2-3 Classical Goodness-of-Fit Tests for Univariate Distributions googledrive 2.0.0 An Interface to Google Drive googlesheets4 1.0.0 Access Google Sheets using the Sheets API V4 goseq 1.48.0 Gene Ontology analyser for RNA-seq and other length biased data GOstats 2.62.0 Tools for manipulating GO and microarrays gower 1.0.0 Gower's Distance gplots 3.1.3 Various R Programming Tools for Plotting Data graph 1.74.0 graph: A package to handle graph data structures graphics 4.2.0 The R Graphics Package grDevices 4.2.0 The R Graphics Devices and Support for Colours and Fonts grid 4.2.0 The Grid Graphics Package gridExtra 2.3 Miscellaneous Functions for \"Grid\" Graphics gridtext 0.1.5 Improved Text Rendering Support for 'Grid' Graphics GSEABase 1.58.0 Gene set enrichment data structures and methods gsl 2.1-7.1 Wrapper for the Gnu Scientific Library gsmoothr 0.1.7 Smoothing tools gtable 0.3.0 Arrange 'Grobs' in Tables gtools 3.9.2.1 Various R Programming Tools Gviz 1.40.1 Plotting data and annotation information along genomic coordinates HAC 1.1-0 Estimation, Simulation and Visualization of Hierarchical Archimedean Copulae (HAC) haplo.stats 1.8.7 Statistical Analysis of Haplotypes with Traits and Covariates when Linkage Phase is Ambiguous hardhat 0.2.0 Construct Modeling Packages haven 2.5.0 Import and Export 'SPSS', 'Stata' and 'SAS' Files HDF5Array 1.24.0 HDF5 backend for DelayedArray objects hexbin 1.28.2 Hexagonal Binning Routines hgu95av2.db 3.13.0 Affymetrix Affymetrix HG_U95Av2 Array annotation data (chip hgu95av2) HI 0.5 Simulation from Distributions Supported by Nested Hyperplanes highr 0.9 Syntax Highlighting for R Source Code HistData 0.8-7 Data Sets from the History of Statistics and Data Visualization Hmisc 4.7-0 Harrell Miscellaneous hms 1.1.1 Pretty Time of Day htmlTable 2.4.0 Advanced Tables for Markdown/HTML htmltools 0.5.2 Tools for HTML htmlwidgets 1.5.4 HTML Widgets for R httpcode 0.3.0 'HTTP' Status Code Helper httpuv 1.6.5 HTTP and WebSocket Server Library httr 1.4.3 Tools for Working with URLs and HTTP hugene10stprobeset.db 8.8.0 Affymetrix hugene10 annotation data (chip hugene10stprobeset) hugene10sttranscriptcluster.db 8.8.0 Affymetrix hugene10 annotation data (chip hugene10sttranscriptcluster) hwriter 1.3.2.1 HTML Writer - Outputs R Objects in HTML Format ids 1.0.1 Generate Random Identifiers igraph 1.3.1 Network Analysis and Visualization Illumina450ProbeVariants.db 1.32.0 Annotation Package combining variant data from 1000 Genomes Project for Illumina HumanMethylation450 Bead Chip probes IlluminaHumanMethylation450kanno.ilmn12.hg19 0.6.1 Annotation for Illumina's 450k methylation arrays IlluminaHumanMethylation450kmanifest 0.4.0 Annotation for Illumina's 450k methylation arrays IlluminaHumanMethylationEPICanno.ilm10b4.hg19 0.6.0 Annotation for Illumina's EPIC methylation arrays IlluminaHumanMethylationEPICmanifest 0.3.0 Manifest for Illumina's EPIC methylation arrays illuminaHumanv4.db 1.26.0 Illumina HumanHT12v4 annotation data (chip illuminaHumanv4) illuminaio 0.38.0 Parsing Illumina Microarray Output Files impute 1.70.0 impute: Imputation for microarray data ini 0.3.1 Read and Write '.ini' Files INLA 22.05.18-2 Full Bayesian Analysis of Latent Gaussian Models using Integrated Nested Laplace Approximations inline 0.3.19 Functions to Inline C, C++, Fortran Function Calls from R insight 0.17.1 Easy Access to Model Information for Various Model Objects interactiveDisplayBase 1.34.0 Base package for enabling powerful shiny web displays of Bioconductor objects intervals 0.15.2 Tools for Working with Points and Intervals ipred 0.9-12 Improved Predictors IRanges 2.30.0 Foundation of integer range manipulation in Bioconductor ismev 1.42 An Introduction to Statistical Modeling of Extreme Values isoband 0.2.5 Generate Isolines and Isobands from Regularly Spaced Elevation Grids ISOcodes 2022.01.10 Selected ISO Codes isva 1.9 Independent Surrogate Variable Analysis iterators 1.0.14 Provides Iterator Construct iterpc 0.4.2 Efficient Iterator for Permutations and Combinations JADE 2.0-3 Blind Source Separation Methods Based on Joint Diagonalization and Some BSS Performance Criteria jpeg 0.1-9 Read and write JPEG images jqr 1.2.3 Client for 'jq', a 'JSON' Processor jquerylib 0.1.4 Obtain 'jQuery' as an HTML Dependency Object jsonify 1.2.1 Convert Between 'R' Objects and Javascript Object Notation (JSON) jsonlite 1.8.0 A Simple and Robust JSON Parser and Generator for R karyoploteR 1.22.0 Plot customizable linear genomes displaying arbitrary data KEGGREST 1.36.0 Client-side REST access to the Kyoto Encyclopedia of Genes and Genomes (KEGG) kernlab 0.9-30 Kernel-Based Machine Learning Lab KernSmooth 2.23-20 Functions for Kernel Smoothing Supporting Wand & Jones (1995) knitr 1.39 A General-Purpose Package for Dynamic Report Generation in R kohonen 3.0.11 Supervised and Unsupervised Self-Organising Maps kpmt 0.1.0 Known Population Median Test labeling 0.4.2 Axis Labeling lambda.r 1.2.4 Modeling Data with Functional Programming later 1.3.0 Utilities for Scheduling Functions to Execute Later with Event Loops lattice 0.20-45 Trellis Graphics for R latticeExtra 0.6-29 Extra Graphical Utilities Based on Lattice lava 1.6.10 Latent Variable Models lazyeval 0.2.2 Lazy (Non-Standard) Evaluation lbfgsb3c 2020-3.2 Limited Memory BFGS Minimizer with Bounds on Parameters with optim() 'C' Interface leafem 0.2.0 'leaflet' Extensions for 'mapview' leaflet 2.1.1 Create Interactive Web Maps with the JavaScript 'Leaflet' Library leaflet.providers 1.9.0 Leaflet Providers leafsync 0.1.0 Small Multiples for Leaflet Web Maps leaps 3.1 Regression Subset Selection learnr 0.10.1 Interactive Tutorials for R lifecycle 1.0.1 Manage the Life Cycle of your Package Functions limma 3.52.1 Linear Models for Microarray Data listenv 0.8.0 Environments Behaving (Almost) as Lists lme4 1.1-29 Linear Mixed-Effects Models using 'Eigen' and S4 lmerTest 3.1-3 Tests in Linear Mixed Effects Models lmom 2.9 L-Moments lmtest 0.9-40 Testing Linear Regression Models locfit 1.5-9.5 Local Regression, Likelihood and Density Estimation longitudinal 1.1.13 Analysis of Multiple Time Course Data loo 2.5.1 Efficient Leave-One-Out Cross-Validation and WAIC for Bayesian Models lotri 0.4.2 A Simple Way to Specify Symmetric, Block Diagonal Matrices lpSolve 5.6.15 Interface to 'Lp_solve' v. 5.5 to Solve Linear/Integer Programs LSD 4.1-0 Lots of Superior Depictions ltm 1.2-0 Latent Trait Models under IRT lubridate 1.8.0 Make Dealing with Dates a Little Easier lumi 2.48.0 BeadArray Specific Methods for Illumina Methylation and Expression Microarrays lwgeom 0.2-8 Bindings to Selected 'liblwgeom' Functions for Simple Features made4 1.70.0 Multivariate analysis of microarray data using ADE4 magic 1.6-0 Create and Investigate Magic Squares magrittr 2.0.3 A Forward-Pipe Operator for R maps 3.4.0 Draw Geographical Maps maptools 1.1-4 Tools for Handling Spatial Objects maptree 1.4-8 Mapping, Pruning, and Graphing Tree Models markdown 1.1 Render Markdown with the C Library 'Sundown' marray 1.74.0 Exploratory analysis for two-color spotted microarray data MASS 7.3-56 Support Functions and Datasets for Venables and Ripley's MASS mathjaxr 1.6-0 Using 'Mathjax' in Rd Files Matrix 1.5-3 Sparse and Dense Matrix Classes and Methods MatrixGenerics 1.8.0 S4 Generic Summary Statistic Functions that Operate on Matrix-Like Objects MatrixModels 0.5-0 Modelling with Sparse and Dense Matrices matrixStats 0.62.0 Functions that Apply to Rows and Columns of Matrices (and to Vectors) mclust 5.4.10 Gaussian Mixture Modelling for Model-Based Clustering, Classification, and Density Estimation mcmc 0.9-7 Markov Chain Monte Carlo memoise 2.0.1 'Memoisation' of Functions MendelianRandomization 0.6.0 Mendelian Randomization Package metadat 1.2-0 Meta-Analysis Datasets metafor 3.4-0 Meta-Analysis Package for R methods 4.2.0 Formal Methods and Classes methylumi 2.42.0 Handle Illumina methylation data mgcv 1.8-40 Mixed GAM Computation Vehicle with Automatic Smoothness Estimation mime 0.12 Map Filenames to MIME Types minfi 1.42.0 Analyze Illumina Infinium DNA methylation arrays miniUI 0.1.1.1 Shiny UI Widgets for Small Screens minpack.lm 1.2-2 R Interface to the Levenberg-Marquardt Nonlinear Least-Squares Algorithm Found in MINPACK, Plus Support for Bounds minqa 1.2.4 Derivative-free optimization algorithms by quadratic approximation missMethyl 1.30.0 Analysing Illumina HumanMethylation BeadChip Data mitools 2.4 Tools for Multiple Imputation of Missing Data mlr 2.19.0 Machine Learning in R mnormt 2.0.2 The Multivariate Normal and t Distributions, and Their Truncated Versions ModelMetrics 1.2.2.2 Rapid Calculation of Model Metrics modelr 0.1.8 Modelling Functions that Work with the Pipe modeltools 0.2-23 Tools and Classes for Statistical Models msm 1.6.9 Multi-State Markov and Hidden Markov Models in Continuous Time mstate 0.3.2 Data Preparation, Estimation and Prediction in Multi-State Models multcomp 1.4-19 Simultaneous Inference in General Parametric Models multtest 2.52.0 Resampling-based multiple hypothesis testing munsell 0.5.0 Utilities for Using Munsell Colours mvtnorm 1.1-3 Multivariate Normal and t Distributions n1qn1 6.0.1-11 Port of the 'Scilab' 'n1qn1' Module for Unconstrained BFGS Optimization nleqslv 3.3.2 Solve Systems of Nonlinear Equations nlme 3.1-157 Linear and Nonlinear Mixed Effects Models nlmixr2 2.0.8 Nonlinear Mixed Effects Models in Population PK/PD nlmixr2data 2.0.7 Nonlinear Mixed Effects Models in Population PK/PD, Data nlmixr2est 2.1.3 Nonlinear Mixed Effects Models in Population PK/PD, Estimation Routines nlmixr2extra 2.0.8 Nonlinear Mixed Effects Models in Population PK/PD, Extra Support Functions nlmixr2plot 2.0.7 Nonlinear Mixed Effects Models in Population PK/PD, Plot Functions nloptr 2.0.2 R Interface to NLopt NLP 0.2-1 Natural Language Processing Infrastructure nnet 7.3-17 Feed-Forward Neural Networks and Multinomial Log-Linear Models nor1mix 1.3-0 Normal aka Gaussian (1-d) Mixture Models (S3 Classes and Methods) npde 3.2 Normalised Prediction Distribution Errors for Nonlinear Mixed-Effect Models numDeriv 2016.8-1.1 Accurate Numerical Derivatives OPE 0.7 Outer-product emulator openssl 2.0.2 Toolkit for Encryption, Signatures and Certificates Based on OpenSSL optextras 2019-12.4 Tools to Support Optimization Possibly with Bounds and Masks optparse 1.7.1 Command Line Option Parser org.Hs.eg.db 3.15.0 Genome wide annotation for Human pander 0.6.5 An R 'Pandoc' Writer parallel 4.2.0 Support for Parallel computation in R parallelly 1.31.1 Enhancing the 'parallel' Package parallelMap 1.5.1 Unified Interface to Parallelization Back-Ends ParamHelpers 1.14 Helpers for Parameters in Black-Box Optimization, Tuning and Machine Learning paran 1.5.2 Horn's Test of Principal Components/Factors pbapply 1.5-0 Adding Progress Bar to '*apply' Functions pbkrtest 0.5.1 Parametric Bootstrap, Kenward-Roger and Satterthwaite Based Methods for Test in Mixed Models pcaPP 2.0-1 Robust PCA by Projection Pursuit pegas 1.1 Population and Evolutionary Genetics Analysis System permute 0.9-7 Functions for Generating Restricted Permutations of Data phangorn 2.8.1 Phylogenetic Reconstruction and Analysis phia 0.2-1 Post-Hoc Interaction Analysis pillar 1.7.0 Coloured Formatting for Columns pixmap 0.4-12 Bitmap Images / Pixel Maps pkgbuild 1.3.1 Find Tools Needed to Build R Packages pkgconfig 2.0.3 Private Configuration for 'R' Packages pkgload 1.2.4 Simulate Package Installation and Attach pkgmaker 0.32.2 Development Utilities for R Packages plinkQC 0.3.4 Genotype Quality Control with 'PLINK' plogr 0.2.0 The 'plog' C++ Logging Library plotly 4.10.0 Create Interactive Web Graphics via 'plotly.js' pls 2.8-0 Partial Least Squares and Principal Component Regression plyr 1.8.7 Tools for Splitting, Applying and Combining Data png 0.1-7 Read and write PNG images poLCA 1.6.0.1 Polytomous Variable Latent Class Analysis polspline 1.1.20 Polynomial Spline Routines polyclip 1.10-0 Polygon Clipping polycor 0.8-1 Polychoric and Polyserial Correlations posterior 1.2.1 Tools for Working with Posterior Distributions poweRlaw 0.70.6 Analysis of Heavy Tailed Distributions prabclus 2.3-2 Functions for Clustering and Testing of Presence-Absence, Abundance and Multilocus Genetic Data pracma 2.3.8 Practical Numerical Math Functions praise 1.0.0 Praise Users PreciseSums 0.5 Accurate Floating Point Sums and Products preprocessCore 1.58.0 A collection of pre-processing functions prettydoc 0.4.1 Creating Pretty Documents from R Markdown prettyunits 1.1.1 Pretty, Human Readable Formatting of Quantities pROC 1.18.0 Display and Analyze ROC Curves processx 3.5.3 Execute and Control System Processes prodlim 2019.11.13 Product-Limit Estimation for Censored Event History Analysis profileModel 0.6.1 Profiling Inference Functions for Various Model Classes progress 1.2.2 Terminal Progress Bars progressr 0.10.0 An Inclusive, Unifying API for Progress Updates promises 1.2.0.1 Abstractions for Promise-Based Asynchronous Programming ProtGenerics 1.28.0 Generic infrastructure for Bioconductor mass spectrometry packages proto 1.0.0 Prototype Object-Based Programming protolite 2.1.1 Highly Optimized Protocol Buffer Serializers proxy 0.4-26 Distance and Similarity Measures ps 1.7.0 List, Query, Manipulate System Processes PSCBS 0.66.0 Analysis of Parent-Specific DNA Copy Numbers pspline 1.0-19 Penalized Smoothing Splines psych 2.2.5 Procedures for Psychological, Psychometric, and Personality Research purrr 0.3.4 Functional Programming Tools qs 0.25.4 Quick Serialization of R Objects quadprog 1.5-8 Functions to Solve Quadratic Programming Problems quanteda 3.2.1 Quantitative Analysis of Textual Data quanteda.corpora 0.9.2 A collection of corpora for quanteda quantmod 0.4.20 Quantitative Financial Modelling Framework quantreg 5.93 Quantile Regression qvalue 2.28.0 Q-value estimation for false discovery rate control qvcalc 1.0.2 Quasi Variances for Factor Effects in Statistical Models R.cache 0.15.0 Fast and Light-Weight Caching (Memoization) of Objects and Results to Speed Up Computations R.devices 2.17.0 Unified Handling of Graphics Devices R.filesets 2.14.0 Easy Handling of and Access to Files Organized in Structured Directories R.huge 0.9.0 Methods for Accessing Huge Amounts of Data [deprecated] R.methodsS3 1.8.1 S3 Methods Simplified R.oo 1.24.0 R Object-Oriented Programming with or without References R.rsp 0.44.0 Dynamic Generation of Scientific Reports R.utils 2.11.0 Various Programming Utilities R2HTML 2.3.3 HTML Exportation for R Objects R2WinBUGS 2.1-21 Running 'WinBUGS' and 'OpenBUGS' from 'R' / 'S-PLUS' R6 2.5.1 Encapsulated Classes with Reference Semantics randomForest 4.7-1.1 Breiman and Cutler's Random Forests for Classification and Regression rapidjsonr 1.2.0 'Rapidjson' C++ Header Files RApiSerialize 0.1.2 R API Serialization rappdirs 0.3.3 Application Directories: Determine Where to Save Data, Caches, and Logs raster 3.5-15 Geographic Data Analysis and Modeling RBGL 1.72.0 An interface to the BOOST graph library rcmdcheck 1.4.0 Run 'R CMD check' from 'R' and Capture Results RColorBrewer 1.1-3 ColorBrewer Palettes Rcpp 1.0.9 Seamless R and C++ Integration RcppArmadillo 0.11.4.2.1 'Rcpp' Integration for the 'Armadillo' Templated Linear Algebra Library RcppEigen 0.3.3.9.2 'Rcpp' Integration for the 'Eigen' Templated Linear Algebra Library RcppGSL 0.3.11 'Rcpp' Integration for 'GNU GSL' Vectors and Matrices RcppParallel 5.1.5 Parallel Programming Tools for 'Rcpp' RcppZiggurat 0.1.6 'Rcpp' Integration of Different \"Ziggurat\" Normal RNG Implementations RCurl 1.98-1.6 General Network (HTTP/FTP/...) Client Interface for R readr 2.1.2 Read Rectangular Text Data readxl 1.4.0 Read Excel Files recipes 0.2.0 Preprocessing and Feature Engineering Steps for Modeling regioneR 1.28.0 Association analysis of genomic regions based on permutation tests registry 0.5-1 Infrastructure for R Package Registries relaimpo 2.2-6 Relative Importance of Regressors in Linear Models rematch 1.0.1 Match Regular Expressions with a Nicer 'API' rematch2 2.1.2 Tidy Output from Regular Expression Matching remotes 2.4.2 R Package Installation from Remote Repositories, Including 'GitHub' renv 0.15.4 Project Environments Repitools 1.42.0 Epigenomic tools reprex 2.0.1 Prepare Reproducible Example Code via the Clipboard reshape 0.8.9 Flexibly Reshape Data reshape2 1.4.4 Flexibly Reshape Data: A Reboot of the Reshape Package restfulr 0.0.13 R Interface to RESTful Web Services rex 1.2.1 Friendly Regular Expressions Rfast 2.0.6 A Collection of Efficient and Extremely Fast R Functions rgdal 1.5-32 Bindings for the 'Geospatial' Data Abstraction Library rgeos 0.5-9 Interface to Geometry Engine - Open Source ('GEOS') rgl 0.108.3.2 3D Visualization Using OpenGL Rgraphviz 2.40.0 Provides plotting capabilities for R graph objects rhdf5 2.40.0 R Interface to HDF5 rhdf5filters 1.8.0 HDF5 Compression Filters Rhdf5lib 1.18.2 hdf5 library as an R package Rhtslib 1.28.0 HTSlib high-throughput sequencing library as an R package Ringo 1.60.0 R Investigation of ChIP-chip Oligoarrays rjson 0.2.21 JSON for R RJSONIO 1.3-1.6 Serialize R Objects to JSON, JavaScript Object Notation rlang 1.0.2 Functions for Base Types and Core R and 'Tidyverse' Features rlecuyer 0.3-5 R Interface to RNG with Multiple Streams rlist 0.4.6.2 A Toolbox for Non-Tabular Data Manipulation rmarkdown 2.14 Dynamic Documents for R Rmpfr 0.8-7 R MPFR - Multiple Precision Floating-Point Reliable Rmpi 0.6-9.2 Interface (Wrapper) to MPI (Message-Passing Interface) rms 6.3-0 Regression Modeling Strategies RMySQL 0.10.23 Database Interface and 'MySQL' Driver for R RNetCDF 2.5-2 Interface to 'NetCDF' Datasets rngtools 1.5.2 Utility Functions for Working with Random Number Generators robustbase 0.95-0 Basic Robust Statistics ROC 1.72.0 utilities for ROC, with microarray focus rootSolve 1.8.2.3 Nonlinear Root Finding, Equilibrium and Steady-State Analysis of Ordinary Differential Equations roxygen2 7.2.0 In-Line Documentation for R rpart 4.1.16 Recursive Partitioning and Regression Trees RPMM 1.25 Recursively Partitioned Mixture Model rprojroot 2.0.3 Finding Files in Project Subdirectories Rsamtools 2.12.0 Binary alignment (BAM), FASTA, variant call (BCF), and tabix file import Rsolnp 1.16 General Non-Linear Optimization RSQLite 2.2.14 SQLite Interface for R rstan 2.21.5 R Interface to Stan rstanarm 2.21.3 Bayesian Applied Regression Modeling via Stan rstantools 2.2.0 Tools for Developing R Packages Interfacing with 'Stan' rstudioapi 0.13 Safely Access the RStudio API Rsubread 2.10.1 Mapping, quantification and variant analysis of sequencing data rtracklayer 1.56.0 R interface to genome annotation files and the UCSC genome browser ruv 0.9.7.1 Detect and Remove Unwanted Variation using Negative Controls rversions 2.1.1 Query 'R' Versions, Including 'r-release' and 'r-oldrel' rvest 1.0.2 Easily Harvest (Scrape) Web Pages Rvmmin 2018-4.17.1 Variable Metric Nonlinear Function Minimization rxode2 2.0.11 Facilities for Simulating from ODE-Based Models rxode2et 2.0.9 Event Table Functions for 'rxode2' rxode2ll 2.0.9 Log-Likelihood Functions for 'rxode2' rxode2parse 2.0.13 Parsing and Code Generation Functions for 'rxode2' rxode2random 2.0.9 Random Number Generation Functions for 'rxode2' s2 1.0.7 Spherical Geometry Operators Using the S2 Geometry Library S4Vectors 0.34.0 Foundation of vector-like and list-like containers in Bioconductor saemix 3.0 Stochastic Approximation Expectation Maximization (SAEM) Algorithm sampling 2.9 Survey Sampling sandwich 3.0-1 Robust Covariance Matrix Estimators sass 0.4.1 Syntactically Awesome Style Sheets ('Sass') scales 1.2.0 Scale Functions for Visualization scam 1.2-12 Shape Constrained Additive Models scatterplot3d 0.3-41 3D Scatter Plot schoolmath 0.4.1 Functions and Datasets for Math Used in School scrime 1.3.5 Analysis of High-Dimensional Categorical Data Such as SNP Data segmented 1.5-0 Regression Models with Break-Points / Change-Points Estimation selectr 0.4-2 Translate CSS Selectors to XPath Expressions sendmailR 1.2-1 send email using R seqinr 4.2-16 Biological Sequences Retrieval and Analysis seqLogo 1.62.0 Sequence logos for DNA sequence alignments sessioninfo 1.2.2 R Session Information sf 1.0-7 Simple Features for R sfheaders 0.4.0 Converts Between R Objects and Simple Feature Objects sfsmisc 1.1-13 Utilities from 'Seminar fuer Statistik' ETH Zurich shape 1.4.6 Functions for Plotting Graphical Shapes, Colors shiny 1.7.1 Web Application Framework for R shinyjs 2.1.0 Easily Improve the User Experience of Your Shiny Apps in Seconds shinystan 2.6.0 Interactive Visual and Numerical Diagnostics and Posterior Analysis for Bayesian Models shinythemes 1.2.0 Themes for Shiny ShortRead 1.54.0 FASTQ input and manipulation siggenes 1.70.0 Multiple Testing using SAM and Efron's Empirical Bayes Approaches sitmo 2.0.2 Parallel Pseudo Random Number Generator (PPRNG) 'sitmo' Header Files slam 0.1-50 Sparse Lightweight Arrays and Matrices sn 2.0.2 The Skew-Normal and Related Distributions Such as the Skew-t and the SUN snow 0.4-4 Simple Network of Workstations SnowballC 0.7.0 Snowball Stemmers Based on the C 'libstemmer' UTF-8 Library snowfall 1.84-6.1 Easier cluster computing (based on snow). softImpute 1.4-1 Matrix Completion via Iterative Soft-Thresholded SVD sourcetools 0.1.7 Tools for Reading, Tokenizing and Parsing R Code sp 1.4-7 Classes and Methods for Spatial Data spam 2.8-0 SPArse Matrix SparseM 1.81 Sparse Linear Algebra sparseMatrixStats 1.8.0 Summary Statistics for Rows and Columns of Sparse Matrices spatial 7.3-15 Functions for Kriging and Point Pattern Analysis spatstat 2.3-4 Spatial Point Pattern Analysis, Model-Fitting, Simulation, Tests spatstat.core 2.4-4 Core Functionality of the 'spatstat' Family spatstat.data 2.2-0 Datasets for 'spatstat' Family spatstat.geom 2.4-0 Geometrical Functionality of the 'spatstat' Family spatstat.linnet 2.3-2 Linear Networks Functionality of the 'spatstat' Family spatstat.random 2.2-0 Random Generation Functionality for the 'spatstat' Family spatstat.sparse 2.1-1 Sparse Three-Dimensional Arrays and Linear Algebra Utilities spatstat.utils 2.3-1 Utility Functions for 'spatstat' spData 2.0.1 Datasets for Spatial Analysis spdep 1.2-4 Spatial Dependence: Weighting Schemes, Statistics splines 4.2.0 Regression Spline Functions and Classes SQUAREM 2021.1 Squared Extrapolation Methods for Accelerating EM-Like Monotone Algorithms stabledist 0.7-1 Stable Distribution Functions StanHeaders 2.21.0-7 C++ Header Files for Stan stars 0.5-5 Spatiotemporal Arrays, Raster and Vector Data Cubes startupmsg 0.9.6 Utilities for Start-Up Messages statmod 1.4.36 Statistical Modeling stats 4.2.0 The R Stats Package stats4 4.2.0 Statistical Functions using S4 Classes stopwords 2.3 Multilingual Stopword Lists stringdist 0.9.8 Approximate String Matching, Fuzzy Text Search, and String Distance Functions stringfish 0.15.7 Alt String Implementation stringi 1.7.6 Character String Processing Facilities stringr 1.4.0 Simple, Consistent Wrappers for Common String Operations SummarizedExperiment 1.26.1 SummarizedExperiment container survey 4.1-1 Analysis of Complex Survey Samples survival 3.3-1 Survival Analysis sva 3.44.0 Surrogate Variable Analysis symengine 0.2.2 Interface to the 'SymEngine' Library sys 3.4 Powerful and Reliable Tools for Running System Commands in R systemfit 1.1-24 Estimating Systems of Simultaneous Equations TAM 4.0-16 Test Analysis Modules tcltk 4.2.0 Tcl/Tk Interface tensor 1.5 Tensor product of arrays tensorA 0.36.2 Advanced Tensor Arithmetic with Named Indices terra 1.5-21 Spatial Data Analysis testthat 3.1.4 Unit Testing for R tgp 2.4-18 Bayesian Treed Gaussian Process Models TH.data 1.1-1 TH's Data Archive threejs 0.3.3 Interactive 3D Scatter Plots, Networks and Globes tibble 3.1.7 Simple Data Frames tictoc 1.0.1 Functions for Timing R Scripts, as Well as Implementations of Stack and List Structures tidyr 1.2.0 Tidy Messy Data tidyselect 1.1.2 Select from a Set of Strings tidyverse 1.3.1 Easily Install and Load the 'Tidyverse' timeDate 3043.102 Rmetrics - Chronological and Calendar Objects tinytex 0.39 Helper Functions to Install and Maintain TeX Live, and Compile LaTeX Documents tkrplot 0.0-26 TK Rplot tm 0.7-8 Text Mining Package tmap 3.3-3 Thematic Maps tmaptools 3.1-1 Thematic Map Tools tmvnsim 1.0-2 Truncated Multivariate Normal Simulation tools 4.2.0 Tools for Package Development topicmodels 0.2-12 Topic Models triebeard 0.3.0 'Radix' Trees in 'Rcpp' truncnorm 1.0-8 Truncated Normal Distribution trust 0.1-8 Trust Region Optimization tseries 0.10-51 Time Series Analysis and Computational Finance TTR 0.24.3 Technical Trading Rules tweenr 1.0.2 Interpolate Data for Smooth Animations TxDb.Hsapiens.UCSC.hg19.knownGene 3.2.2 Annotation package for TxDb object(s) tzdb 0.3.0 Time Zone Database Information ucminf 1.1-4.1 General-Purpose Unconstrained Non-Linear Optimization udunits2 0.13.2.1 Udunits-2 Bindings for R units 0.8-0 Measurement Units for R Vectors UpSetR 1.4.0 A More Scalable Alternative to Venn and Euler Diagrams for Visualizing Intersecting Sets urca 1.3-0 Unit Root and Cointegration Tests for Time Series Data urltools 1.7.3 Vectorised Tools for URL Handling and Parsing usethis 2.1.6 Automate Package and Project Setup utf8 1.2.2 Unicode Text Processing utils 4.2.0 The R Utils Package uuid 1.1-0 Tools for Generating and Handling of UUIDs V8 4.2.0 Embedded JavaScript and WebAssembly Engine for R VariantAnnotation 1.42.1 Annotation of Genetic Variants vctrs 0.4.1 Vector Helpers vegan 2.6-2 Community Ecology Package VGAM 1.1-6 Vector Generalized Linear and Additive Models VineCopula 2.4.4 Statistical Inference of Vine Copulas viridis 0.6.2 Colorblind-Friendly Color Maps for R viridisLite 0.4.0 Colorblind-Friendly Color Maps (Lite Version) vpc 1.2.2 Create Visual Predictive Checks vroom 1.5.7 Read and Write Rectangular Text Data Quickly vsn 3.64.0 Variance stabilization and calibration for microarray data waldo 0.4.0 Find Differences Between R Objects wateRmelon 2.2.0 Illumina 450 and EPIC methylation array normalization and metrics WGCNA 1.71 Weighted Correlation Network Analysis whisker 0.4 {{mustache}} for R, Logicless Templating widgetframe 0.3.1 'Htmlwidgets' in Responsive 'iframes' widgetTools 1.74.0 Creates an interactive tcltk widget withr 2.5.0 Run Code 'With' Temporarily Modified Global State wk 0.6.0 Lightweight Well-Known Geometry Parsing xfun 0.31 Supporting Functions for Packages Maintained by 'Yihui Xie' xgxr 1.1.1 Exploratory Graphics for Pharmacometrics XML 3.99-0.9 Tools for Parsing and Generating XML Within R and S-Plus xml2 1.3.3 Parse XML xopen 1.0.0 Open System Files, 'URLs', Anything xtable 1.8-4 Export Tables to LaTeX or HTML xts 0.12.1 eXtensible Time Series XVector 0.36.0 Foundation of external vector representation and manipulation in Bioconductor yaml 2.3.5 Methods to Convert R Data to YAML and Back zip 2.2.0 Cross-Platform 'zip' Compression zlibbioc 1.42.0 An R packaged zlib-1.2.5 zoo 1.8-10 S3 Infrastructure for Regular and Irregular Time Series (Z's Ordered Observations)"},{"location":"Other_Services/Aristotle/","title":"Aristotle","text":""},{"location":"Other_Services/Aristotle/#overview","title":"Overview","text":"

Aristotle is an interactive, Linux-based compute service for teaching, running on four nodes, each with 128 gigabytes of RAM and 16 cores. The nodes run the Red Hat Enterprise Linux operating system (RHEL 7) and have a subset of the RCPS software stack available.

The main aim of this service is to allow specific teaching courses to run that need to run Linux/UNIX applications, but it is available to all UCL users.

Warning

Aristotle is made available but is provided with minimal staff time and no budget. Any user may completely occupy the service and there is no system in place to prevent that.

"},{"location":"Other_Services/Aristotle/#access","title":"Access","text":"

Anyone with a UCL userid and within the UCL institutional firewall can access Aristotle by connecting via ssh to:

aristotle.rc.ucl.ac.uk\n

This address can point to more than one actual server (via DNS round-robin). To connect to a specific server from the set, you will need to know its number: for example, the second server has the address aristotle02.rc.ucl.ac.uk. When you connect, you should be shown which one you are connected to on your command line.

The userid and password you need to connect with are those provided to you by Information Services Division.

If you experience difficulties with your login, please make sure that you are typing your UCL user ID and your password correctly. If you still cannot get access, please contact us at rc-support@ucl.ac.uk.

If you are outside the UCL firewall, you will need to connect to the Gateway system first and then SSH in to Aristotle from there.

"},{"location":"Other_Services/Aristotle/#user-environment","title":"User Environment","text":"

Aristotle runs Red Hat Enterprise Linux 7 and NFS mounts the RCPS Software Stack. As this machine is intended for teaching, work has focused on getting specific applications required for specific courses to work and these are:

Packages are available through modules and users should consult the relevant modules documentation.

"},{"location":"Other_Services/JupyterHub/","title":"JupyterHub","text":""},{"location":"Other_Services/JupyterHub/#overview","title":"Overview","text":"

The Research Computing team provide a JupyterHub service (jupyter.org) primarily to support teaching and is part of the Data Science Platform. It uses the central UNIX filestore for user data.

"},{"location":"Other_Services/JupyterHub/#access","title":"Access","text":"

To access the service go to the following URL:

https://jupyter.data-science.rc.ucl.ac.uk/

and login with your UCL userid and password.

The Data Science Platform and all of its components are only accessible from the UCL network. When off campus, you will need to connect to the UCL VPN first or when working from China, the China Connect Service.

Please note: The main aim of this service is to support teaching and the service should not be used for computationally intensive research. If your use starts to affect the experience of other users, we reserve the right to terminate sessions without notice. For computationally intensive research you should be using the Myriad Cluster.

"},{"location":"Other_Services/RStudio/","title":"RStudio","text":""},{"location":"Other_Services/RStudio/#overview","title":"Overview","text":"

The Research Computing team currently runs 2 supported instances of RStudio:

"},{"location":"Other_Services/RStudio/#httpsrstudiodata-sciencercuclacuk","title":"https://rstudio.data-science.rc.ucl.ac.uk/","text":"

This instance is for general teaching use and is part of the Data Science Platform. It uses the central UNIX filestore for user data. Access is blocked by default due to licencing restrictions.

Staff who would like to use the service to deliver teaching should email rc-support@ucl.ac.uk to request access (please include your UCL username). To grant access to students, please pre-register the course by emailing the above address and provide the SITS code of the relevant module(s) and a pdf or link to the syllabus. Students who are registered on those SITS modules will then be added automatically.

In addition we have a smalll number of Named Researcher licenses for RStudio Pro which also run on this instance. Staff and Research Postgraduates who would like to have access to one of these licenses should email rc-support@ucl.ac.uk to request access explaining why they need to use RStudio.

The Data Science Platform and all of its components are only accessible from the UCL network. When off campus, you will need to connect to the UCL VPN first or when working from China, the China Connect Service.

Please note: The main aim of this service is to support teaching and the service should not be used for computationally intensive research. If your use starts to affect the experience of other users, we reserve the right to terminate sessions without notice. For computationally intensive research you should be using the Myriad Cluster.

"},{"location":"Other_Services/RStudio/#httpsecon-myriadrcuclacuk","title":"https://econ-myriad.rc.ucl.ac.uk/","text":"

This instance is for research use by members of the Economics department. It uses the Myriad filesystem for user data.

"},{"location":"Other_Services/RStudio/#installing-r-packages","title":"Installing R Packages","text":"

Users can install R packages in their home directories, but we recommend that if you are planning on teaching a course, you make sure the packages you want to use are pre-installed on the system. This saves time and reduces the load on the server.

The main version of R used by the RStudio server copies it's packages from the Myriad Cluster, so any package available there should be available in RStudio too. There's an automatically updated list here: R packages. Alternatively, check the list of available packages from within RStudio itself.

Requests to install packages can be sent to rc-support@ucl.ac.uk. Please give as much notice as possible when requesting packages as these requests will be handled on a best efforts basis by the research computing applications support team.

"},{"location":"Other_Services/RStudio/#troubleshooting-and-problem-pre-emption","title":"Troubleshooting and problem pre-emption","text":"

For all of the services, please take care to either run q() in the R window or press the red logout button in the top right hand corner when you are done with the window, DO NOT just close the tab. This decreases the chance of there being stale sessions and future issues with logging in.

"},{"location":"Other_Services/RStudio/#not-being-able-to-reach-the-landing-login-page","title":"Not being able to reach the landing (login) page","text":"

If you cannot reach the landing page, then please first try getting there using private browsing and if that works then clear your cookies and cache. In most browsers you can do this for a certain time range, though look at the documentation for the browser you are using.

"},{"location":"Other_Services/RStudio/#r-session-not-starting-or-rstudio-initialisation-error","title":"R session not starting or RStudio Initialisation Error","text":"

If you get an error pop-up RStudio Initialisation Error: Unable to connect to service or an ever-spinning loading screen you can try and resolve the problem using one of the methods below or get in touch with RC support.

There are 2 courses of action for the 2 supported services:

~/.local/share/rstudio/sessions/

If doing this doesn't resolve your issues, get in touch with RC support .

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/","title":"UCL UK e-Science Certificates","text":"

UCL Information Services serves as a local Registration Authority for the authentication of applications for e-Science Certificates.

A valid e-Science certificate is required to gain access to the resources of the National e-Infrastructure Service (NES) (amongst others).

Brief information to help you in applying for an e-Science Certificate is provided below. More detailed information can be found on the NGS Support website.

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#scope-of-the-ucl-registration-authority","title":"Scope of the UCL Registration Authority","text":"

In general, the UCL Registration Authority (RA) can only approve personal and server certificate requests for members of UCL and those associated with projects based at UCL. However we have approved personal certificate requests for members of other London institutions without local RAs on request.

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#before-you-apply-for-a-certificate","title":"Before you Apply for a Certificate","text":"

The recommended method for applying for a certificate is to use the Certificate Wizard. So:

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#applying-for-a-certificate","title":"Applying for a Certificate","text":"

You will be asked for a number of items when completing your request including:

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#extra-items-for-server-certificates","title":"Extra items for Server Certificates","text":"

There are two extra items for certificates for servers:

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#after-your-request-has-been-submitted","title":"After Your Request has been Submitted","text":"

After you have submitted your request, it has to be authenticated by your Registration Authority (RA) before the certificate is issued by the UK e-Science Certification Authority (CA). For authentication the UK e-Science CA requires that you present yourself in person to your RA with an appropriate form of photo-ID and your PIN. You will be asked to explain why you need a UK e-Science Certificate.

The RA for UCL is based in Informations Services Division (ISD). To arrange an appointment please email grid-ra AT ucl.ac.uk in the first instance.

Valid forms of Photo-ID are any of the following:

We are required to make and log a photocopy of your photo-ID.

If you have none of the above forms of photo-ID available, contact us for advice by e-mail at grid-ra AT ucl.ac.uk. Please don't just turn up with an alternative as we may not be able to accept it.

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#extra-requirements-for-students","title":"Extra Requirements for Students","text":"

In addition to the above, students should provide a letter (on department paper) from their project supervisor explaining why they need a certificate.

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#extra-requirements-for-servers","title":"Extra Requirements for Servers","text":"

In addition to the above, you need to provide a letter from your department explaining that you are responsible for this server. The letter should be on departmental stationary and be signed by your head of department.

"},{"location":"Other_Services/UCL_UK_e-Science_Certificates/#obtaining-your-certificate","title":"Obtaining Your Certificate","text":"

After your request has been authenticated by your Registration Authority, it is forwarded to the UK e-Science Certification Authority for final creation (this stage is called signing the certificate). Signing is normally done on the same or next working day.

When your certificate is ready the CA will e-mail you using the e-mail address that you provided with details of how to download your certificate. If you used the recommend method to request it, then you can download it into the Certificate Wizard application using the Refresh button.

You should now make a backup of your certificate using the Export button in the Certificate Wizard application.

"},{"location":"Paid-For_Resources/How_to_Use/","title":"Using Paid-For Resources","text":"

Paid resources may be in the form of priority access (Gold), dedicated nodes or both.

Users with access to Gold have tools for monitoring budgets and usage.

Users with access to dedicated nodes have a tool for monitoring current use of their nodes.

"},{"location":"Paid-For_Resources/How_to_Use/#why-we-recommend-priority-access","title":"Why we recommend priority access","text":"

We recommend priority access via Gold rather than dedicated nodes in most circumstances because:

"},{"location":"Paid-For_Resources/How_to_Use/#priority-access-via-gold","title":"Priority access via Gold","text":"

If you have priority access, this is managed by a resource called Gold. Gold is divided into allocations, which have start and end dates and a given amount of Gold associated with them. Any unused Gold expires after the allocation period ends.

On Myriad, one Gold = one core hour. If you run a job that asks for a wallclock time of 3hrs and 10 cores, then the job costs 30 Gold.

Gold is reserved when you submit a job, so the entire 30 will be reserved and taken away from the available balance when you run qsub. After the job ends, how long it ran for is checked, and any unused Gold is put back for anyone to use.

For example, you asked for 3 hours and 10 cores, but your job finished in 2 hours. When you submit the job, it will reserve 30 Gold and your budget will go down by 30. When it ends, the final charge is only 20 Gold, so 10 Gold gets put back in your available budget.

"},{"location":"Paid-For_Resources/How_to_Use/#view-your-gold-budgets","title":"View your Gold budgets","text":"

To see the Gold budgets available to you, run:

budgets\n

You will see something like this:

Project    Machines  Balance\n---------  --------  -----------\nhpc.999     ANY       124560.00\nhpc.998     ANY       0.00\n

The project column shows which budgets you have access to and the balance shows how much is left unused in the current allocation.

"},{"location":"Paid-For_Resources/How_to_Use/#jobscript-additions-for-gold-jobs","title":"Jobscript additions for Gold jobs","text":"

You choose whether you want a specific job to be a Gold job or a normal priority job. For a Gold job, add these to your jobscript:

#$ -P Gold\n#$ -A hpc.xx\n

You can also pass these in on the command line to the qsub and qrsh commands: 

qsub -P Gold -A hpc.xx myscript.sh\n
"},{"location":"Paid-For_Resources/How_to_Use/#viewing-allocation-dates","title":"Viewing allocation dates","text":"

You can look at all the start and end dates for your allocations:

glsalloc -p hpc.xx\n

Output will look like this:

Id  Account Projects StartTime  EndTime    Amount    Deposited Description    \n--- ------- -------- ---------- ---------- --------- --------- -------------- \n001 01      hpc.999  -infinity  infinity        0.00      0.00 Auto-Generated \n002 01      hpc.999  2021-12-01 2022-03-01 105124.00 205124.00                \n003 01      hpc.999  2022-03-01 2022-06-01 205124.00 205124.00                \n004 01      hpc.999  2022-06-01 2022-09-01 205124.00 205124.00 \n

Allocations begin and end at approximately 00:05 on the date mentioned.

"},{"location":"Paid-For_Resources/How_to_Use/#monitoring-gold-usage","title":"Monitoring Gold usage","text":"

You can view some information about when your Gold was used, in which jobs, and by whom.

# show help\ngstatement --man\n\n# show statement between the given dates\ngstatement -p hpc.xx -s 2020-12-01 -e 2021-12-01\n\n# give a summary between the given dates\ngstatement -p hpc.xx -s 2020-12-01 -e 2021-12-01 --summarize\n
"},{"location":"Paid-For_Resources/How_to_Use/#dedicated-nodes","title":"Dedicated nodes","text":"

For dedicated nodes, only members of your project are allowed to run jobs on your node. Your project is usually set by default so you do not need to specify it in your jobscript. You can check this by looking at qstat -j $JOB_ID for an existing job ID, and looking at the project line near the bottom.

"},{"location":"Paid-For_Resources/How_to_Use/#jobscript-additions-for-dedicated-nodes","title":"Jobscript additions for dedicated nodes","text":"

If the project is not being set by default, for a job to be eligible to run on your nodes you will need to specify your project in your jobscript:

#\u00a0Specify\u00a0project\n#$\u00a0-P\u00a0<project>\n

This will allow a job to run on your nodes, but it can also be scheduled on general-use nodes if some are available first. This should be the main way you run jobs.

If you need to, you can force jobs to run on your nodes only. This is suitable when you have arranged policies on your nodes that are different from the normal policies (eg. a longer maximum wallclock time), as it means your policies will be in effect instead of the general policies.

#\u00a0Specify\u00a0paid\u00a0flag\u00a0to\u00a0force\u00a0running\u00a0on\u00a0paid\u00a0nodes\u00a0only,\u00a0with\u00a0your\u00a0policies\n#$\u00a0-l\u00a0paid=1\n
"},{"location":"Paid-For_Resources/How_to_Use/#check-what-is-running-on-your-nodes","title":"Check what is running on your nodes","text":"

We have a script named whatsonmynode, that runs qhost -j for all the nodes belonging to your project, so you can see which nodes you have, what is running on them and from which user.

module\u00a0load\u00a0userscripts\nwhatsonmynode\u00a0<project>\n
"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/","title":"Purchasing in Myriad","text":"

Sept 2023: We cannot buy any nodes or add paid time in Myriad for the time being.

This is because we have no more space in the datacentre to add new hardware, which is needed for priority access/Gold as well as dedicated node purchases.

Please raise your requests with rc-support@ucl.ac.uk but be aware that we cannot confirm our ability to fulfill them until we know what future space we will have available.

Researchers may purchase additional resources to be used as part of the Myriad High Performance Computing cluster if the free service does not meet their needs. These resources can be made available in one of two ways:

Costs will include backend infrastructure \u2013 racks, switches, cables etc \u2013 needed to integrate the compute nodes into the facility. Both options can be costed into research proposals. These costs will be stated in an agreed Statement of Work document giving details of the number of each node type to be purchased, which must be approved by ARC and the purchaser before the purchase can go ahead.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#deadline","title":"Deadline","text":"

The deadline for making requests to purchase Myriad nodes is the end of April. Before this date, researchers wishing to buy compute nodes must approve an agreed Statement of Work and transfer the funds to ARC. We can't place orders after this date because the hardware might not arrive before the end of UCL's financial year.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#faqs","title":"FAQs","text":""},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#can-i-add-the-cost-of-purchasing-nodes-to-a-grant-application","title":"Can I add the cost of purchasing nodes to a grant application?","text":"

If you are putting together a grant application and think that you may need to ask for the cost of additional computing resources to be covered, please contact us. We will be able to assess your requirements, recommend appropriate hardware, and then provide an estimate and a supporting statement.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#can-you-give-me-advice-on-what-hardware-to-buy","title":"Can you give me advice on what hardware to buy?","text":"

Yes, we\u2019d be happy to discuss this with you. Please contact rc-support@ucl.ac.uk.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#what-type-of-nodes-can-we-purchase","title":"What type of nodes can we purchase?","text":"

We currently have three types of node in Myriad, which are well tested and work reliably. These are:

If you require an alternative/custom specification, we can\u2019t guarantee that we will be able to accommodate this on the cluster, but we\u2019re happy to look into it.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#i-want-to-apply-for-more-than-50k-worth-of-equipment-will-we-have-to-go-through-a-tender-process","title":"I want to apply for more than \u00a350k worth of equipment, will we have to go through a tender process?","text":"

No. We have a framework agreement with the vendor which covers all hardware purchases.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#i-know-what-hardware-i-need-can-you-send-me-a-quote","title":"I know what hardware I need. Can you send me a quote?","text":"

Even if you know what hardware you need, before we can send you a quote, we will need to agree on a detailed specification. Please email rc-support@ucl.ac.uk with the following information:

We will then either send you a specification to approve, or ask to meet to discuss your requirements further. Once this is agreed, we aim to get back to you with a quote within two weeks.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#how-do-i-manage-who-has-permission-to-use-our-nodespriority-queue","title":"How do I manage who has permission to use our nodes/priority queue?","text":"

When you purchase nodes or priority cycles, we will ask you for a list of usernames of people who have permission to use the resource \u2014 access is managed using access control lists on Myriad. If your list of users is an entire department, we can automatically generate this list nightly. Resource owners or designated resource administrators can request a change of membership of these groups by submitting a ticket in Remedy Force or emailing rc-support@ucl.ac.uk.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#what-is-the-difference-between-paying-for-priority-cycles-and-purchasing-dedicated-nodes","title":"What is the difference between paying for priority cycles and purchasing dedicated nodes?","text":"

Priority cycles is the better option for most people as it provides greater flexibility: priority cycles can be used across many nodes at once, and there is an entire allocation period to use them. Dedicated hardware however would need to be in use 24/7 in order to get the most out of it. Researchers might want dedicated nodes if they have particular requirements which mean they can only run their work on their own nodes; e.g., they have purchased non-standard nodes, or the software they are using requires a static licence tied to a particular node.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#will-my-3-month-priority-cycle-allocation-roll-over-to-the-next-quarter-if-i-dont-use-it","title":"Will my 3-month priority cycle allocation roll over to the next quarter if I don\u2019t use it?","text":"

No.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#i-want-the-flexibility-of-priority-cycles-but-my-funder-requires-an-invoice-for-specific-hardware","title":"I want the flexibility of priority cycles, but my funder requires an invoice for specific hardware.","text":"

Even if you require an invoice for specific hardware, you can still convert the physical hardware into priority cycles. We will add the nodes you purchased to the general pool and give you the equivalent in priority cycles.

"},{"location":"Paid-For_Resources/Purchasing_in_Myriad/#what-happens-to-nodes-i-have-purchased-once-theyve-reached-the-end-of-warranty","title":"What happens to nodes I have purchased once they\u2019ve reached the end of warranty?","text":"

The hardware is run for the life-time of the service, however if you buy dedicated nodes and they run out of warranty, they will not be replaced.

"},{"location":"Software_Guides/ANSYS/","title":"ANSYS","text":"

License checking

Current as of Dec 2021, the -ac app=cfx license check does not work after ANSYS renamed all products and licenses. Remove this line from your jobscripts for the time being or jobs will not be able to start.

ANSYS/CFX and ANSYS/Fluent are commercial fluid dynamics packages.

The most recent version of ANSYS we have installed on the clusters is ANSYS 2021.R2 including ANSYS/CFX, ANSYS/Fluent, ANSYS Mechanical and the ANSYS Electromagnetics Suite (AnsysEM). Some other products included in the ANSYS Campus Agreement are also available including Autodyn.

Older versions of some ANSYS products including ANSYS/CFX and ANSYS/Fluent are also available - 2019.R3, 19.1 for example. However ANSYS Inc changed the way they license their software at the beginning of 2021, causing some products from versions before 2020 to have issues getting a valid license from the lciense server.

Before these applications can be run, the user needs to go though a number of set up steps. These are detailed here.

To see the versions available, type

module avail ansys\n

The desired ANSYS module needs to be loaded by issuing a command like:

module\u00a0load\u00a0ansys/2021.r2\n

This will set up various necessary config directories for you.

Module for UCL staff and Students only

these ANSYS modules are for UCL staff and Students only as they use the UCL ANSYS License Manager. Users from external Institutions on Young (the MMM Hub cluster) need to use modules specific to their institution instead.

The ANSYS applications are intended to be run primarily within batch jobs however you may run short (less than 5 minutes execution time) interactive tests on the Login Nodes and longer (up to two hours) on an interactive session on a compute node using qrsh. Interactive work can be done using the ANSYS interactive tools provided you have X-windows functionality enabled though your ssh connection. See our User Guide for more information about enabling X-windows functionality and using the qrsh command to start interactive sessions.

UCL's campus-wide license covers 125 instances with 512 HPC licenses (for parallel jobs) available for running CFX, Fluent and AnsysEM jobs and in order to make sure that jobs only run if there are licenses available, it is necessary for users to request ANSYS licenses with their jobs, by adding -ac app=cfx to their job submission.

"},{"location":"Software_Guides/ANSYS/#ansyscfx","title":"ANSYS/CFX","text":"

CFX handles its own parallelisation, so a number of complex options need to be passed in job scripts to make it run correctly.

"},{"location":"Software_Guides/ANSYS/#example-single-node-multi-threaded-ansyscfx-jobscript","title":"Example single node multi-threaded ANSYS/CFX jobscript","text":"

Here is an example runscript for running cfx5solve multi-threaded on a given .def file.

#!/bin/bash -l\n\n# ANSYS 2021.R2: Batch script to run cfx5solve on the StaticMixer.def example \n# file, single node multi-threaded (12 threads),\n\n# Force bash as the executing shell.\n\n#$ -S /bin/bash\n\n# Request 15 munutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:15:0\n\n# Request 1 gigabyte of RAM per core.\n#$ -l mem=1G\n\n# Set the name of the job.\n#$ -N StaticMixer_thread_12\n\n# Select 12 threads.\n#$ -pe smp 12\n\n# Request ANSYS licences\n# Dec 2021: comment out this check as not currently working\n###$ -ac app=cfx\n\n# Set the working directory to somewhere in your scratch space. In this\n# case the subdirectory cfxtests-2021.R2\n#$ -wd /home/<your_UCL_id>/Scratch/cfxtests-2021.R2\n\n# Load the ANSYS module to set up your environment\n\nmodule load ansys/2021.r2\n\n# Copy the .def file into the working (current) directory\n\ncp /home/<your userid>/cfx_examples/StaticMixer.def .\n\n# Run cfx5solve - Note: -max-elapsed-time needs to be set to the same\n# time as defined by 2 above.\n\ncfx5solve -max-elapsed-time \"15 [min]\" -def StaticMixer.def -par-local -partition $OMP_NUM_THREADS\n

You will need to change the -wd /home/<your_UCL_id>/Scratch/cfxtests-2021.R2 location and may need to change the memory, wallclock time, number of threads and job name directives as well. Replace the .def file with your one and modify the -max-elapsed-time value if needed. The simplest form of qsub command can be used to submit the job eg:

qsub run-StaticMixer-thr.sh\n

Output files will be saved in the job's working directory.

"},{"location":"Software_Guides/ANSYS/#example-multi-node-mpi-ansyscfx-jobscript","title":"Example multi-node MPI ANSYS/CFX jobscript","text":"

Here is an example runscript for running cfx5solve on more than one node (using MPI) on a given .def file. 

#!/bin/bash -l\n\n# ANSYS 2021.R2: Batch script to run cfx5solve on the StaticMixer.def example \n# file, distributed parallel (80 cores).\n\n# Using ANSYS 2021 licence manager running on UCL central licence server.\n\n# 1. Force bash as the executing shell.\n#$ -S /bin/bash\n\n# 2. Request one hour of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:60:0\n\n# 3. Request 2 gigabyte of RAM per core.\n#$ -l mem=2G\n\n# 4. Set the name of the job.\n#$ -N StaticMixer_P_dist_80_NLC\n\n# 5. Select the MPI parallel environment and 80 processors.\n#$ -pe mpi 80\n\n# 6. Request ANSYS licences $ not inserted so currently inactive. Job will queue until\n# suficient licenses are available when active.\n# -ac app=cfx\n\n# 7. Set the working directory to somewhere in your scratch space.  In this\n# case the subdirectory cfxtests-18.0\n#$ -wd /home/<your_UCL_id>/Scratch/cfxtests-2021.R2\n\n# 8. Load the ANSYS module to set up your environment\n\nmodule load ansys/2021.r2\n\n# 9. Copy the .def file into the working (current) directory\n\ncp /home/<your_UCL_id>/Software/ANSYS/cfx_examples/StaticMixer.def .\n\n# 10. SGE puts the machine file in $TMPDIR/machines. Use this to generate the \n# string CFX_NODES needed by cfx5solve\n\nexport CFX_NODES=`cfxnodes $TMPDIR/machines`\n\n# 11. Run cfx5solve - Note: -max-elapsed-time needs to be set to the same\n# time as defined by 2 above.\n\n# Run with default MPI.\ncfx5solve -max-elapsed-time \"60 [min]\" -def StaticMixer.def -par-dist $CFX_NODES\n

Please copy if you wish and edit it to suit your jobs. You will need to change the -wd /home/<your_UCL_userid>/Scratch/cfxtests-19.1 location and may need to change the memory, wallclock time, number of MPI processors and job name directives as well. Replace the .def file with your one and modify the -max-elapsed-time value if needed. The simplest form of qsub command can be used to submit the job eg:

qsub\u00a0run-StaticMixer-par.sh\n

Output files will be saved in the job's working directory.

"},{"location":"Software_Guides/ANSYS/#running-cfx-with-mpi-on-myriad","title":"Running CFX with MPI on Myriad","text":"

The default supplied Intel MPI doesn't work on Myriad. Instead you need to use the supplied IBM MPI. This can be done by adding: -start-method \"IBM MPI Distributed Parallel\" to the cfx5solve command. Also the maximum number of MPI processors you can request is 36.

"},{"location":"Software_Guides/ANSYS/#troubleshooting-cfx","title":"Troubleshooting CFX","text":"

If you are getting licensing errors when trying to run a parallel job and you have an older version's ~/.ansys/v161/licensing/license.preferences.xml file, delete it. It does not work with the newer license server. (This applies to all older versions, not just v161).

"},{"location":"Software_Guides/ANSYS/#ansysfluent","title":"ANSYS/Fluent","text":"

Fluent handles its own parallelisation, so a number of complex options need to be passed in job scripts to make it run correctly.

The .in file mentioned in the scripts below is a Fluent journal file, giving it the list of commands to carry out in batch mode.

"},{"location":"Software_Guides/ANSYS/#example-serial-ansysfluent-jobscript","title":"Example serial ANSYS/Fluent jobscript","text":"

Here is an example jobscript for running Fluent in serial mode (1 core).

#!/bin/bash -l\n\n# ANSYS 2021.R2: Batch script to run ANSYS/fluent in serial mode \n# (1 core). \n\n# Request 2 hours of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=2:0:0\n\n# Request 2 gigabytes of RAM.\n#$ -l mem=2G\n\n# Set the name of the job.\n#$ -N Fluent_ser1\n\n# Request ANSYS licences\n# Dec 2021: comment out this check as not currently working\n###$ -ac app=cfx\n\n# Set the working directory to somewhere in your scratch space.  In this\n# case the subdirectory fluent-tests-19.1\n#$ -wd /home/<your_UCL_userid>/Scratch/fluent-tests-2021.R2\n\n# Load the ANSYS module to set up your environment\n\nmodule load ansys/2021.r2\n\n# Copy Fluent input files into the working (current) directory\n\ncp <path to your input files>/test-1.cas .\ncp <path to your input files>/test-1.in .\n\n# Run fluent in 2D single precision (-g no GUI). For double precision use\n# 2ddp. For 3D use 3d or 3ddp. \n\nfluent 2d -g < test-1.in\n

Please copy if you wish and edit it to suit your jobs. You will need to change the -wd /home/<your_UCL_id>/Scratch/fluent-tests-2021.R2 location and may need to change the memory, wallclock time, and job name as well. Replace the .cas and .in files with your ones. The simplest form of qsub command can be used to submit the job eg:

qsub\u00a0run-ANSYS-fluent-ser.sh\n

Output files will be saved in the job's working directory.

"},{"location":"Software_Guides/ANSYS/#example-parallel-mpi-ansysfluent-jobscript","title":"Example parallel (MPI) ANSYS/Fluent jobscript","text":"

Here is an example runscript for running Fluent in parallel potentially across more than one node.

#!/bin/bash -l\n\n# ANSYS 2021.R2: Batch script to run ANSYS/fluent distributed parallel \n# (80 cores). \n\n# Request 2 hours of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=2:0:0\n\n# Request 2 gigabytes of RAM per core.\n#$ -l mem=2G\n\n# Set the name of the job.\n#$ -N Fluent_par80\n\n# Select the MPI parallel environment and 80 processors.\n#$ -pe mpi 80\n\n# Request 25 Gb TMPDIR space (if on a cluster that supports this)\n#$ -l tmpfs=25G\n\n# Request ANSYS licences\n# Dec 2021: comment out this check as not currently working\n###$ -ac app=cfx\n\n# Set the working directory to somewhere in your scratch space.  In this\n# case the subdirectory fluent-tests-19.1\n#$ -wd /home/<your_UCL_userid>/Scratch/fluent-tests-2021.R2\n\n# Load the ANSYS module to set up your environment\n\nmodule load ansys/2021.r2\n\n# Copy Fluent input files into the working (current) directory\n\ncp <path to your input files>/test-1.cas .\ncp <path to your input files>/test-1.in .\n\n# Run fluent  in 3D single precision (-g no GUI). For double precision use\n# 3ddp. For 2D use 2d or 2ddp. \n# Do not change -t, -mpi, -pinfiniband and -cnf options.\n\nfluent 3ddp -t$NSLOTS -mpi=intelmpi -cnf=$TMPDIR/machines -g < test-1.in\n

Please copy if you wish and edit it to suit your jobs. You will need to change the -wd /home/<your_UCL_id>/Scratch/fluent-tests-2021.r2 location and may need to change the memory, wallclock time, number of MPI processors and job name as well. Replace the .cas and .in files with your ones. The simplest form of qsub command can be used to submit the job eg: 

qsub\u00a0run-ANSYS-fluent-par-80.sh\n

Output files will be saved in the job's working directory.

If you want to use IBM Platform MPI instead of Intel MPI, then replace the -mpi=intelmpi with -mpi=ibmmpi.

"},{"location":"Software_Guides/ANSYS/#troubleshooting-fluent","title":"Troubleshooting Fluent","text":"

If you are getting licensing errors when trying to run a parallel job and you have an older version's ~/.ansys/v161/licensing/license.preferences.xml file, delete it. It does not work with the newer license server. (This applies to all older versions, not just v161).

Fluent 14 required -mpi=pcmpi -pinfiniband in the parallel options: if you have older scripts remember to remove this.

"},{"location":"Software_Guides/ANSYS/#ansys-mechanical","title":"ANSYS Mechanical","text":"

ANSYS Mechanical handles its own parallelisation, and needs an additional setting to work on our clusters. It also only appears to work with Intel MPI. Here is an example jobscript for running in parallel potentially across more than one node, for example on the Kathleen cluster.

#!/bin/bash -l\n\n# ANSYS 2021.R2: Batch script to run ANSYS Mechanical solver\n# file, distributed parallel (80 cores).\n\n# Using ANSYS 2021.R2 licence manager running on UCL central licence server.\n\n# Force bash as the executing shell.\n#$ -S /bin/bash\n\n# Request one hour of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=1:00:0\n\n# Request 2 gigabyte of RAM per core. (Must be an integer)\n#$ -l mem=2G\n\n# Set the name of the job.\n#$ -N Mech_P_dist_80\n\n# Select the MPI parallel environment and 80 processors.\n#$ -pe mpi 80\n\n# Request ANSYS licences $ inserted so currently active.Job will queue until\n# suficient licences are available.\n# Dec 2021: comment out this check as not currently working\n###$ -ac app=cfx\n\n# Set the working directory to somewhere in your scratch space.  In this\n# case the subdirectory ANSYS_Mech\n#$ -wd /home/<your_UCL_username>/Scratch/ANSYS_Mech\n\n# Load the ANSYS module to set up your environment\nmodule load ansys/2021.r2\n\n# Copy the .in file into the working (current) directory\ncp ~/ANSYS/steady_state_input_file.dat .\n\n# 10. SGE puts the machine file in $TMPDIR/machines. Use this to generate the \n# string CFX_NODES needed by ansys195 which requires : as the separator.\n\nexport CFX_NODES=`cfxnodes_cs $TMPDIR/machines`\necho $CFX_NODES\n\n# Need to set KMP_AFFINTY to get round error: OMP: System error #22: Invalid argument\n\nexport KMP_AFFINITY=disabled\n\n# Run ansys mechanical - Note: use ansys195 instead of ansys and -p argument\n# needed to switch to a valid UCL license.\n\nansys212 -dis -mpi intelmpi -machines $CFX_NODES -b < steady_state_input_file.dat\n

Please copy if you wish and edit it to suit your jobs. You will need to change the -wd /home/<your_UCL_id>/Scratch/ANSYS_Mech location and may need to change the memory, wallclock time, number of MPI processors and job name as well. Replace the .dat file with your one. The simplest form of qsub command can be used to submit the job eg: 

qsub\u00a0ansys-mech-2021.R2-ex.sh\n

Output files will be saved in the job's working directory.

If you have access to Kathleen, this test input and jobscript are available at /home/ccaabaa/Software/ANSYS/steady_state_input_file.dat and /home/ccaabaa/Software/ANSYS/ansys-mech-2021.R2-ex.sh

"},{"location":"Software_Guides/ANSYS/#ansys-electromagnetic-suite-ansysem","title":"ANSYS Electromagnetic Suite (AnsysEM)","text":"

The AnsysEM products handle their own parallelisation so a number of complex options need to be passed in job scripts to make it run correctly. Also additional module commands are required.

Here is an example jobscript for running in parallel potentially across more than one node, for example on the Kathleen cluster.

#!/bin/bash -l\n\n# AnsysEM 2021 R2: Batch script to run one of the Ansys Electromagnetics Products\n# example simulations on Kathleen - distributed parallel (80 cores)\n\n# 1. Force bash as the executing shell.\n#$ -S /bin/bash\n\n# 2. Request one hour of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=1:00:0\n\n# 3. Request 2 gigabyte of RAM per core.\n#$ -l mem=2G\n\n# 4. Set the name of the job.\n#$ -N DiffSL_P_dist_80\n\n# 5. Select the MPI parallel environment and 80 processors - two nodes.\n#$ -pe mpi 80\n\n# 7. Request ANSYS licences $ inserted so currently active. Job will queue until\n# suficient licenses are available. Not currently active.\n# -ac app=cfx\n\n# 8. Set the working directory to somewhere in your scratch space.  In this\n# case the directory the job was submitted from,\n#$ -cwd\n\n# 9. Load the ANSYS module to set up your environment\n\nmodule load ansys/2021.r2\nmodule load xorg-utils/X11R7.7\nmodule load giflib/5.1.1\n\n# 10. Run ansysedt\n\nansysedt -ng -Distributed -machinelist num=$NSLOTS -batchoptions 'HFSS/MPIVendor'='Intel' -batchoptions 'TempDirectory'=\"${TMPDIR}\" -batchoptions 'HFSS/HPCLicenseType'='pool'  -batchoptions 'HFSS-IE/HPCLicenseType'='pool' -BatchSolve differential_stripline.aedt\n
"},{"location":"Software_Guides/Installing_Software/","title":"Installing Software","text":""},{"location":"Software_Guides/Installing_Software/#installing-software","title":"Installing software","text":"

If you want to request that software be installed centrally, you can email us at rc-support@ucl.ac.uk. When you send in a request please address the following questions so that the install can be properly prioitised and planned,

The requests will be added to the issues in our buildscripts repository. The buildscripts themselves are there too, so you can see how we built and installed our central software stack.

You can install software yourself in your space on the cluster. Below are some tips for installing packages for languages such as Python or Perl as well as compiling software.

"},{"location":"Software_Guides/Installing_Software/#no-sudo","title":"No sudo!","text":"

You cannot install anything using sudo (and neither can we!). If the instructions tell you to do that, read further to see if they also have instructions for installing in user space, or for doing an install from source if they are RPMs.

Alternatively, just leave off the sudo from the command they tell you to run and look for an alternative way to give it an install location if it tries to install somewhere that isn't in your space (examples for some common build systems are below).

"},{"location":"Software_Guides/Installing_Software/#download-source-code","title":"Download source code","text":"

Use wget or curl to download the source code for the software you want to install to your account on the cluster. You can use tar to extract source archives named like tar.gz or .tgz or .tar.bz2 among others and unzip for .zip files. xz --decompress will expand .xz files.

wget https://www.example.com/program.tar.gz\ntar -xvf program.tar.gz\n

You will not be able to use a package manager like yum, and will need to follow the manual installation instructions for a user-space install (not using sudo).

"},{"location":"Software_Guides/Installing_Software/#set-up-modules","title":"Set up modules","text":"

Before you start compiling, you need to make sure you have the right compilers, libraries and other tools available for your software. If you haven't changed anything, you will have the default modules loaded.

Check what the instructions for your software tell you about compiling it. If the website doesn't say much, the source code will hopefully have a README or INSTALL file.

You may want to use a different compiler - the default is the Intel compiler.

module avail compilers will show you all the compiler modules available. Most Open Source software tends to assume you're using GCC and OpenMPI (if it uses MPI) and is most tested with that combination, so if it doesn't tell you otherwise you may want to begin there (do check what the newest modules available are - the below is correct at time of writing):

# unload your current compiler and mpi modules\nmodule unload -f compilers mpi\n# load the GNU compiler\nmodule load compilers/gnu/4.9.2\n\n# these three modules are only needed on Myriad\nmodule load numactl/2.0.12\nmodule load binutils/2.29.1/gnu-4.9.2\nmodule load ucx/1.8.0/gnu-4.9.2\n\n# load OpenMPI\nmodule load mpi/openmpi/4.0.3/gnu-4.9.2\n
"},{"location":"Software_Guides/Installing_Software/#newer-versions-of-gcc-and-glibcxx","title":"Newer versions of GCC and GLIBCXX","text":"

The software you want to run may require newer compilers or a precompiled binary may say that it needs a newer GLIBCXX to be able to run. You can access these as follows:

# make all the newer versions visible\nmodule load beta-modules\n# unload current compiler, mpi and gcc-libs modules\nmodule unload -f compilers mpi gcc-libs\n# load GCC 10.2.0\nmodule load gcc-libs/10.2.0\nmodule load compilers/gnu/10.2.0\n

The gcc-libs module contains the actual compiler and libraries, while the compilers/gnu module sets environment variables that are likely to be picked up by build systems, telling them what the C, C++ and Fortran compilers are called.

"},{"location":"Software_Guides/Installing_Software/#glibc-version-error","title":"GLIBC version error","text":"

If you get an error saying that a precompiled binary that you are installing needs a newer GLIBC (not GLIBCXX) then this has been compiled on a newer operating system and will not work on our clusters. Look for a binary that was created for CentOS 7 (we have RHEL 7) or build the program from source if possible.

"},{"location":"Software_Guides/Installing_Software/#build-systems","title":"Build systems","text":"

Most software will use some kind of build system to manage how files are compiled and linked and in what order. Here are a few common ones.

"},{"location":"Software_Guides/Installing_Software/#automake-configure","title":"Automake configure","text":"

Automake will generate the Makefile for you and hopefully pick up sensible options through configuration. You can give it an install prefix to tell it where to install (or you can build it in place and not use make install at all).

./configure\u00a0--prefix=/home/username/place/you/want/to/install\nmake\n#\u00a0if\u00a0it\u00a0has\u00a0a\u00a0test\u00a0suite,\u00a0good\u00a0idea\u00a0to\u00a0use\u00a0it\nmake\u00a0test\u00a0\nmake\u00a0install\n

If it has more configuration flags, you can use ./configure --help to view them.

Usually configure will create a config.log: you can look in there to find if any tests have failed or things you think should have been picked up haven't.

"},{"location":"Software_Guides/Installing_Software/#cmake","title":"CMake","text":"

CMake is another build system. It will have a CMakeFile or the instructions will ask you to use cmake or ccmake rather than make. It also generates Makefiles for you. ccmake is a terminal-based interactive interface where you can see what variables are set to and change them, then repeatedly configure until everything is correct, generate the Makefile and quit. cmake is the commandline version. The interactive process tends to go like this:

ccmake\u00a0CMakeLists.txt\n#\u00a0press\u00a0c\u00a0to\u00a0configure\u00a0-\u00a0will\u00a0pick\u00a0up\u00a0some\u00a0options\n#\u00a0press\u00a0t\u00a0to\u00a0toggle\u00a0advanced\u00a0options\n#\u00a0keep\u00a0making\u00a0changes\u00a0and\u00a0configuring\u00a0until\u00a0no\u00a0more\u00a0errors\u00a0or\u00a0changes\n#\u00a0press\u00a0g\u00a0to\u00a0generate\u00a0and\u00a0exit\nmake\n#\u00a0if\u00a0it\u00a0has\u00a0a\u00a0test\u00a0suite,\u00a0good\u00a0idea\u00a0to\u00a0use\u00a0it\nmake\u00a0test\u00a0\nmake\u00a0install\n

The options that you set using ccmake can also be passed on the commandline to cmake with -D. This allows you to script an install and run it again later. CMAKE_INSTALL_PREFIX is how you tell it where to install.

# making a build directory allows you to clean it up more easily\nmkdir build\ncd build\ncmake .. -DCMAKE_INSTALL_PREFIX=/home/username/place/you/want/to/install\n

If you need to rerun cmake/ccmake and reconfigure, remember to delete the CMakeCache.txt file first or it will still use your old options. Turning on verbose Makefiles in cmake is also useful if your code didn't compile first time - you'll be able to see what flags the compiler or linker is actually being given when it fails.

"},{"location":"Software_Guides/Installing_Software/#make","title":"Make","text":"

Your code may come with a Makefile and have no configure, in which case the generic way to compile it is as follows:

make\u00a0targetname\n

There's usually a default target, which make on its own will use. make all is also frequently used. If you need to change any configuration options, you'll need to edit those sections of the Makefile (usually near the top, where the variables/flags are defined).

Here are some typical variables you may want to change in a Makefile.

These are what compilers/mpi wrappers to use - these are also defined by the compiler modules, so you can see what they should be. Intel would be icc, icpc, ifort, while the GNU compiler would be gcc, g++, gfortran. If this is a program that can be compiled using MPI and only has a variable for CC, then set that to mpicc.

CC=gcc\nCXX=g++\nFC=gfortran\nMPICC=mpicc\nMPICXX=mpicxx\nMPIF90=mpif90\n

CFLAGS and LDFLAGS are flags for the compiler and linker respectively, and there might be LIBS or INCLUDE in the Makefile as well. When linking a library with the name libfoo, use -lfoo.

CFLAGS=\"-I/path/to/include\"\nLDFLAGS=\"-L/path/to/foo/lib\u00a0-L/path/to/bar/lib\"\nLDLIBS=\"-lfoo\u00a0-lbar\"\n

Remember to make clean first if you are recompiling with new options. This will delete object files from previous attempts.

"},{"location":"Software_Guides/Installing_Software/#blas-and-lapack","title":"BLAS and LAPACK","text":"

BLAS and LAPACK are linear algebra libraries that are provided as part of MKL, OpenBLAS or ATLAS. There are several different OpenBLAS and ATLAS modules for different compilers. MKL is available as part of each Intel compiler module.

Your code may try to link -lblas -llapack: this isn't the right way to use BLAS and LAPACK with MKL or ATLAS (though our OpenBLAS now has symlinks that mean this will work).

"},{"location":"Software_Guides/Installing_Software/#mkl","title":"MKL","text":"

When you have an Intel compiler module loaded, typing 

echo $MKLROOT\n

will show you that MKL is available.

"},{"location":"Software_Guides/Installing_Software/#easy-linking-of-mkl","title":"Easy linking of MKL","text":"

If you can, try to use -mkl as a compiler flag - if that works, it should get all the correct libraries linked in the right order. Some build systems do not work with this however and need explicit linking.

"},{"location":"Software_Guides/Installing_Software/#intel-mkl-link-line-advisor","title":"Intel MKL link line advisor","text":"

It can be complicated to get the correct link line for MKL, so Intel has provided a tool which will give you the link line with the libraries in the right order.

Pick the version of MKL you are using (for the Intel 2018 compiler it should be Intel(R) MKL 2018.0), and these options:

You'll get something like this:

${MKLROOT}/lib/intel64/libmkl_blas95_ilp64.a ${MKLROOT}/lib/intel64/libmkl_lapack95_ilp64.a -L${MKLROOT}/lib/intel64 -lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lmkl_blacs_intelmpi_ilp64 -lpthread -lm -ldl\n

and compiler options:

-i8 -I${MKLROOT}/include/intel64/ilp64 -I${MKLROOT}/include\n

It is a good idea to double check the library locations given by the tool are correct: do an ls ${MKLROOT}/lib/intel64 and make sure the directory exists and contains the libraries. In the past there have been slight path differences between tool and install for some versions.

"},{"location":"Software_Guides/Installing_Software/#openblas","title":"OpenBLAS","text":"

We have native threads, OpenMP and serial versions of OpenBLAS. Type module avail openblas to see the available versions.

"},{"location":"Software_Guides/Installing_Software/#linking-openblas","title":"Linking OpenBLAS","text":"

Our OpenBLAS modules now contain symlinks for libblas and liblapack that both point to libopenblas. This means that the default -lblas -llapack will in fact work.

This is how you would normally link OpenBLAS:

-L${OPENBLASROOT}/lib -lopenblas\n

If code you are compiling requires separate entries for BLAS and LAPACK, set them both to -lopenblas.

"},{"location":"Software_Guides/Installing_Software/#troubleshooting-openmp-loop-warning","title":"Troubleshooting: OpenMP loop warning","text":"

If you are running a threaded program and get this warning:

OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.\n

Then tell OpenBLAS to use only one thread by adding the below to your jobscript (this overrides $OMP_NUM_THREADS for OpenBLAS only):

export OPENBLAS_NUM_THREADS=1\n

If it is your own code, you can also set it in the code with the function

void openblas_set_num_threads(int num_threads);\n

You can avoid this error by compiling with one of the native-threads or serial OpenBLAS modules instead of the openmp one.

"},{"location":"Software_Guides/Installing_Software/#atlas","title":"ATLAS","text":"

We would generally recommend using OpenBLAS instead at present, but we do have ATLAS modules.

"},{"location":"Software_Guides/Installing_Software/#dynamic-linking-atlas","title":"Dynamic linking ATLAS","text":"

There is one combined library each for serial and threaded ATLAS (in most circumstances you probably want the serial version).

Serial:

-L${ATLASROOT}/lib -lsatlas\n

Threaded:

-L${ATLASROOT}/lib -ltatlas\n
"},{"location":"Software_Guides/Installing_Software/#static-linking-atlas","title":"Static linking ATLAS","text":"

There are multiple libraries to link.

Serial:

-L${ATLASROOT}/lib -llapack -lf77blas -lcblas -latlas\n

Threaded:

-L${ATLASROOT}/lib -llapack -lptf77blas -lptcblas -latlas\n
"},{"location":"Software_Guides/Installing_Software/#troubleshooting-libgfortran-or-lifcore-cannot-be-found","title":"Troubleshooting: libgfortran or lifcore cannot be found","text":"

If you get a runtime error saying that libgfortran.so cannot be found, you need to add -lgfortran to your link line.

The Intel equivalent is -lifcore.

You can do a module show on the compiler module you are using to see where the Fortran libraries are located if you need to give a full path to them.

"},{"location":"Software_Guides/Installing_Software/#installing-additional-packages-for-an-existing-scripting-language","title":"Installing additional packages for an existing scripting language","text":""},{"location":"Software_Guides/Installing_Software/#python","title":"Python","text":"

There are python2/recommended and python3/recommended module bundles you will see if you type module avail python. These use a virtualenv, have a lot of Python packages installed already, like numpy and scipy (see the Python package list) and have pip set up for you.

"},{"location":"Software_Guides/Installing_Software/#load-the-gnu-compiler","title":"Load the GNU compiler","text":"

Our Python installs were built with GCC. You can run them without problems with the default Intel compilers loaded because it also depends on the gcc-libs/4.9.2 module. However, when you are installing your own Python packages you should make sure you have the GNU compiler module loaded. This is to avoid the situation where you build your package with the Intel compiler and then try to run it with our GNU-based Python. If it compiled any C code, it will be unable to find Intel-specific instructions and give you errors.

Change your compiler module:

module unload compilers\nmodule load compilers/gnu/4.9.2\n

If you get an error like this when trying to run something, you built a package with the Intel compiler.

undefined symbol: __intel_sse2_strrchr\n
"},{"location":"Software_Guides/Installing_Software/#install-your-own-packages-in-the-same-virtualenv","title":"Install your own packages in the same virtualenv","text":"

This will use our central virtualenv and the packages we have already installed.

# for Python 2\npip install --user <python2pkg>\n# for Python 3\npip3 install --user <python3pkg>\n

These will install into .python2local or .python3local in your home directory.

If your own installed Python packages get into a mess, you can delete (or rename) the whole .python3local and start again.

"},{"location":"Software_Guides/Installing_Software/#using-your-own-virtualenv","title":"Using your own virtualenv","text":"

If you need different packages that are not compatible with the centrally installed versions (eg. what you are trying to install depends on a different version of something we have already installed) then you can create a new virtualenv and only packages you are installing yourself will be in it.

In this case, you do not want our virtualenv with our packages to also be active. We have two types of Python modules. If you type module avail python there are \"bundles\" which are named like python3/3.7 - these include our virtualenv and packages. Then there are the base modules for just python itself, like python/3.7.4.

When using your own virtualenv, you want to load one of the base python modules.

# load a base python module (you will always need to do this)\nmodule load python/3.7.4\n# create the new virtualenv, with any name you want\nvirtualenv <DIR>\n# activate it\nsource <DIR>/bin/activate\n

Your bash prompt will change to show you that a different virtualenv is active. (This one is called venv).

(venv) [uccacxx@login03 ~]$ \n

deactivate will deactivate your virtualenv and your prompt will return to normal.

You only need to create the virtualenv the first time.

"},{"location":"Software_Guides/Installing_Software/#error-while-loading-shared-libraries","title":"Error while loading shared libraries","text":"

You will always need to load the base python module before activating your virtualenv or you will get an error like this:

python3: error while loading shared libraries: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory\n
"},{"location":"Software_Guides/Installing_Software/#installing-via-setuppy","title":"Installing via setup.py","text":"

If you need to install by downloading a package and using setup.py, you can use the --user flag and as long as one of our python module bundles are loaded, it will install into the same .python2local or .python3local as pip does and your packages will be found automatically.

python setup.py install --user\n

If you want to install to a different directory in your space to keep this package separate, you can use --prefix instead. You'll need to add that location to your $PYTHONPATH and $PATH as well so it can be found. Some install methods won't create the prefix directory you requested for you automatically, so you would need to create it yourself first.

This type of install makes it easier for you to only have this package in your paths when you want to use it, which is helpful if it conflicts with something else.

# add location to PYTHONPATH so Python can find it\nexport PYTHONPATH=/home/username/your/path/lib/python3.7/site-packages:$PYTHONPATH\n# if necessary, create lib/pythonx.x/site-packages in your desired install location\nmkdir -p /home/username/your/path/lib/python3.7/site-packages\n# do the install\npython setup.py install --prefix=/home/username/your/path\n

It will tend to tell you at install time if you need to change or create the $PYTHONPATH directory.

To use this package, you'll need to add it to your paths in your jobscript or .bashrc. Check that the PATH is where your Python executables were installed.

export PYTHONPATH=/home/username/your/path/lib/python3.7/site-packages:$PYTHONPATH\nexport PATH=/home/username/your/path/bin:$PATH\n

It is very important that you keep the :$PYTHONPATH or :$PATH at the end of these - you are putting your location at the front of the existing contents of the path. If you leave them out, then only your package location will be found and nothing else.

"},{"location":"Software_Guides/Installing_Software/#troubleshooting-remove-your-pip-cache","title":"Troubleshooting: remove your pip cache","text":"

If you built something and it went wrong, and are trying to reinstall it with pip and keep getting errors that you think you should have fixed, you may still be using a previous cached version. The cache is in .cache/pip in your home directory, and you can delete it.

You can prevent caching entirely by installing using pip3 install --user --no-cache-dir <python3pkg>

"},{"location":"Software_Guides/Installing_Software/#troubleshooting-python-script-executable-paths","title":"Troubleshooting: Python script executable paths","text":"

If you have an executable Python script (eg. something you run using pyutility and not python pyutility.py) that begins like this: 

#!/usr/bin/python2.6\n

and fails because that Python doesn't exist in that location or isn't the one that has the additional packages installed, then you should change it so it uses the first Python found in your environment instead, which will be the one from the Python module you've loaded.

#!/usr/bin/env python\n
"},{"location":"Software_Guides/Matlab/","title":"MATLAB","text":"

MATLAB is a numerical computing environment and proprietary programming language developed by MathWorks.

Our MATLAB installs include all the toolboxes included in UCL's Total Academic Headcount-Campus licence plus the MATLAB Parallel Server. We also have the NAG toolbox for Matlab available.

You can submit single node multi-threaded MATLAB jobs, single node jobs which use the Parallel Computing Toolbox and the MATLAB Parallel Server (MPS) and MATLAB GPU jobs. Currently MATLAB jobs can only be run on Myriad however we are working with MathWorks to allow the submission of multi-node MPS jobs on Kathleen.

You can also submit jobs to Myriad from MATLAB running on your own desktop or laptop.

"},{"location":"Software_Guides/Matlab/#setup","title":"Setup","text":"

You need to load MATLAB once from a login node before you can submit any jobs. This allows it to set up your ~/.matlab directory as a symbolic link to ~/Scratch/.matlab so that the compute nodes can write to it.

# on a login node\nmodule load xorg-utils/X11R7.7\nmodule load matlab/full/r2021a/9.10\n

We have other versions of MATLAB installed. You can run module avail matlab to see all the available installed versions.

"},{"location":"Software_Guides/Matlab/#single-node-multi-threaded-batch-jobs","title":"Single node multi-threaded batch jobs","text":"

This is the simplest way to start using MATLAB on the cluster.

You will need a .m file containing the MATLAB commands you want to carry out.

Here is an example jobscript which you would submit using the qsub command, after you have loaded the MATLAB module once on a login node as mentioned in Setup:

#!/bin/bash -l\n\n# Batch script to run a multi-threaded MATLAB job under SGE.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM per core. \n#$ -l mem=1G\n\n# Request 15 gigabytes of TMPDIR space (default is 10 GB)\n#$ -l tmpfs=15G\n\n# Request a number of threads (which will use that number of cores). \n# On Myriad you can set the number of threads to a maximum of 36. \n#$ -pe smp 36\n\n# Request one MATLAB licence - makes sure your job doesn't start \n# running until sufficient licenses are free.\n#$ -l matlab=1\n\n# Set the name of the job.\n#$ -N Matlab_multiThreadedJob1\n\n# Set the working directory to somewhere in your scratch space.\n# This is a necessary step as compute nodes cannot write to $HOME.\n# Replace \"<your_UCL_id>\" with your UCL user ID.\n# This directory must already exist.\n#$ -wd /home/<your_UCL_id>/Scratch/Matlab_examples\n\n# Your work should be done in $TMPDIR\ncd $TMPDIR\n\nmodule load xorg-utils/X11R7.7\nmodule load matlab/full/r2021a/9.10\n# outputs the modules you have loaded\nmodule list\n\n# Optional: copy your script and any other files into $TMPDIR.\n# This is only practical if you have a small number of files.\n# If you do not copy them in, you must always refer to them using a\n# full path so they can be found, eg. ~/Scratch/Matlab_examples/analyse.m\n\ncp ~/Scratch/Matlab_examples/myMatlabJob.m $TMPDIR\ncp ~/Scratch/Matlab_examples/initialise.m $TMPDIR\ncp ~/Scratch/Matlab_examples/analyse.m $TMPDIR\n\n# These echoes output what you are about to run\necho \"\"\necho \"Running matlab -nosplash -nodisplay < myMatlabJob.m ...\"\necho \"\"\n\nmatlab -nosplash -nodesktop -nodisplay < myMatlabJob.m\n# Or if you did not copy your files:\n# matlab -nosplash -nodesktop -nodisplay < ~/Scratch/Matlab_examples/myMatlabJob.m\n\n# tar up all contents of $TMPDIR back into your space\ntar zcvf $HOME/Scratch/Matlab_examples/files_from_job_${JOB_ID}.tgz $TMPDIR\n\n# Make sure you have given enough time for the copy to complete!\n

Alternative syntax: Instead of using Unix input redirection like this:

matlab -nosplash -nodesktop -nodisplay < $Matlab_infile\n

you can also do:

matlab -nosplash -nodesktop -nodisplay -r $Matlab_infile\n

NOTE: You should use the -r syntax if your input file contains function definitions as using < treats the script as keyboard input and MATLAB does not allow you to define functions directly on the command line.

"},{"location":"Software_Guides/Matlab/#run-without-the-jvm-to-reduce-overhead","title":"Run without the JVM to reduce overhead","text":"

You can give the -nojvm option to tell MATLAB to run without the Java Virtual Machine. This will speed up startup time, possibly execution time, and remove some memory overhead, but will prevent you using any tools that require Java (eg, tools that use the Java API for I/O and networking like URLREAD, or call Java object methods).

"},{"location":"Software_Guides/Matlab/#run-single-threaded","title":"Run single-threaded","text":"

Most of the time, MATLAB will create many threads and use them as it wishes. If you know your job is entirely single-threaded, you can force MATLAB to run with only one thread on one core, which will allow you to have more jobs running at once.

To request one core only, set #$ -pe smp 1 in your jobscript.

Run MATLAB like this:

matlab -nosplash -nodesktop -nodisplay -nojvm -singleCompThread < $Matlab_infile\n

The -singleCompThread forces MATLAB to run single-threaded, and the -nojvm tells it to run without the Java Virtual Machine, as above.

"},{"location":"Software_Guides/Matlab/#using-the-matlab-gui-interactively","title":"Using the MATLAB GUI interactively","text":"

You can run MATLAB interactively for short amounts of time on the login nodes (please do not do this if your work will be resource-intensive). You can also run it interactively in a qrsh session on the compute nodes.

Launching with matlab will give you the full graphical user interface - you will need to have logged in to the cluster with X-forwarding on for this to work.

Launching with matlab -nodesktop -nodisplay will give you the MATLAB terminal.

[//]: # Up to this point on revision.

"},{"location":"Software_Guides/Matlab/#submitting-jobs-using-the-matlab-parallel-server-mps","title":"Submitting jobs using the MATLAB Parallel Server (MPS)","text":"

You must have loaded the MATLAB module once from a login node as described in Setup before you can submit any MATLAB MPS jobs.

MATLAB PCS jobs must be submitted from an interactive or scripted Matlab session which can be running on the cluster login nodes, from another MATLAB job or on your own machine.

MATLAB MPS jobs will currently only work inside a single node on our clusters. On Myriad this means a maximum of 36 workers can be used per job.

"},{"location":"Software_Guides/Matlab/#importing-the-cluster-profile","title":"Importing the cluster profile","text":"

You need to import the cluster profile into your MATLAB environment and set it as the default before you can submit DCS jobs. This only needs doing once. The imported profile will be saved in your MATLAB settings directory.

Importing the profile can be done either by calling MATLAB functions or via the graphical interface. The profile is stored here (for R2021a): 

/shared/ucl/apps/Matlab/toolbox_local/R2021a/myriad_R2021a.mlsettings\n
"},{"location":"Software_Guides/Matlab/#import-using-matlab-functions","title":"Import using MATLAB functions","text":"

Run these functions from a MATLAB session:

profile_Myriad = parallel.importProfile ('/shared/ucl/apps/Matlab/toolbox_local/R2021a/myriad_R2021a.mlsettings');\nparallel.defaultClusterProfile ('myriad_R2021a');\n
"},{"location":"Software_Guides/Matlab/#import-from-matlab-gui","title":"Import from MATLAB GUI","text":"

To import using the graphical interface instead, do this.

  1. From the Home tab select the Parallel menu and click Create and Manage Clusters....
  2. The Cluster Profile Manager window will open.
  3. Select Import and in the Import Profiles from file window navigate to the myriad_R2021a.mlsettings file shown above and select Open.
  4. Select the resulting myriad_R2021a profile and click Set as Default. The Cluster Profile Manager window should now look like this:

In both cases after you exit MATLAB your cluster profile is saved for future use.

"},{"location":"Software_Guides/Matlab/#environment-variables-needed-for-job-submission","title":"Environment variables needed for job submission","text":"

We have set up four Grid Engine environment variables to assist with job submission from within MATLAB. These are needed to pass in job resource parameters that aren't supported by the internal MATLAB job submission mechanism.

and two project and Gold related variables. Most users will not need to use either of these:

-ac exclusive prevents anything else running on the same node as your job, even if you aren't using all the cores. This is no longer a necessary option for MATLAB jobs.

There are two ways to set these:

1) Before starting your MATLAB session, using the usual Bash method of exporting environment variables:

export SGE_CONTEXT=exclusive\nexport SGE_OPT=h_rt=0:15:0,mem=2G,tmpfs=15G\nexport SGE_PROJECT=<your project ID>\nexport SGE_ACCOUNT=<your Gold project>\n

2) Inside your MATLAB session, using MATLAB's setenv function:

setenv ('SGE_CONTEXT', 'exclusive');\nsetenv ('SGE_OPT', 'h_rt=0:15:0,mem=2G,tmpfs=15G'); \nsetenv ('SGE_PROJECT', '<your project ID>');\nsetenv ('SGE_ACCOUNT', '<your Gold project>');\n
"},{"location":"Software_Guides/Matlab/#example-a-simple-mps-job","title":"Example: a simple MPS job","text":"

This submits a job from inside a MATLAB session running on a login node. You need to start MATLAB from a directory in Scratch - jobs will inherit this as their working directory.

This is an example where you have only one MATLAB source file.

1) Change to the directory in Scratch you want the job to run from and set the SGE environment variables.

cd ~/Scratch/Matlab_examples\nexport SGE_OPT=h_rt=0:10:0,mem=2G,tmpfs=15G\n

2) Either start the MATLAB GUI:

matlab\n

or start a MATLAB terminal session:

matlab -nodesktop -nodisplay\n

3) Inside MATLAB, create a cluster object using the cluster profile:

c = parcluster ('myriad_R2021a');\n

4) Use your cluster object to create a job object of the type you need. For this example the job is a parallel job with communication between MATLAB workers of type \"Single Program Multiple Data\":

myJob = createCommunicatingJob (c, 'Type', 'SPMD');\n

5) Set the number of workers:

num_workers = 8;\n

The maximum value you can set here on Myriad is 36.

6) Tell the job the files needed to be made available to each worker - in this example there is only one file:

myJob.AttachedFiles = {'colsum.m'};\n

colsum.m contains the simple magic square example from the MATLAB manual \"Parallel Computing Toolbox User's Guide\".

7) Set the minimum and maximum number of workers for the job (we are asking for an exact number here by setting them the same):

myJob.NumWorkersRange = [num_workers, num_workers];\n

8) Create a MATLAB task to be executed as part of the job. Here it consists of executing the MATLAB function colsum. The other arguments say that the task returns one parameter and there are no input arguments to the colsum function:

task = createTask (myJob, @colsum, 1, {});\n

9) Submit the job:

submit (myJob);\n

Your job is now submitted to the scheduler and you can see its queue status in qstat as normal. If you were using the MATLAB GUI you can also monitor jobs by selecting Monitor Jobs from the Parallel menu on the Home tab.

10) When the job has completed get the results using:

results = fetchOutputs(myJob)\n

You can access the job log from MATLAB using:

logMess = getDebugLog (c, myJob);\n
"},{"location":"Software_Guides/Matlab/#example-a-mps-job-with-more-than-one-input-file","title":"Example: a MPS job with more than one input file","text":"

This example has several input files. The job type is \"MATLAB Pool\". A \"Pool\" job runs the specified task function with a MATLAB pool available to run the body of parfor loops or spmd blocks and is the default job type. This example was kindly supplied to assist in testing MATLAB by colleagues from CoMPLEX.

The first part of creating the job is the same as the above example apart from the longer runtime and larger amount of memory per core:

1) Change into a directory in Scratch, set the SGE variables and launch MATLAB:

cd ~/Scratch/Matlab_examples\nexport SGE_OPT=h_rt=1:0:0,mem=4G,tmpfs=15G\nmatlab\n

to launch the GUI or:

matlab -nodesktop -nodisplay\n

to start a terminal session. 

c = parcluster ('myriad_R2021a');\n

2) Using our cluster object create a job object of type \"Pool\":

myJob2 = createCommunicatingJob (c, 'Type', 'Pool');\n

3) Set the number of workers and another variable used by the example:

num_workers = 8;\nsimulation_duration_ms = 1000;\n

4) Tell the job all the input files needed to be made available to each worker as a cell array:

myJob2.AttachedFiles = {\n'AssemblyFiniteDifferencesMatrix.m'\n'AssemblyFiniteDifferencesRightHandSide.m'\n'CellModelsComputeIonicCurrents.m'\n'CellModelsGetVoltage.m'\n'CellModelsInitialise.m'\n'CellModelsSetVoltage.m'\n'GetStimuliForTimeStep.m'\n'SubmitMonodomainJob.m'\n'RegressionTest.m'\n'RunAndVisualiseMonodomainSimulation.m'\n'SolveLinearSystem.m'\n'luo_rudy_1991_iionic.m'\n'luo_rudy_1991_time_deriv.m'};\n

5) Set the minimum and maximum number of workers for the job:

myJob2.NumWorkersRange = [num_workers, num_workers];\n

6) Create a MATLAB task to be executed as part of the job. For this example it will consist of executing the MATLAB function RunAndVisualiseMonodomainSimulation. The rest of the arguments indicate that the task returns three parameters and there are five input arguments to the function. These are passed as a cell array:

task = createTask (myJob2, @RunAndVisualiseMonodomainSimulation, 3, {5000, simulation_duration_ms, 1.4, 1.4, false});\n

7) Submit the job:

submit (myJob2);\n

As before use fetchOutputs to collect the results.

If you closed your session, you can get your results by:

c = parcluster ('myriad_R2021a');                  # get a cluster object\njobs = findJob(c)                                  # get a list of jobs submitted to that cluster\njob = jobs(3);                                     # pick a particular job\nresults = fetchOutputs(job)\n

You can get other information: diary(job) will give you the job diary, and load(job) will load the workspace.

"},{"location":"Software_Guides/Matlab/#further-reading","title":"Further reading","text":"

There is a lot more information about using the MATLAB Distributed Computing Server in the MATLAB manual: Parallel Computing Toolbox User\u2019s Guide.

"},{"location":"Software_Guides/Matlab/#submitting-matlab-jobs-from-your-workstationlaptop","title":"Submitting MATLAB jobs from your workstation/laptop","text":"

You can submit MATLAB jobs to Myriad from MATLAB sessions running on your own desktop workstation or laptop systems provided they are running the same version of MATLAB and your computer is within the UCL firewall.

With MATLAB R2021a you can currently submit jobs to Myriad. Support for R2018b is also still available.

"},{"location":"Software_Guides/Matlab/#prerequisites","title":"Prerequisites","text":"
  1. You must already have an account on the clusters!
  2. Have MATLAB R2021a (or R2018b) installed on your local workstation/laptop. The local version must match the version running jobs. MATLAB R2021a can be downloaded from the UCL Software Database.
  3. Your local workstation/laptop installation of MATLAB must include the Parallel Computing toolbox. This is included in the UCL TAH MATLAB license and may be installed automatically. Home tab > Add-Ons > Get Add-Ons You will find the Parallel Computing Toolbox in the Workflows category.
  4. If your local workstation/laptop is not directly connected to the UCL network (at home for example), you need to have the UCL VPN client installed and running on it.
"},{"location":"Software_Guides/Matlab/#remote-setup","title":"Remote setup","text":"

1) On the cluster you are using (Myriad in this case) create a directory to hold remotely submitted job details. For example:

mkdir ~/Scratch/Matlab_remote_jobs\n

This directory needs to be in your Scratch directory as compute nodes need to be able to write to it. You should not use this directory for anything else.

2) On your local workstation/laptop create a directory to hold information about jobs that have been submitted to the cluster. Again you should not use this directory for anything else.

3) Download either the the support files for remote submission to Myriad for R2021a or support files for remote submission to Myriad for R2018b. Make sure you download the correct one for your version of MATLAB!

4) This step MUST be done while Matlab is shut down. Unzip the archive into MATLAB's local toolbox directory. Default locations for the local toolbox directory are:

5) Download the parallelProfileMyriad function to your local workstation/laptop. It will need to be unzipped. This function create a cluster profile for Myriad for R2021a or R2018b.

6) Start MATLAB, navigate to where you saved the parallelProfileMyriad.m file and run the function by typing:

parallelProfileMyriad\n

at your MATLAB prompt (in your MATLAB Command Window if running the MATLAB GUI) and answer the questions.

"},{"location":"Software_Guides/Matlab/#submitting-a-job-to-the-cluster","title":"Submitting a job to the cluster","text":"

1) You need to set the Grid Engine support environment variables on your local computer. Eg. in your MATLAB session set:

setenv ('SGE_CONTEXT', 'exclusive');                # optional\nsetenv ('SGE_OPT', 'h_rt=0:15:0,mem=2G,tmpfs=15G'); \nsetenv ('SGE_PROJECT', '<your project ID>');        # optional\nsetenv ('SGE_ACCOUNT', '<your Gold project>');      # optional\n

2) In your MATLAB session create a cluster object using the cluster profile created by the parallelProfile... functions. For Myriad:

c = parcluster ('myriad_R2021a');\n

3) You can now create and submit jobs in a similar way to that shown in the MPS examples above starting from step 4 in the simple MPS job example or step 2 in the MPS job with multiple input files example.

"},{"location":"Software_Guides/Matlab/#viewing-your-results","title":"Viewing your results","text":"

After submitting your job remotely from your desktop, you can close MATLAB and come back later. To see your jobs:

Click \"Parallel > Monitor jobs\"

This will bring up the job monitor where you can see the status of your jobs and whether they are finished. MATLAB numbers the jobs sequentially.

Right-click on a job and choose \"fetch outputs\".

This is what will be executed (for job4 on Myriad):

myCluster = parcluster('myriad_R2021a');\njob4 = myCluster.findJob('ID',4);\njob4_output = fetchOutputs(job4);\n

The Workspace will show the available data and you can view your results. The data is fetched from the Matlab_remote_jobs directory you created on Myriad (or Legion) in Remote setup step 1, so that will also have files and directories in it called job1, job2 and so on.

If you have already fetched the data, you can view the results straight away by selecting that job. If you need to reload everything, you can right-click on the job and the option will be to load variables instead.

"},{"location":"Software_Guides/Matlab/#writing-intermediate-results","title":"Writing intermediate results","text":"

If you want to explicitly write out intermediate results, you need to provide a full path to somewhere in Scratch otherwise MATLAB will try to write them in your home, which isn't writable by the compute nodes.

"},{"location":"Software_Guides/Matlab/#troubleshooting-remote-jobs","title":"Troubleshooting remote jobs","text":"

If you get a message like this when retrieving your outputs then something has gone wrong in your job:

Task with ID xxx returned 0 outputs but 1 were expected\n

You need to retrieve the debug log to find out what happened. Example:

myCluster = parcluster('myriad_R2021a');\njob4 = myCluster.findJob('ID',4);\n\njobLog = getDebugLog (myCluster, job4);\njobLog\n

There will be a lot of output. Look for lines related to errors happening in your own code.

"},{"location":"Software_Guides/Matlab/#running-matlab-on-gpus","title":"Running MATLAB on GPUs","text":"

This uses MATLAB's Mandelbrot Set GPU example.

#!/bin/bash -l\n\n# Batch script to run a GPU MATLAB job on Myriad.\n\n# Request 15 minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:15:0\n\n# Request 2 gigabytes of RAM per core.\n#$ -l mem=2G\n\n# Request 15 gigabytes of TMPDIR space (default is 10 GB)\n#$ -l tmpfs=15G\n\n# Request 1 GPU\n#$ -l gpu=1\n\n# Request 1 CPU core. (Max on Myriad is 36)\n#$ -pe smp 1\n\n# Request one MATLAB licence - makes sure your job doesn't start \n# running until sufficient licenses are free.\n#$ -l matlab=1\n\n# Set the name of the job.\n#$ -N Matlab_GPU_Job1\n\n# Set the working directory to somewhere in your scratch space.\n# This is a necessary step as compute nodes cannot write to $HOME.\n# Replace \"<your_UCL_id>\" with your UCL user ID.\n# This directory must already exist.\n#$ -wd /home/<your_UCL_id>/Scratch/Matlab_examples\n\n# Your work should be done in $TMPDIR\ncd $TMPDIR\n\n# Optional: Copy your script and any other files into $TMPDIR.\n# If not, you must always refer to them using a full path.\ncp /home/ccaabaa/Software/Matlab/Mandelbrot_GPU.m $TMPDIR\n\nmodule unload compilers mpi\nmodule load compilers/gnu/4.9.2\nmodule load xorg-utils/X11R7.7\nmodule load matlab/full/r2021a/9.10\nmodule list\n\n# These echoes output what you are about to run\necho \"\"\necho \"Running matlab -nosplash -nodisplay < Mandelbrot_GPU.m ...\"\necho \"\"\nmatlab -nosplash -nodesktop -nodisplay < Mandelbrot_GPU.m\n\n# tar up all contents of $TMPDIR back into your space\ntar zcvf $HOME/Scratch/Matlab_examples/files_from_job_${JOB_ID}.tgz $TMPDIR\n\n# Make sure you have given enough time for the copy to complete!\n
"},{"location":"Software_Guides/Other_Software/","title":"Other Software","text":"

We maintain a large software stack that is available across all our clusters (licenses permitting).

We use environment modules to let you manage which specific versions of software packages you are using.

"},{"location":"Software_Guides/Other_Software/#general-use-of-environment-modules","title":"General use of environment modules","text":"

We have a default set of modules that everyone has loaded when they log in: these include the current default compiler and MPI, some utili ties to make your life easier and some text editors.

"},{"location":"Software_Guides/Other_Software/#summary-of-module-commands","title":"Summary of module commands","text":"
module avail            # shows available modules\nmodule whatis           # shows available modules with brief explanations\nmodule list             # shows your currently loaded modules\n\nmodule load <module>    # load this module\nmodule unload <module>  # unload this module\nmodule purge            # unload all modules\n\nmodule show <module>    # Shows what the module requires and what it sets up\nmodule help <module>    # Shows a longer text description for the software\n
"},{"location":"Software_Guides/Other_Software/#find-out-if-a-software-package-is-installed-and-load-it","title":"Find out if a software package is installed and load it","text":"

Generically, the way you find out if a piece of software is installed is to run

module load beta-modules\nmodule avail packagename\n

By loading beta-modules you will also be able to see newer versions of GCC and the software that has been built using them.

Then module avail gives you a list of all the modules we have that match the name you searched for. You can then type 

module show packagename\n

and it will show you the other software dependencies this module has: these have to be loaded first. It also shows where the software is installed and what environment variables it sets up.

Once you have found the modules you want to load, it is good practice to refer to them using their full name, including the version. If you use the short form (package rather than package/5.1.2/gnu-4.9.2) then a matching module will be loaded, but if we install a different version, your jobs may begin using the new one and you would not know which version created your results. Different software versions may not be compatible or may have different default settings, so this is undesirable.

You may need to unload current modules in order to load some requirements (eg different compiler, different MPI).

This example switches from Intel compiler and MPI modules to GNU ones.

module unload -f compilers mpi\nmodule load compilers/gnu/4.9.2\nmodule load mpi/openmpi/3.1.4/gnu-4.9.2\n

You can use the short name when unloading things because there is usually only one match in your current modules.

The last part of a module name usually tells you what compiler it was built with and which version of that compiler. There may be GNU compiler versions and Intel compiler versions of the same software available.

Once the module is loaded, you should have all the usual executables in your path, and can use its commands. You load modules in exactly the same way inside a jobscript.

"},{"location":"Software_Guides/Other_Software/#notes-on-how-to-run-specific-packages","title":"Notes on how to run specific packages","text":"

The packages below have slightly complex commands needed to run them, or different settings needed on our clusters. These are examples of what should be added to your jobscripts. Change the module load command to the version you want to load and check that the dependencies are the same.

The top of a jobscript should contain your resource requests. See also examples of full jobscripts .

"},{"location":"Software_Guides/Other_Software/#abaqus","title":"ABAQUS","text":"

ABAQUS is a commercial software suite for finite element analysis and computer-aided engineering.

You must be authorised by the Mech Eng Department before you can be added to the group controlling access to ABAQUS (legabq).

A serial interactive analysis can be run on the compute nodes (via a qrsh session) like this:

abaqus interactive job=myJobSerial input=myInputFile.inp\n

A parallel job can be run like this (fill in your own username):

module load abaqus/2017\n\nINPUT_FILE=/home/<username>/ABAQUS/heattransfermanifold_cavity_parallel.inp\nABAQUS_ARGS=\nABAQUS_PARALLELSCRATCH=/home/<username>/Scratch/Abaqus/parallelscratch\n# creates a parallel scratch dir and a new working dir for this job\nmkdir -p $ABAQUS_PARALLELSCRATCH\nmkdir -p $JOB_NAME.$JOB_ID\ncd $JOB_NAME.$JOB_ID\ncp $INPUT_FILE .\n\nINPUT=$(basename $INPUT_FILE)\nabaqus interactive cpus=$NSLOTS mp_mode=mpi job=$INPUT.$JOB_ID input=$INPUT \\\n       scratch=$ABAQUS_PARALLELSCRATCH $ABAQUS_ARGS\n
"},{"location":"Software_Guides/Other_Software/#beast","title":"BEAST","text":"

BEAST is an application for Bayesian MCMC analysis of molecular sequences orientated towards rooted, time-measured phylogenies inferred using strict or relaxed molecular clock models.

Note that FigTree and Tracer are available as standalone modules. The addons DISSECT, MODEL_SELECTION, and SNAPP are installed for BEAST. 

cd $TMPDIR\n\nmodule load java/1.8.0_45\nmodule load beast/2.3.0\n\nbeast -threads $OMP_NUM_THREADS ~/Scratch/BEAST/gopher.xml\n\n# tar up all contents of $TMPDIR back into your space\ntar zcvf $HOME/Scratch/BEAST/files_from_job_$JOB_ID.tar.gz $TMPDIR\n
"},{"location":"Software_Guides/Other_Software/#bowtie","title":"Bowtie","text":"

Bowtie 1 and 2 are tools for aligning sequencing reads to their reference sequences.

Bowtie 1 and 2 are available. For reads longer than about 50 bp Bowtie 2 is generally faster, more sensitive, and uses less memory than Bowtie 1. For relatively short reads (e.g. less than 50 bp) Bowtie 1 is sometimes faster and/or more sensitive. For further differences, see How is Bowtie 2 different from Bowtie 1?.

Bowtie sets $BT1_HOME and Bowtie2 sets $BT2_HOME. You can have both modules loaded at once. 

cd $TMPDIR\nmodule load bowtie2/2.2.5\n\n# Run Bowtie2 example from getting started guide:\n# http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml#getting-started-with-bowtie-2-lambda-phage-example\nbowtie2-build $BT2_HOME/example/reference/lambda_virus.fa lambda_virus\nbowtie2 -x lambda_virus -U $BT2_HOME/example/reads/reads_1.fq -S eg1.sam\n\n# tar up all contents of $TMPDIR back into your space\ntar zcvf $HOME/Scratch/Bowtie2_output/files_from_job_$JOB_ID.tgz $TMPDIR\n
"},{"location":"Software_Guides/Other_Software/#castep","title":"CASTEP","text":"

CASTEP is a full-featured materials modelling code based on a first-principles quantum mechanical description of electrons and nuclei.

module load castep/17.21/intel-2017\n# Gerun is our mpirun wrapper which sets the machinefile and number of \n# processes to the amount you requested with -pe mpi.\ngerun castep.mpi input\n

CASTEP 19 has different pre-reqs:

module unload -f compilers mpi\nmodule load compilers/intel/2019/update4\nmodule load mpi/intel/2019/update4/intel\nmodule load castep/19.1.1/intel-2019\n# Gerun is our mpirun wrapper which sets the machinefile and number of \n# processes to the amount you requested with -pe mpi.\ngerun castep.mpi input\n

If you have access to the source code and wish to build your own copy, it has been suggested that compiling with these options (on Grace) gave a build that ran about 10% faster than the default compilation options:

make COMMS_ARCH=mpi SUBARCH=mpi FFT=mkl MATHLIBS=mkl10 BUILD=fast\n

Do check for numerical accuracy with any optimisations you carry out.

"},{"location":"Software_Guides/Other_Software/#cctools","title":"Cctools","text":"

Provides the Parrot connector to CVMFS, the CernVM File System.

By default, the cctools module sets the following:

export PARROT_CVMFS_REPO=<default-repositories> \nexport PARROT_ALLOW_SWITCHING_CVMFS_REPOSITORIES=yes \nexport HTTP_PROXY=DIRECT;\nexport PARROT_HTTP_PROXY=DIRECT;\n

Example usage - will list the contents of the repository then exit:

module load cctools/7.0.11/gnu-4.9.2\nparrot_run bash\nls /cvmfs/alice.cern.ch\nexit\n

That will create the cache in /tmp/parrot.xxxxx on the login nodes when run interactively. To use in a job, you will want to put the cache somewhere in your space that the compute nodes can access. You can set the cache to be in your Scratch, or to $TMPDIR on the nodes if it just needs to exist for the duration of that job.

export PARROT_CVMFS_ALIEN_CACHE=</path/to/cache>\n
"},{"location":"Software_Guides/Other_Software/#cfd-ace","title":"CFD-ACE","text":"

CFD-ACE+ is a commercial computational fluid dynamics solver developed by ESI Group. It solves the conservation equations of mass, momentum, energy, chemical species and other scalar transport equations using the finite volume method. These equations enable coupled simulations of fluid, thermal, chemical, biological, electrical and mechanical phenomena.

The license is owned by the Department of Mechanical Engineering who must give permission for users to be added to the group lgcfdace.

module load cfd-ace/2018.0\n\nCFD-SOLVER -model 3Dstepchannel_060414.DTF -num $NSLOTS -wd `pwd` \\ \n   -hosts $TMPDIR/machines -rsh=ssh -decomp -metis -sim 1 -platformmpi -job\n
"},{"location":"Software_Guides/Other_Software/#comsol","title":"COMSOL","text":"

COMSOL Multiphysics is a cross-platform finite element analysis, solver and multiphysics simulation software.

Electrical Engineering have a group license for version 52 and must give permission for users to be added to the group legcomsl. Chemical Engineering have a Departmental License for version 53 and members of that department may be added to the group lgcomsol. 

# Run a parallel COMSOL job\n\n# Versions 52 and 52a have this additional module prerequisite\nmodule load xulrunner/3.6.28/gnu-4.9.2\n\n# pick the version to load\nmodule load comsol/53a\n\n# Parallel multinode options:\n# $NHOSTS gets the number of nodes the job is running on and\n# $TMPDIR/machines is the machinefile that tells it which nodes.\n# These are automatically set up in a \"-pe mpi\" job environment.\ncomsol -nn $NHOSTS -clustersimple batch -f $TMPDIR/machines -inputfile micromixer_batch.mph \\ \n       -outputfile micromixer_batch_output_${JOB_ID}.mph\n\n# On Myriad you need to specify the fabric:\ncomsol batch -f $TMPDIR/machines -np $NSLOTS -mpifabrics shm:tcp \\ \n    -inputfile micromixer_batch.mph -outputfile micromixer_batch_output_${JOB_ID}.mph\n
"},{"location":"Software_Guides/Other_Software/#cp2k","title":"CP2K","text":"

CP2K performs atomistic and molecular simulations.

To see all available versions type

module load beta-modules\nmodule avail cp2k\n

To load CP2K 8.2:

module unload -f compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load compilers/gnu/10.2.0\n\n# These three modules only needed on Myriad\nmodule load numactl/2.0.12\nmodule load binutils/2.36.1/gnu-10.2.0\nmodule load ucx/1.9.0/gnu-10.2.0\n\nmodule load mpi/openmpi/4.0.5/gnu-10.2.0\nmodule load openblas/0.3.13-openmp/gnu-10.2.0\nmodule load cp2k/8.2/ompi/gnu-10.2.0\n\n# Gerun is our mpirun wrapper which sets the machinefile and number of \n# processes to the amount you requested with -pe mpi.\ngerun cp2k.popt < input.in > output.out\n

For CP2K 4.1 there is also a Chemistry department version with submission script generator. To access it:

module load chemistry-modules\nmodule load submission-scripts\n

The command submitters will then list the submitters available. You can then run cp2k.submit which will ask you questions in order to create a suitable jobscript.

The cp2k.submit submitter takes up to 6 arguments, and any omitted will be asked for interactively:

cp2k.submit \u00abinput_file\u00bb \u00abcores\u00bb \u00abversion\u00bb \u00abmaximum_run_time\u00bb \u00abmemory_per_core\u00bb \u00abjob_name\u00bb\n

So, for example:

cp2k.submit water.inp 8 4.1 2:00:00 4G mywatermolecule\n

would request a job running CP2K 4.1 with the input file water.inp, on 8 cores, with a maximum runtime of 2 hours, with 4 gigabytes of memory per core, and a job name of mywatermolecule.

"},{"location":"Software_Guides/Other_Software/#crystal","title":"CRYSTAL","text":"

CRYSTAL is a general-purpose program for the study of crystalline solids. The CRYSTAL program computes the electronic structure of periodic systems within Hartree Fock, density functional or various hybrid approximations.

CRYSTAL is commercial software which is available free of charge to UK academics. You must obtain a license from Crystal Solutions: How to get CRYSTAL - Academic UK license. You need to create an account and then request to be upgraded to Academic UK. Access to CRYSTAL is enabled by being a member of the reserved application group legcryst. For proof of access we accept emails from CRYSTAL saying your account has been upgraded to \"Academic UK\", or a screenshot of your account page showing you have the full download available rather than just the demo version. 

module unload -f mpi\nmodule load mpi/openmpi/2.1.2/intel-2017\nmodule load crystal17/v1.0.1\n\n# 9. Create a directory for this job and copy the input file into it. \nmkdir test00\ncd test00\ncp ~/Scratch/Crystal17/test_cases/inputs/test00.d12 INPUT\n\n# Gerun is our mpirun wrapper which sets the machinefile and number of\n# processes to the amount you requested with -pe mpi.\n# The CRYSTAL module sets $CRYxx_EXEDIR and $VERSION environment variables.\ngerun $CRY17_EXEDIR/$VERSION/Pcrystal\n
# Similarly, for Pproperties the command would be\ngerun $CRY17_EXEDIR/$VERSION/Pproperties\n

For CRYSTAL 17 v1.0.2, the modules and path are slightly different and you would do this instead:

module unload -f compilers mpi\nmodule load compilers/intel/2017/update4\nmodule load mpi/intel/2017/update3/intel\nmodule load crystal17/v1.0.2/intel-2017\n\n# Create a directory for this job and copy the input file into it.\nmkdir test00\ncd test00\ncp ~/Scratch/Crystal17/test_cases/inputs/test00.d12 INPUT\n\n# Gerun is our mpirun wrapper which sets the machinefile and number of\n# processes to the amount you requested with -pe mpi.\n# The CRYSTAL module sets $CRYxx_EXEDIR and $VERSION environment variables.\ngerun $CRY17_EXEDIR/Pcrystal\n
"},{"location":"Software_Guides/Other_Software/#freesurfer","title":"FreeSurfer","text":"

FreeSurfer is a set of tools for analysis and visualization of structural and functional brain imaging data.

Freesurfer can use threads to run on multiple cores in one node: request the number with -pe smp in the resource-request part of your jobscript.

#$ -pe smp 4\n\nmodule load xorg-utils/X11R7.7\nmodule load freesurfer/6.0.0\nexport SUBJECTS_DIR=~/Scratch/FreeSurfer_examples/subjects\n\n# -openmp $NSLOTS runs with the number of threads you requested\nrecon-all -openmp $NSLOTS -i sample-001.nii.gz -s bert -all\n
"},{"location":"Software_Guides/Other_Software/#gamess","title":"GAMESS","text":"

The General Atomic and Molecular Electronic Structure System (GAMESS) is a general ab initio quantum chemistry package.

The GAMESS module should be loaded once from a login node before submitting a job - this creates the ~/Scratch/gamess directory for you which is used as USERSCR to write some scratch files during the job. If you don't want to keep these files and would prefer them to be written to $TMPDIR instead, you can put export $GAMESS_USERSCR=$TMPDIR in your jobscript after the module load command.

module unload compilers mpi\nmodule load compilers/intel/2015/update2\nmodule load mpi/intel/2015/update3/intel\nmodule load gamess/5Dec2014_R1/intel-2015-update2\n\n# Optional: set where the USERSCR files go. \n# By default, the module sets it to ~/Scratch/gamess\nexport $GAMESS_USERSCR=$TMPDIR\n\nrungms exam01.inp 00 $NSLOTS $(ppn)\n
"},{"location":"Software_Guides/Other_Software/#gatk","title":"GATK","text":"

The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyze high-throughput sequencing data.

Version 4 of GATK is BSD-licensed so does not require a group to control access to the software.

Version 3 of GATK requires you to agree to the GATK license before we can add you to the leggatk group which gives you access: you can do this by downloading GATK 3 from The Broad Institute GATK download page, reading the license, and telling us you agree to it. You may need to create a gatkforums account before you can download.

GATK 3 uses Java 1.7 (the system Java) so you do not need to load a Java module. GATK 4 uses 1.8 so you need to load java/1.8.0_92 first.

GATK 4.2.5.0 or newer uses the newest version of Java 8, so you need to load java/temurin-8.

Load the version you want, then to run GATK you should either prefix the .jar you want to run with $GATKPATH:

java -Xmx2g -jar $GATKPATH/GenomeAnalysisTK.jar OPTION1=value1 OPTION2=value2...\n

Or we provide wrappers, so you can run it one of these ways instead:

GenomeAnalysisTK OPTION1=value1 OPTION2=value2...\n
gatk OPTION1=value1 OPTION2=value2...\n

If you want to use some of the newer tools in GATK 4 which rely on Python/Conda, you must use GATK >= 4.2.5.0 and additionally set up your miniconda environment. With 4.2.5.0 this means:

module load java/temurin-8\nmodule load gatk/4.2.5.0\nmodule load python/miniconda3/4.10.3\nsource $UCL_CONDA_PATH/etc/profile.d/conda.sh \nconda activate $GATK_CONDA\n

(For newer versions of GATK it will tell you which version of miniconda to load)

"},{"location":"Software_Guides/Other_Software/#gaussian","title":"Gaussian","text":"

Access to Gaussian 09 or Gaussian 16 are controlled by membership of separate groups. UCL has a site license so UCL users can be added on request.

Gaussian is too resource-intensive to ever be run on the login nodes.

"},{"location":"Software_Guides/Other_Software/#multithreaded-shared-memory-gaussian-jobs","title":"Multithreaded shared memory Gaussian jobs","text":"

The main Gaussian executable lets you run jobs that use from 1 core up to a full node. When using more than one core, make sure your input file contains %NProcShared= with the number of cores your job is requesting.

$GAUSS_SCRDIR is where Gaussian puts temporary files which can use a lot of space. On Myriad in a job this is created inside $TMPDIR by default. On diskless clusters, this is set this to a directory in your Scratch instead: loading one of the Gaussian modules will handle this automatically and show where it has created the directory.

# Example for Gaussian 16\n\n# Set up runtime environment\nmodule load gaussian/g16-a03/pgi-2016.5\nsource $g16root/g16/bsd/g16.profile\n\n# Run g16 job\ng16 input.com\n
# Example for Gaussian 09\n\n# Setup runtime environment\nmodule load gaussian/g09-d01/pgi-2015.7\nsource $g09root/g09/bsd/g09.profile\n\n# Run g09 job\ng09 input.com\n
"},{"location":"Software_Guides/Other_Software/#linda-parallel-gaussian-jobs","title":"Linda parallel Gaussian jobs","text":"

Only currently working for Gaussian 09.

Gaussian Linda jobs can run across multiple nodes.

# Select the MPI parallel environment and 80 cores total\n#$ -pe mpi 80\n\n# 8. Select number of threads per Linda worker (value of NProcShared in your\n#     Gaussian input file. This will give 80/40 = 2 Linda workers.\nexport OMP_NUM_THREADS=40\n\n# Setup g09 runtime environment\nmodule load gaussian/g09-d01/pgi-2015.7\nsource $g09root/g09/bsd/g09.profile\n\n# Pre-process g09 input file to include nodes allocated to job\necho \"Running: lindaConv testdata.com $JOB_ID $TMPDIR/machines\"\necho ''\n$lindaConv testdata.com $JOB_ID $TMPDIR/machines\n\n# Run g09 job\n\necho \"Running: g09 \\\"job$JOB_ID.com\\\"\"\n\n# communication needs to be via ssh not the Linda default\nexport GAUSS_LFLAGS='-v -opt \"Tsnet.Node.lindarsharg: ssh\"'\n\ng09 \"job$JOB_ID.com\"\n
"},{"location":"Software_Guides/Other_Software/#troubleshooting-memory-errors","title":"Troubleshooting: Memory errors","text":"

If you encounter errors like:

Out-of-memory error in routine ShPair-LoodLd2 (IEnd= 257724 MxCore= 242934)\n\nUse %mem=48MW to provide the minimum amount of memory required to complete this step.\n

Try adding this to your jobscript:

export GAUSS_MEMDEF=48000000\n

You may need to increase this value even more to allow it to run.

"},{"location":"Software_Guides/Other_Software/#troubleshooting-no-space-left-on-device","title":"Troubleshooting: No space left on device","text":"

If you get this error

  g_write: No space left on device\n

The $GAUSS_SCRDIR is probably full - if it was on a cluster that has local disks and is using $TMPDIR you should increase the amount of tmpfs you are requesting in your jobscript. Otherwise check lquota for your data usage and potentially request a larger Scratch.

"},{"location":"Software_Guides/Other_Software/#gromacs","title":"GROMACS","text":"

We have many versions of GROMACS installed, some built with Plumed. The module name will indicate this.

Which executable you should run depends on the problem you wish to solve. For both single and double precision version builds, serial binaries and an MPI binary for mdrun (mdrun_mpi for newer versions, gmx_mpi for Plumed and some older versions) are provided. Double precision binaries have a _d suffix (so gmx_d, mdrun_mpi_d, gmx_mpi_d etc).

You can see what the executable names are by running module show gromacs/2021.2/gnu-7.3.0 for example and then running the ls command on the bin directory that the module tells you that version is installed in.

# Example for GPU gromacs/2021.5/cuda-11.3\nmodule load beta-modules\nmodule unload -f compilers mpi gcc-libs\nmodule load gcc-libs/10.2.0\nmodule load compilers/gnu/10.2.0\nmodule load\u00a0python3/3.9-gnu-10.2.0\u00a0\nmodule load cuda/11.3.1/gnu-10.2.0\nmodule load mpi/openmpi/4.0.5/gnu-10.2.0\nmodule load\u00a0gromacs/2021.5/cuda-11.3\n\n# Run GROMACS - the executables are gmx_cuda, gmx_mpi_cuda and mdrun_mpi_cuda\n
# Example for gromacs/2021.2/gnu-7.3.0\nmodule load beta-modules\nmodule unload -f compilers mpi gcc-libs\nmodule load gcc-libs/7.3.0\nmodule load compilers/gnu/7.3.0\nmodule load mpi/openmpi/3.1.4/gnu-7.3.0\nmodule load python3\nmodule load gromacs/2021.2/gnu-7.3.0\n\n# Run GROMACS - replace with mdrun command line suitable for your job!\n\ngerun mdrun_mpi -v -stepout 10000\n
# Example for gromacs/2019.3/intel-2018\nmodule unload -f compilers mpi\nmodule load compilers/intel/2018/update3\nmodule load mpi/intel/2018/update3/intel\nmodule load gromacs/2019.3/intel-2018\n\n# Run GROMACS - replace with mdrun command line suitable for your job!\n\ngerun mdrun_mpi -v -stepout 10000\n
# Plumed example for gromacs/2019.3/plumed/intel-2018\nmodule unload -f compilers mpi\nmodule load compilers/intel/2018/update3 \nmodule load mpi/intel/2018/update3/intel \nmodule load libmatheval \nmodule load flex \nmodule load plumed/2.5.2/intel-2018\nmodule load gromacs/2019.3/plumed/intel-2018\n\n# Run GROMACS - replace with mdrun command line suitable for your job!\n\ngerun gmx_mpi -v -stepout 10000\n
"},{"location":"Software_Guides/Other_Software/#passing-in-options-to-gromacs-non-interactively","title":"Passing in options to GROMACS non-interactively","text":"

Some GROMACS executables like trjconv normally take interactive input. You can't do this in a jobscript, so you need to pass in the input you would normally type in. There are several ways of doing this, mentioned at GROMACS Documentation - Using Commands in Scripts. The simplest is to echo the input in and keep your gmx options as they would normally be. If the inputs you would normally type were 3 and 3, then you can do this:

echo 3 3 | gmx whatevercommand -options\n
"},{"location":"Software_Guides/Other_Software/#checkpoint-and-restart","title":"Checkpoint and restart","text":"

GROMACS has built-in checkpoint and restart ability, so you can use this if your runs will not complete in the maximum 48hr wallclock time.

Have a look at the GROMACS manual for full details, as there are more options than mentioned here.

You can tell GROMACS to write a checkpoint file when it is approaching the maximum wallclock time available, and then exit.

In this case, we had asked for 48hrs wallclock. This tells GROMACS to start from the last checkpoint if there is one, and write a new checkpoint just before it reaches 47 hrs runtime.

gerun mdrun_mpi -cpi -maxh 47 <options>\n

The next job you submit with the same script will carry on from the checkpoint the last job wrote. You could use job dependencies to submit two identical jobs at the same time and have one dependent on the other, so it won't start until the first finishes - have a look at man qsub for the -hold_jid option.

You can also write checkpoints at given intervals:

# Write checkpoints every 120 mins, start from checkpoint if there is one.\ngerun mdrun_mpi -cpi -cpt 120 <options>\n
"},{"location":"Software_Guides/Other_Software/#hammock","title":"Hammock","text":"

Hammock is a tool for peptide sequence clustering. It is able to cluster extremely large amounts of short peptide sequences into groups sharing sequence motifs. Typical Hammock applications are NGS-based experiments using large combinatorial peptide libraries, e.g. Phage display.

Hammock has to be installed in your own space to function, so we provide a hammock module that contains the main dependencies and creates a quick-install alias:

# on the login nodes\nmodule unload compilers\nmodule load hammock/1.0.5\ndo-hammock-install\n

This will install Hammock 1.0.5 in your home, edit settings.prop to use clustal-omega and hmmer from our modules and tell it to write temporary files in your Scratch directory (in the form Hammock_temp_time). 

# in your jobscript\nmodule unload compilers\nmodule load hammock/1.0.5\n\n# This copies the MUSI example that comes with Hammock into your working\n# directory and runs it. The module sets $HAMMOCKPATH for you. \n# You must set the output directory to somewhere in Scratch with -d. \n# Below makes a different outputdir per job so multiple runs don't overwrite files.\n\ncp $HAMMOCKPATH/../examples/MUSI/musi.fa .\noutputdir=~/Scratch/hammock-examples/musi_$JOB_ID\nmkdir -p $outputdir\necho \"Running java -jar $HAMMOCKPATH/Hammock.jar full -i musi.fa -d $outputdir\"\n\njava -jar $HAMMOCKPATH/Hammock.jar full -i musi.fa -d $outputdir\n
"},{"location":"Software_Guides/Other_Software/#hopspack","title":"HOPSPACK","text":"

HOPSPACK (Hybrid Optimization Parallel Search PACKage) solves derivative-free optimization problems using an open source, C++ software framework.

We have versions of HOPSPACK built using the GNU compiler and OpenMPI, and the Intel compiler and MPI. This example shows the GNU version. Serial and parallel versions are available, HOPSPACK_main_mpi and HOPSPACK_main_serial.

module unload compilers\nmodule unload mpi\nmodule load compilers/gnu/4.9.2\nmodule load mpi/openmpi/1.8.4/gnu-4.9.2\nmodule load atlas/3.10.2/gnu-4.9.2\nmodule load hopspack/2.0.2/gnu-4.9.2\n\n# Add the examples directory we are using to our path. \n# Replace this with the path to your own executables.\nexport PATH=$PATH:~/Scratch/examples/1-var-bnds-only/\n\n# Run parallel HOPSPACK.\n# Gerun is our mpirun wrapper which sets the machinefile and number of\n# processes to the amount you requested with -pe mpi.\ngerun HOPSPACK_main_mpi ~/Scratch/examples/1-var-bnds-only/example1_params.txt > example1_output.txt\n
"},{"location":"Software_Guides/Other_Software/#idl","title":"IDL","text":"

IDL is a complete environment and language for the analysis and visualisation of scientific and other technical data. It can be used for everything from quick interactive data exploration to building complex applications.

Single-threaded jobscript:

cd $TMPDIR\n\nmodule load idl/8.4.1\n\n# Copy IDL source files to $TMPDIR\ncp ~/Scratch/IDL/fib.pro $TMPDIR\ncp ~/Scratch/IDL/run1.pro $TMPDIR\n\nidl -queue -e @run1.pro\n\n# tar up all contents of $TMPDIR back into your space\ntar zcvf $HOME/Scratch/IDL_output/files_from_job_$JOB_ID.tgz $TMPDIR\n

Parallel jobscript:

cd $TMPDIR\n\nmodule load idl/8.1\n\n# this sets the IDL thread pool: do not change this\nexport IDL_CPU_TPOOL_NTHREADS=$OMP_NUM_THREADS\n\n# Copy IDL source files to $TMPDIR\ncp ~/Scratch/IDL/fib.pro $TMPDIR\ncp ~/Scratch/IDL/run2mp.pro $TMPDIR\n\nidl -queue -e @run2mp.pro\n\n# tar up all contents of $TMPDIR back into your space\ntar zcvf $HOME/Scratch/IDL_output/files_from_job_$JOB_ID.tgz $TMPDIR\n
"},{"location":"Software_Guides/Other_Software/#jags","title":"JAGS","text":"

JAGS (Just Another Gibbs Sampler) is a program for analysis of Bayesian hierarchical models using Markov Chain Monte Carlo (MCMC) simulation not wholly unlike BUGS.

Use this to use JAGS in standalone command line mode:

module unload compilers mpi\nmodule load compilers/gnu/4.9.2\nmodule load openblas/0.2.14/gnu-4.9.2\nmodule load jags/4.2.0/gnu.4.9.2-openblas\n

We have also added JAGS support to r/recommended using the rjags and R2jags R packages.

"},{"location":"Software_Guides/Other_Software/#lammps","title":"LAMMPS","text":"

LAMMPS is an open source parallel molecular dynamics code which exhibits good scaling in a wide range of environments.

The LAMMPS binaries are called lmp_$cluster and all have an lmp_default symlink which can be used.

LAMMPS-8Dec15 and later were built with additional packages kspace, replica, rigid, and class2.

The versions from lammps-16Mar18-basic_install onwards (not lammps/16Mar18/intel-2017) have most of the included packages built. There are also userintel and gpu versions from this point.

We do not install the LAMMPS user packages as part of our central install, but you can build your own version with the ones that you want in your space.

module -f unload compilers mpi\nmodule load compilers/intel/2018\nmodule load mpi/intel/2018\nmodule load lammps/16Mar18/basic/intel-2018\n\n# Gerun is our mpirun wrapper which sets the machinefile and number of\n# processes to the amount you requested with -pe mpi.\ngerun $(which lmp_default) -in inputfile\n

For the latest version of LAMMPS we have installed which is 29th September 2021 Update 2 where the binaries are called lmp_mpi for the MPI version and lmp_gpu for the GPU version:

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load compilers/gnu/10.2.0\nmodule load mpi/openmpi/4.0.5/gnu-10.2.0\nmodule load python3/3.9-gnu-10.2.0\nmodule load lammps/29sep21up2/basic/gnu-10.2.0\n\ngerun lmp_mpi -in inputfile\n

for the basic MPI version and:

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load compilers/gnu/10.2.0\n\n# these three modules for Myriad only\nmodule load numactl/2.0.12\nmodule load binutils/2.36.1/gnu-10.2.0\nmodule load ucx/1.9.0/gnu-10.2.0\n\nmodule load mpi/openmpi/4.0.5/gnu-10.2.0\nmodule load cuda/11.3.1/gnu-10.2.0\nmodule load python3/3.9-gnu-10.2.0\nmodule load lammps/29sep21up2/gpu/gnu-10.2.0\n\ngerun lmp_gpu -sf gpu -pk gpu 1 -in inputfile\n

for the version with GPU support which is only available on clusters with GPUs. The MPI version is available on all clusters. On Myriad the numactl, binutils and ucx modules are additionally needed by OpenMPI.

LAMMPS 29th September 2021 Update 2 has been built with the GNU compilers, OpenMPI and CUDA for the GPU version.

We also have Intel installs:

# LAMMPS 29 Sep 2021 Update 2 with Intel compilers and INTEL package\nmodule unload -f compilers mpi\nmodule load compilers/intel/2020/release\nmodule load mpi/intel/2019/update6/intel\nmodule load python/3.9.10\nmodule load lammps/29sep21up2/userintel/intel-2020\n\ngerun lmp_mpi -in inputfile\n
# LAMMPS 29 Sep 2021 Update 2 for GPU with Intel compilers\nmodule unload -f compilers mpi\nmodule load beta-modules\nmodule load compilers/intel/2020/release\nmodule load mpi/intel/2019/update6/intel\nmodule load python/3.9.10\nmodule load cuda/11.3.1/gnu-10.2.0\nmodule load lammps/29sep21up2/gpu/intel-2020\n\ngerun lmp_gpu -sf gpu -pk gpu 1 -in inputfile\n
"},{"location":"Software_Guides/Other_Software/#meme-suite","title":"MEME Suite","text":"

MEME Suite: Motif-based sequence analysis tools. This install is for the command-line tools and connects to their website for further analysis. 

module unload compilers\nmodule unload mpi\nmodule load compilers/gnu/4.9.2\nmodule load mpi/openmpi/1.8.4/gnu-4.9.2\nmodule load perl/5.22.0\nmodule load python2/recommended\nmodule load ghostscript/9.16/gnu-4.9.2\nmodule load meme/4.10.1_4\n
"},{"location":"Software_Guides/Other_Software/#mirdeep2","title":"miRDeep2","text":"

Discovering known and novel miRNAs from deep sequencing data, miRDeep2 is a completely overhauled tool which discovers microRNA genes by analyzing sequenced RNAs. The tool reports known and hundreds of novel microRNAs with high accuracy in seven species representing the major animal clades. 

module load squid/1.9g\nmodule load randfold/2.0\nmodule load perl/5.22.0\nmodule load bowtie/1.1.2\nmodule load python/2.7.9\nmodule load viennarna/2.1.9\nmodule load mirdeep/2.0.0.7\n
"},{"location":"Software_Guides/Other_Software/#misomisopy","title":"MISO/misopy","text":"

MISO (Mixture of Isoforms) is a probabilistic framework that quantitates the expression level of alternatively spliced genes from RNA-Seq data, and identifies differentially regulated isoforms or exons across samples.

misopy is available as part of the python2/recommended bundle.

MISO can run multithreaded on one node, or can submit multiple independent single-core jobs at once using the --use-cluster option.

If you want to use MISO's ability to create and submit jobs itself, you need a MISO settings file like the one shown below. You give your job options as arguments to the qsub command in the cluster_command line.

Settings files can be used with the --settings-filename=SETTINGS_FILENAME option. You will also need to put your module unload and load commands in your .bashrc if using MISO's own job submission, because you are no longer including them in a jobscript.

Example miso_settings.txt. Multithreaded jobs will use num_processors. num_processors is ignored if --use-cluster is specified:

[data]\nfilter_results = True\nmin_event_reads = 20\n\n[cluster]\ncluster_command = \"qsub -l h_rt=00:10:00 -l mem=1GB -wd ~/Scratch\"\n\n[sampler]\nburn_in = 500\nlag = 10\nnum_iters = 5000\nnum_chains = 6\nnum_processors = 4\n
"},{"location":"Software_Guides/Other_Software/#molpro","title":"MOLPRO","text":"

Molpro is a complete system of ab initio programs for molecular electronic structure calculations.

Molpro 2015.1.3 was provided as binary only and supports communication over Ethernet and not Infiniband - use this one on single-node jobs primarily.

Molpro 2015.1.5 was built from source with the Intel compilers and Intel MPI, so can be run multi-node.

Molpro 2020.1 is a more recent binary install and supports both.

module load molpro/2015.1.5/intel-2015-update2\n\n# Example files available in /shared/ucl/apps/molpro/2015.1.5/intel-2015-update2/molprop_2015_1_linux_x86_64_i8/examples/\n# If this is a multi-node job you need to set the wavefunction directory to \n# somewhere in Scratch with -W. For a single-node job -W should be in $TMPDIR.\n# You can use $SGE_O_WORKDIR to refer to the directory you set with -wd in your jobscript.\n# $NSLOTS will use the number of cores you requested with -pe mpi.\n\necho \"Running molpro -n $NSLOTS -W $TMPDIR h2o_scf.com\"\n\nmolpro -n $NSLOTS -W $TMPDIR h2o_scf.com\n

On Myriad, if you get this error with the MPI 2015 install, please use the binary 2015.1.3 install.

libi40iw-i40iw_ucreate_qp: failed to create QP, unsupported QP type: 0x4\n

Output: MOLPRO can end up writing very many small output files, and this is detrimental to the performance of a parallel filesystem like Lustre. If you are running jobs on Myriad then you should set your -I -d and (especially) -W directories to be in $TMPDIR so they can be accessed quickly and not slow down other jobs. At the end of the job, copy back the data you want to keep into your Scratch.

If you are running parallel multi-node jobs and the directories need to be readable by all the nodes, then you need to write to Scratch.

"},{"location":"Software_Guides/Other_Software/#mrtrix","title":"MRtrix","text":"

MRtrix provides a set of tools to perform diffusion-weighted MRI white matter tractography in the presence of crossing fibres. 

module load python3/recommended\nmodule load qt/4.8.6/gnu-4.9.2\nmodule load eigen/3.2.5/gnu-4.9.2\nmodule load fftw/3.3.6-pl2/gnu-4.9.2\nmodule load mrtrix/3.0rc3/gnu-4.9.2/nogui\n

You must load these modules once from a login node before submitting a job. It copies a .mrtrix.conf to your home directory the first time you run this module from a login node, which sets:

  Analyse.LeftToRight: false\n  NumberOfThreads: 4\n

You need to alter NumberOfThreads to what you are using in your job script before you submit a job.

The MRtrix GUI tools are unavailable: mrview and shview in MRtrix 3 cannot be run over a remote X11 connection so are not usable on our clusters. To use these tools you will need a local install on your own computer.

"},{"location":"Software_Guides/Other_Software/#mutect","title":"MuTect","text":"

MuTect is a tool developed at the Broad Institute for the reliable and accurate identification of somatic point mutations in next generation sequencing data of cancer genomes. It is built on top of the GenomeAnalysisToolkit (GATK), which is also developed at the Broad Institute, so it uses the same command-line conventions and (almost all) the same input and output file formats.

MuTect requires you to agree to the GATK license before we can add you to the lgmutect group which gives you access: you can do this by downloading MuTect from The Broad Institute CGA page. You may need to create a gatkforums account before you can download.

MuTect is currently not compatible with Java 1.8, so you need to use the system Java 1.7. Set up your modules as follows: 

module load mutect/1.1.7\n

Then to run MuTect, you should either prefix the .jar you want to run with $MUTECTPATH:

java -Xmx2g -jar $MUTECTPATH/mutect-1.1.7.jar OPTION1=value1 OPTION2=value2...\n

Or we provide wrappers, so you can run it this way instead: 

mutect OPTION1=value1 OPTION2=value2...\n
"},{"location":"Software_Guides/Other_Software/#namd","title":"NAMD","text":"

NAMD is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems.

We have several different types of install, some of them suited to particular clusters only. To see all the versions, type module avail namd.

These examples are running the apoa1 benchmark, available from the NAMD website.

"},{"location":"Software_Guides/Other_Software/#multicore-gpu","title":"Multicore GPU","text":"

This version of NAMD runs within one GPU node. It can run on multiple GPUs on that node, but not across multiple different nodes. NAMD uses the CPUs and GPUs together so it is recommended you request all the cores on the node if you are requesting all the GPUs.

For best performance of simulations it is recommended that you use an entire node, all the CPUs and all the available GPUs.

# request a number of CPU cores and GPUs\n#$ -pe smp 10\n#$ -l gpu=1\n\nmodule load namd/2.14/multicore-gpu\n\n# ${NSLOTS} will get the number of cores you asked for with -pe smp.\n# +setcpuaffinity is recommended to make sure threads are assigned to specific CPUs.\n\nnamd2 +p${NSLOTS} +setcpuaffinity apoa1_nve_cuda.namd\n
"},{"location":"Software_Guides/Other_Software/#ofi","title":"OFI","text":"

This version of NAMD is for clusters with OmniPath interconnects (not Myriad). It can run across multiple nodes. The OFI versions should use significantly less memory than the older MPI-based installs.

module unload -f compilers mpi\nmodule load compilers/intel/2019/update5\nmodule load mpi/intel/2019/update5/intel\nmodule load namd/2.14/ofi/intel-2019\n\n# ${NSLOTS} will get the number of cores you asked for with -pe.\n\ncharmrun +p${NSLOTS} namd2 apoa1.namd\n
"},{"location":"Software_Guides/Other_Software/#ofi-smp","title":"OFI-SMP","text":"

This version of NAMD runs with threads (smp) and processes and is for clusters with OmniPath interconnects (not Myriad). It can run across multiple nodes. The OFI versions should use significantly less memory than the older MPI-based installs.

module unload -f compilers mpi\nmodule load compilers/intel/2019/update5\nmodule load mpi/intel/2019/update5/intel\nmodule load namd/2.14/ofi-smp/intel-2019\n\n# ${NSLOTS} will get the number of cores you asked for with -pe.\n# +setcpuaffinity is recommended to make sure threads are assigned to specific CPUs.\n# ++ppn is the number of PEs (or worker threads) to create for each process.\n\ncharmrun +p${NSLOTS} namd2 apoa1.namd ++ppn2 +setcpuaffinity\n
"},{"location":"Software_Guides/Other_Software/#ofi-smp-gpu","title":"OFI-SMP-GPU","text":"

This version of NAMD runs with threads (smp) and processes and is for clusters with OmniPath interconnects as well as GPUs (not Myriad). It can run across multiple nodes.

# request a number of CPU cores and GPUs\n#$ -pe smp 24\n#$ -l gpu=2\n\nmodule unload -f compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/7.3.0\nmodule load compilers/intel/2019/update5\nmodule load mpi/intel/2019/update5/intel\nmodule load cuda/11.3.1/gnu-10.2.0\nmodule load namd/2.14/ofi-smp-gpu/intel-2019\n\n# ${NSLOTS} will get the number of cores you asked for with -pe.\n# +setcpuaffinity is recommended to make sure threads are assigned to specific CPUs.\n# ++ppn is the number of PEs (or worker threads) to create for each process.\n\n# The number of GPU devices must be a multiple of the number of NAMD processes\n# since processes cannot share GPUs.\n# Here we have ++ppn12 for 12 threads, and charmrun works out we have 2 NAMD processes \n# available for the 2 GPUs.\n\ncharmrun +p${NSLOTS} namd2 apoa1_nve_cuda.namd ++ppn12 +setcpuaffinity \n
"},{"location":"Software_Guides/Other_Software/#mpi","title":"MPI","text":"

These are older versions. It is recommended to run the OFI versions above instead if possible.

module load fftw/2.1.5/intel-2015-update2\nmodule load namd/2.13/intel-2018-update3\n\n# GErun is our mpirun wrapper that gets $NSLOTS and the machinefile for you.\n\ngerun namd2 apoa1.namd \n
"},{"location":"Software_Guides/Other_Software/#nextflow","title":"Nextflow","text":"

We do not currently have central installs of Nextflow, but a group of UCL researchers have contributed a config file and instructions for Myriad at the nf-core/configs repository

Nextflow containers can be run using Singularity.

"},{"location":"Software_Guides/Other_Software/#nonmem","title":"NONMEM","text":"

NONMEM\u00ae is a nonlinear mixed effects modelling tool used in population pharmacokinetic / pharmacodynamic analysis.

We have one build that uses the GNU compiler and ATLAS and an Intel build using MKL. Both use Intel MPI.

This example uses the Intel build.

jobDir=example1_parallel_$JOB_ID\nmkdir $jobDir\n\n# Copy control and datafiles to jobDir\ncp /shared/ucl/apps/NONMEM/examples/foce_parallel.ctl $jobDir\ncp /shared/ucl/apps/NONMEM/examples/example1b.csv $jobDir\ncd $jobDir\n\nmodule unload compilers mpi\nmodule load compilers/intel/2015/update2\nmodule load mpi/intel/2015/update3/intel\nmodule load nonmem/7.3.0/intel-2015-update2\n\n# Create parafile for job using $TMPDIR/machines\nparafile.sh $TMPDIR/machines > example1.pnm\n\nnmfe73 foce_parallel.ctl example1.res -parafile=example1.pnm -background -maxlim=1 > example1.log\n
"},{"location":"Software_Guides/Other_Software/#nwchem","title":"NWChem","text":"

NWChem applies theoretical techniques to predict the structure, properties, and reactivity of chemical and biological species ranging in size from tens to millions of atoms.

You should load the NWChem module you wish to use once from a login node, as it will create a symlinked .nwchemrc in your home. 

module unload compilers mpi\nmodule load compilers/intel/2017/update4\nmodule load mpi/intel/2017/update3/intel\nmodule load python/2.7.12\nmodule load nwchem/6.8-47-gdf6c956/intel-2017\n\n# $NSLOTS will get the number of processes you asked for with -pe mpi.\nmpirun -np $NSLOTS -machinefile $TMPDIR/machines nwchem hpcvl_sample.nw\n
"},{"location":"Software_Guides/Other_Software/#nwchem-troubleshooting","title":"NWChem troubleshooting","text":"

If you get errors like this

{    0,    3}:  On entry to PDSTEDC parameter number   10 had an illegal value\n

then you are coming across an error in Intel MKL 2018, and should make sure you change to the Intel 2017 compiler module as shown above. (MKL versions are bundled with the corresponding Intel compiler modules).

If your run terminates with an error saying

ARMCI supports block process mapping only\n

then you are probably trying to use round-robin MPI process placement, which ARMCI does not like. gerun uses round-robin for Intel MPI by default as it works better in most cases. Use mpirun instead of gerun:

mpirun -np $NSLOTS -machinefile $TMPDIR/machines nwchem input.nw\n

If you get an error complaining about $NWCHEM_NWPW_LIBRARY similar to this:

warning:::::::::::::: from_compile\nNWCHEM_NWPW_LIBRARY is: <\n/dev/shm/tmp.VB3DpmjULc/nwchem-6.6/src/nwpw/libraryps/>\nbut file does not exist or you do not have access to it !\n------------------------------------------------------------------------\nnwpwlibfile: no nwpw library found 0\n

then your ~/.nwchemrc symlink is likely pointing to a different version that you used previously. Deleting the symlink and loading the module you want to use will recreate it correctly.

"},{"location":"Software_Guides/Other_Software/#orca","title":"ORCA","text":"

ORCA is an ab initio, DFT, and semi-empirical SCF-MO package.

module unload compilers \nmodule unload mpi\nmodule load compilers/gnu/4.9.2\nmodule load mpi/openmpi/3.1.4/gnu-4.9.2\nmodule load orca/4.2.1-bindist/gnu-4.9.2\n\norca input.inp > output.out\n

If you want to run ORCA in parallel using MPI, the jobscript will be the same but you will need to add the !PAL keyword to your input file to tell it how many processes to use. (You do not use mpirun or gerun with ORCA).

"},{"location":"Software_Guides/Other_Software/#picard","title":"Picard","text":"

Picard comprises Java-based command-line utilities that manipulate SAM files, and a Java API (SAM-JDK) for creating new programs that read and write SAM files. Both SAM text format and SAM binary (BAM) format are supported.

Picard requires a Java 1.8 module to be loaded.

module load java/1.8.0_92\nmodule load picard-tools/2.18.9\n

To run Picard you can prefix the .jar you want to run with $PICARDPATH and give the full command, or we have wrappers:

java -Xmx2g -jar $PICARDPATH/picard.jar PicardCommand TMP_DIR=$TMPDIR OPTION1=value1 OPTION2=value2...\n

The wrappers allow you to run commands like this - in this case our wrapper sets TMP_DIR for you as well: 

PicardCommand OPTION1=value1 OPTION2=value2...\n

Temporary files: by default, Picard writes temporary files into /tmp rather than into $TMPDIR. These are not cleaned up after your job ends, and means future runs can fail because /tmp is full (and requesting more tmpfs in your job doesn't make it larger). If you run Picard with the full java -jar command then give Picard the TMP_DIR=$TMPDIR option as our example above does to get it to write in the correct place.

"},{"location":"Software_Guides/Other_Software/#quantum-espresso","title":"Quantum Espresso","text":"

Quantum Espresso is an integrated suite of Open-Source computer codes for electronic-structure calculations and materials modelling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials. 

module load xorg-utils\nmodule load quantum-espresso/6.1-impi/intel2017\n\n# Set the path here to where ever you keep your pseudopotentials.\nexport ESPRESSO_PSEUDO=$HOME/qe-psp\n\n# Gerun is our mpirun wrapper which sets the machinefile and number of\n# processes to the amount you requested with -pe mpi.\ngerun pw.x -in input.in >output.out\n
"},{"location":"Software_Guides/Other_Software/#repast-hpc","title":"Repast HPC","text":"

Repast for High Performance Computing (Repast HPC) is a next generation agent-based modelling system intended for large-scale distributed computing platforms. It implements the core Repast Simphony concepts (e.g. contexts and projections), modifying them to work in a parallel distributed environment. 

module unload compilers\nmodule unload mpi\nmodule load compilers/gnu/4.9.2\nmodule load hdf/5-1.8.15/gnu-4.9.2\nmodule load netcdf/4.3.3.1/gnu-4.9.2\nmodule load netcdf-fortran/4.4.1/gnu-4.9.2\nmodule load mpi/openmpi/1.8.4/gnu-4.9.2\nmodule load python/2.7.9\nmodule load boost/1_54_0/mpi/gnu-4.9.2\nmodule load netcdf-c++/4.2/gnu-4.9.2\nmodule load repast-hpc/2.1/gnu-4.9.2\n

The module sets the environment variables $REPAST_HPC_INCLUDE, $REPAST_HPC_LIB_DIR and $REPAST_HPC_LIB.

"},{"location":"Software_Guides/Other_Software/#root","title":"ROOT","text":"

ROOT provides a set of OO frameworks for handling, analysing, and visualising large amounts of data. Included are specialised storage methods, methods for histograming, curve fitting, function evaluation, minimization etc. ROOT includes a built-in CINT C++ interpreter. 

module unload compilers mpi\nmodule load compilers/gnu/4.9.2\nmodule load fftw/3.3.4/gnu-4.9.2\nmodule load gsl/1.16/gnu-4.9.2\nmodule load root/6.04.00/gnu-4.9.2\n\n# run root in batch mode\nroot -b -q myMacro.C > myMacro.out\n
"},{"location":"Software_Guides/Other_Software/#sas","title":"SAS","text":"

SAS is a statistics package providing a wide range of tools for data management, analysis and presentation. 

cd $TMPDIR\n\nmodule load sas/9.4/64\n\n# copy all your input files into $TMPDIR\ncp ~/Scratch/sas_input/example1/* $TMPDIR\n\nsas example1.in\n\n# tar up all contents of $TMPDIR back into your space\ntar cvzf $HOME/Scratch/SAS_output/files_from_job_$JOB_ID.tgz $TMPDIR\n
"},{"location":"Software_Guides/Other_Software/#starccm","title":"StarCCM+","text":"

StarCCM+ is a commercial CFD package that handles fluid flows, heat transfer, stress simulations, and other common applications of such.

Before running any StarCCM+ jobs on the clusters you must load the StarCCM+ module on a login node. This is so the module can set up two symbolic links in your home directory to directories created in your Scratch area so that user settings etc can be written by running jobs.

module load star-ccm+/13.06.012\n

Here is the jobscript example.

# Request one license per core - makes sure your job doesn't start \n# running until sufficient licenses are free.\n#$ -l ccmpsuite=1\n\nmodule load star-ccm+/13.06.012\n\nstarccm+ -np $NSLOTS -machinefile $TMPDIR/machines -rsh ssh -batch my_input.sim\n
"},{"location":"Software_Guides/Other_Software/#hfi-error","title":"hfi error","text":"

If you get an error like this:

hfi_wait_for_device: The /dev/hfi1_0 device failed to appear after 15.0 seconds: Connection timed out\n

then you need to add -fabric ibv to your options as shown in the example script.

It is trying to use an OmniPath device on a cluster that has InfiniBand, so the fabric needs to be changed. If you have this left over in jobscripts from Grace, you need to remove it on Kathleen.

"},{"location":"Software_Guides/Other_Software/#starcd","title":"StarCD","text":"

StarCD is a commercial package for modelling and simulating combustion and engine dynamics.

You must request access to the group controlling StarCD access (legstarc) to use it. The license is owned by the Department of Mechanical Engineering who will need to approve your access request. 

# Request one license per core - makes sure your job doesn't start \n# running until sufficient licenses are free.\n#$ -l starsuite=1\n\nmodule load star-cd/4.28.050\n\n# run star without its tracker process as this causes multicore jobs to die early\nstar -notracker\n

StarCD uses IBM Platform MPI by default. You can also run StarCD simulations using Intel MPI by changing the command line to:

star -notracker -mpi=intel\n

Simulations run using Intel MPI may run faster than they do when using IBM Platform MPI.

If being run on a diskless cluster without available $TMPDIR like Kathleen, then StarCD will create a $HPC_SCRATCH location to store its temporary files when the module is loaded. In a job this is set to $HOME/Scratch/STAR_ScrDirs/[randomLabel] and it will make this directory and notify that it did this in your .e file. You can delete the randomly-named directory after the job ends. To set the location yourself, after you load the module you can set it to any other existing directory instead:

export HPC_SCRATCH=/path/to/desired/location\n
"},{"location":"Software_Guides/Other_Software/#statamp","title":"Stata/MP","text":"

Stata is a statistics, data management, and graphics system. Stata/MP is the version of the package that runs on multiple cores.

We have a sixteen user license of Stata/MP. Our license supports Stata running on up to four cores per job. 

# Select 4 OpenMP threads (the most possible)\n#$ -pe smp 4\n\ncd $TMPDIR\nmodule load stata/15\n\n# copy files to $TMPDIR\ncp myfile.do $TMPDIR\n\nstata-mp -b do myfile.do\n\n# tar up all contents of $TMPDIR back into your space\ntar zcvf $HOME/Scratch/Stata_output/files_from_job_$JOB_ID.tar.gz $TMPDIR\n
"},{"location":"Software_Guides/Other_Software/#torch","title":"Torch","text":"

Torch is a scientific computing framework with wide support for machine learning algorithms that puts GPUs first.

We provide a torch-deps module that contains the main Torch dependencies and creates a quick-install alias, do-torch-install. This uses Torch's installation script to git clone the current distribution and install LuaJIT, LuaRocks and Torch in ~/torch. 

module unload compilers mpi\nmodule load torch-deps\n\ndo-torch-install\n

You should load these same modules in your jobscript when using the version of torch this installs.

"},{"location":"Software_Guides/Other_Software/#turbomole","title":"Turbomole","text":"

Turbomole is an ab initio computational chemistry program that implements various quantum chemistry methods. Turbomole has a Chemistry-wide license. Reserved application group legtmole for Chemistry users only.

There are scripts you can use to generate Turbomole jobs for you:

/shared/ucl/apps/turbomole/turbomole-mpi.submit\n/shared/ucl/apps/turbomole/turbomole-smp.submit\n

They will ask you which version you want to use, how much memory, how many cores etc and set up and submit the job for you.

Use the first for MPI jobs and the second for single-node shared memory threaded jobs.

"},{"location":"Software_Guides/Other_Software/#varscan","title":"VarScan","text":"

VarScan is a platform-independent mutation caller for targeted, exome, and whole-genome resequencing data generated on Illumina, SOLiD, Life/PGM, Roche/454, and similar instruments. 

module load java/1.8.0_45\nmodule load varscan/2.3.9\n

Then to run VarScan, you should either prefix the .jar you want to run with $VARSCANPATH:

java -Xmx2g -jar $VARSCANPATH/VarScan.v2.3.9.jar OPTION1=value1 OPTION2=value2...\n

Or we provide wrappers, so you can run it this way instead:

varscan OPTION1=value1 OPTION2=value2...\n
"},{"location":"Software_Guides/Other_Software/#vasp","title":"VASP","text":"

The Vienna Ab initio Simulation Package (VASP) is a computer program for atomic scale materials modelling, e.g. electronic structure calculations and quantum-mechanical molecular dynamics, from first principles.

VASP is licensed software. To gain access, you need to email us letting us know what email address you are named on a VASP license using. You can also mention the name and email of the main VASP license holder and the license number if you have it, though this is not necessary. We will then check in the VASP portal if we can give you access. We will add you to the legvasp5 or legvasp6 reserved application groups depending on which versions you are licensed for, and remove you when VASP tell us you no longer have access.

The VASP executables for current versions are named like this:

"},{"location":"Software_Guides/Other_Software/#vasp-5","title":"VASP 5","text":"
# vasp 5\nmodule unload -f compilers mpi\nmodule load compilers/intel/2017/update1\nmodule load mpi/intel/2017/update1/intel\nmodule load vasp/5.4.4-18apr2017/intel-2017-update1\n\n# Gerun is our mpirun wrapper which sets the machinefile and number of\n# processes to the amount you requested with -pe mpi.\ngerun vasp_std > vasp_output.$JOB_ID\n

Note: although you can run VASP using the default Intel 2018 compiler this can lead to numerical errors in some types of simulation. In those cases we recommend switching to the specific compiler and MPI version used to build that install (mentioned at the end of the module name). We do this in the example above.

Building your own VASP: You may also install your own copy of VASP in your home if you have access to the source, and we provide a simple VASP individual install script (tested with VASP 5.4.4, no patches). You need to download the VASP source code into your home directory and then you can run the script following the instructions at the top.

"},{"location":"Software_Guides/Other_Software/#vasp-6","title":"VASP 6","text":"
# vasp 6\nmodule unload -f compilers mpi\nmodule load compilers/intel/2019/update5\nmodule load mpi/intel/2019/update5/intel\nmodule load vasp/6.3.0-24Jan2022/intel-2019-update5\n\ngerun vasp_std > vasp_output.$JOB_ID\n
"},{"location":"Software_Guides/Other_Software/#vasp-6-gpu","title":"VASP 6 GPU","text":"

This is the OpenACC GPU port of VASP. The VASP documentation has some information about suitable numbers of MPI processes vs GPUs.

# vasp 6 GPU\n\n# request a gpu\n#$ -l gpu=1\n\nmodule unload -f compilers mpi\nmodule load compilers/nvidia/hpc-sdk/22.1\nmodule load fftw/3.3.10/nvidia-22.1\nmodule load vasp/6.3.0-24Jan2022/nvidia-22.1-gpu\n\ngerun vasp_std > vasp_output.$JOB_ID\n
"},{"location":"Software_Guides/Other_Software/#xmds","title":"XMDS","text":"

XMDS allows the fast and easy solution of sets of ordinary, partial and stochastic differential equations, using a variety of efficient numerical algorithms.

We have XMDS 3 and XMDS 2 installed.

For XMDS 3.0.0 you will need to load the modules on a login node and run xmds3-setup to set up XMDS.

module unload compilers\nmodule unload mpi\nmodule load compilers/gnu/4.9.2\nmodule load mpi/intel/2015/update3/gnu-4.9.2\nmodule load python3/3.7\nmodule load fftw/3.3.4-impi/gnu-4.9.2\nmodule load hdf/5-1.8.15/gnu-4.9.2\nmodule load xmds/3.0.0\n\n# run this on a login node to set up XMDS\nxmds3-setup\n

You can also build the current developmental version from SVN in your space by running create-svn-xmds3-inst.

For XMDS 2.2.2 you will need to load the modules on a login node and run xmds2-setup to set up XMDS. 

module unload compilers\nmodule unload mpi\nmodule load compilers/gnu/4.9.2\nmodule load mpi/intel/2015/update3/gnu-4.9.2\nmodule load python2/recommended\nmodule load fftw/3.3.4-impi/gnu-4.9.2\nmodule load hdf/5-1.8.15/gnu-4.9.2\nmodule load xmds/2.2.2\n\n# run this on a login node to set up XMDS\nxmds2-setup\n

Note that the create-svn-xmds-inst SVN install using the 2.2.2 modules will no longer work since the release of XMDS 3.0.0 (see above to use that).

"},{"location":"Software_Guides/R/","title":"R","text":"

Type module avail r to see the currently available versions of R.

The current version will always also exist as r/recommended - this is a module bundle and loading it will also load its many dependencies.

module show r/recommended shows you exactly which versions loading this module will give you.

R can be run on a single core or multithreaded using many cores (some commands can run threaded automatically, otherwise you may wish to look at R's parallel package).

doMPI, Rmpi and snow allow multi-node parallel jobs using MPI to be run.

List of additional R packages shows you what packages are installed and available for the current R version.

"},{"location":"Software_Guides/R/#setup","title":"Setup","text":"

Before you can use R interactively, you need to load the R module using: 

module -f unload compilers mpi gcc-libs\nmodule load r/recommended\n
"},{"location":"Software_Guides/R/#example-serial-jobscript","title":"Example serial jobscript","text":"

This script runs R using only one core.

#!/bin/bash -l\n\n# Example jobscript to run a single core R job\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n# Change this to suit your requirements.\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM. Change this to suit your requirements.\n#$ -l mem=1G\n\n# Set the name of the job. You can change this if you wish.\n#$ -N R_job_1\n\n# Set the working directory to somewhere in your scratch space.  This is\n# necessary because the compute nodes cannot write to your $HOME\n# NOTE: this directory must exist.\n# Replace \"<your_UCL_id>\" with your UCL user ID\n#$ -wd /home/<your_UCL_id>/Scratch/R_output\n\n# Your work must be done in $TMPDIR (serial jobs particularly) \ncd $TMPDIR\n\n# Load the R module and run your R program\nmodule -f unload compilers mpi gcc-libs\nmodule load r/recommended\n\nR --no-save < /home/username/Scratch/myR_job.R > myR_job.out\n\n# Preferably, tar-up (archive) all output files to transfer them back \n# to your space. This will include the R_output file above.\ntar zcvf $HOME/Scratch/R_output/files_from_job_$JOB_ID.tgz $TMPDIR\n\n# Make sure you have given enough time for the copy to complete!\n

You will need to change the -wd /home/<your_UCL_id>/Scratch/R_output location and the location of your R input file, called myR_job.R here. myR_job.out is the file we are redirecting the output into. The output file is saved in the tar archive produced by the last command in the runscript and will be in $HOME/Scratch/R_output.

If your jobscript is called run-R.sh then your job submission command would be:

qsub run-R.sh\n
"},{"location":"Software_Guides/R/#example-shared-memory-threaded-parallel-job","title":"Example shared memory threaded parallel job","text":"

This script uses multiple cores on the same node. It cannot run across multiple nodes.

#!/bin/bash -l\n\n# Example jobscript to run an OpenMP threaded R job across multiple cores on one node.\n# This may be using the foreach packages foreach(...) %dopar% for example.\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n# Change this to suit your requirements.\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM per core. Change this to suit your requirements.\n#$ -l mem=1G\n\n# Set the name of the job. You can change this if you wish.\n#$ -N R_jobMC_2\n\n# Select 12 threads. The number of threads here must equal the number of worker \n# processes in the registerDoMC call in your R program.\n#$ -pe smp 12\n\n# Set the working directory to somewhere in your scratch space.  This is\n# necessary because the compute nodes cannot write to your $HOME\n# NOTE: this directory must exist.\n# Replace \"<your_UCL_id>\" with your UCL user ID\n#$ -wd /home/<your_UCL_id>/Scratch/R_output\n\n# Your work must be done in $TMPDIR\ncd $TMPDIR\n\n# Load the R module and run your R program\nmodule -f unload compilers mpi gcc-libs\nmodule load r/recommended\n\nR --no-save < /home/username/Scratch/myR_job.R > myR_job.out\n\n# Preferably, tar-up (archive) all output files to transfer them back \n# to your space. This will include the R_output file above.\ntar zcvf $HOME/Scratch/R_output/files_from_job_$JOB_ID.tgz $TMPDIR\n\n# Make sure you have given enough time for the copy to complete!\n

You will need to change the -wd /home/<your_UCL_id>/Scratch/R_output location and the location of your R input file, called myR_job.R here. myR_job.out is the file we are redirecting the output into. The output file is saved in the tar archive produced by the last command in the runscript and will be in $HOME/Scratch/R_output.

If your jobscript is called run-R.sh then your job submission command would be:

qsub run-R.sh\n
"},{"location":"Software_Guides/R/#example-multi-node-parallel-job-using-rmpi-and-snow","title":"Example multi-node parallel job using Rmpi and snow","text":"

This script uses Rmpi and snow to allow it to run across multiple nodes using MPI.

#!/bin/bash -l\n\n# Example jobscript to run an R MPI parallel job\n\n# Request ten minutes of wallclock time (format hours:minutes:seconds).\n#$ -l h_rt=0:10:0\n\n# Request 1 gigabyte of RAM per process.\n#$ -l mem=1G\n\n# Request 15 gigabytes of TMPDIR space per node (default is 10 GB)\n#$ -l tmpfs=15G\n\n# Set the name of the job.\n#$ -N snow_monte_carlo\n\n# Select the MPI parallel environment with 32 processes\n#$ -pe mpi 32\n\n# Set the working directory to somewhere in your scratch space.  This is\n# necessary because the compute nodes cannot write to your $HOME\n# NOTE: this directory must exist.\n# Replace \"<your_UCL_id>\" with your UCL user ID\n#$ -wd /home/<your_UCL_id>/Scratch/R_output\n\n# Load the R module\nmodule -f unload compilers mpi gcc-libs\nmodule load r/recommended\n\n# Copy example files in to the working directory (not necessary if already there)\ncp ~/R/Examples/snow_example.R .\ncp ~/R/Examples/monte_carlo.R .\n\n# Run our MPI job. GERun is our wrapper for mpirun, which launches MPI jobs  \ngerun RMPISNOW < snow_example.R > snow.out.${JOB_ID}\n

The output file is saved in $HOME/Scratch/R_examples/snow/snow.out.${JOB_ID}.

If your jobscript is called run-R-snow.sh then your job submission command would be:

qsub run-R-snow.sh\n
"},{"location":"Software_Guides/R/#example-r-script-using-rmpi-and-snow","title":"Example R script using Rmpi and snow","text":"

This R script has been written to use Rmpi and snow and can be used with the above jobscript. It is snow_example.R above.

#Load the snow and random number package.\nlibrary(snow)\nlibrary(Rmpi)\n\n# This example uses the already installed LEcuyers RNG library(rlecuyer)\nlibrary(rlecuyer)\n\n# Set up our input/output\nsource('./monte_carlo.R')\nsink('./monte_carlo_output.txt')\n\n# Get a reference to our snow cluster that has been set up by the RMPISNOW\n# script.\ncl <- getMPIcluster ()\n\n# Display info about each process in the cluster\nprint(clusterCall(cl, function() Sys.info()))\n\n# Load the random number package on each R process\nclusterEvalQ (cl, library (rlecuyer))\n\n# Generate a seed for the pseudorandom number generator, unique to each\n# processor in the cluster.\n\n#Uncomment below line for default (unchanging) random number seed.\n#clusterSetupRNG(cl, type = 'RNGstream')\n\n#The lines below set up a time-based random number seed.  Note that \n#this only demonstrates the virtues of changing the seed; no guarantee\n#is made that this seed is at all useful.  Comment out if you uncomment\n#the above line.\ns <- sum(strtoi(charToRaw(date()), base = 32))\nclusterSetupRNGstream(cl, seed=rep(s,6))\n\n#Choose which of the following blocks best fit your own needs.\n\n# BLOCK 1\n# Set up the input to our Monte Carlo function.\n# Input is identical across the batch, only RNG seed has changed. \n# For this example, both clusters will roll one die. \n\nnrolls <- 2\nprint(\"Roll the dice once...\")\noutput <- clusterCall(cl, monte_carlo, nrolls)\noutput\nprint(\"Roll the dice again...\")\noutput <- clusterCall(cl, monte_carlo, nrolls)\noutput\n\n# Output should show the results of two rolls of a six-sided die.\n\n#BLOCK 2\n# Input is different for each processor\nprint(\"Second example: coin flip plus 3 dice\")\ninput <- array(1:2)  # Set up array of inputs, with each entry\ninput[1] <- 1        #   corresponding to one processor.\ninput[2] <- 3\nparameters <- array(1:2)  # Set up inputs that will be used by each cluster.\nparameters[1] <- 2        #   These will be passed to monte_carlo as its\nparameters[2] <- 6        #   second argument.\noutput <- clusterApply(cl, input, monte_carlo, parameters)\n\n# Output should show the results of a coin flip and the roll of three \n# six-sided die.\n\n# Output the output.\noutput\n\ninputStrings <- array(1:2)\ninputStrings[1] <- 'abc'\ninputStrings[2] <- 'def'\noutput <- clusterApply(cl, inputStrings, paste, 'foo')\noutput\n\n#clusterEvalQ(cl, sinkWorkerOutput(\"snow_monte_carlo.out\"))\n\n# Clean up the cluster and release the relevant resources.\nstopCluster(cl)\nsink()\n\nmpi.quit()\n

This is monte_carlo.R which is called by snow_example.R:

monte_carlo <- function(x, numsides=6){\n  streamname <- .lec.GetStreams ()\n  dice <- .lec.uniform.int(streamname[1], n = 1, a=1, b=numsides)\n  outp <- sum(dice)\n  return(outp)\n}\n

This example is based on SHARCNET's Using R and MPI.

"},{"location":"Software_Guides/R/#using-your-own-r-packages","title":"Using your own R packages","text":"

If we do not have R packages installed centrally that you wish to use, you can install them in your space on the cluster and tell R where to find them.

First you need to tell R where to install your package to and where to look for user-installed packages, using the R library path.

"},{"location":"Software_Guides/R/#set-your-r-library-path","title":"Set your R library path","text":"

There are several ways to modify your R library path so you can pick up packages that you have installed in your own space.

The easiest way is to add them to the R_LIBS environment variable (insert the correct path):

export R_LIBS=/your/local/R/library/path:$R_LIBS\n

This is a colon-separated list of directories that R will search through.

Setting that in your terminal will let you install to that path from inside R and should also be put in your jobscript (or your .bashrc) when you submit a job using those libraries. This appends your directory to the existing value of $R_LIBS rather than overwriting it so the centrally-installed libraries can still be found.

You can also change the library path for a session from within R:

.libPaths(c('~/MyRlibs',.libPaths()))\n

This puts your directory at the beginning of R's search path, and means that install.packages() will automatically put packages there and the library() function will find libraries in your local directory.

"},{"location":"Software_Guides/R/#install-an-r-package","title":"Install an R package","text":"

To install, after setting your library path:

From inside R, you can do

install.packages('package_name', repos=\"http://cran.r-project.org\")\n

Or if you have downloaded the tar file, you can do

R CMD INSTALL -l /home/username/your_R_libs_directory package.tar.gz\n

If you want to keep some libraries separate, you can have multiple colon-separated paths in your $R_LIBS and specify which one you want to install into with R CMD INSTALL.

"},{"location":"Software_Guides/R/#bioconductor","title":"BioConductor","text":"

If you are installing extra packages for BioConductor, check that you are using the same version that the R module you have loaded is using.

Eg. you can find the BioConductor 3.15 package downloads here.

"},{"location":"Software_Guides/Singularity/","title":"Using Singularity on Our Clusters","text":"

Singularity is installed on all our clusters. You can use containers you have downloaded in your space.

Run singularity --version to see which version we currently have installed.

"},{"location":"Software_Guides/Singularity/#set-up-cache-locations-and-bind-directories","title":"Set up cache locations and bind directories","text":"

The cache directories should be set to somewhere in your space so they don't fill up /tmp on the login nodes.

The bindpath mentioned below specifies what directories are made available inside the container - only your home is bound by default so you need to add Scratch.

You can either use the singularity-env environment module for this, or run the commands manually.

module load singularity-env\n

or:

# Create a .singularity directory in your Scratch\nmkdir $HOME/Scratch/.singularity\n\n# Create cache subdirectories we will use / export\nmkdir $HOME/Scratch/.singularity/tmp\nmkdir $HOME/Scratch/.singularity/localcache\nmkdir $HOME/Scratch/.singularity/pull\n\n# Set all the Singularity cache dirs to Scratch\nexport SINGULARITY_CACHEDIR=$HOME/Scratch/.singularity\nexport SINGULARITY_TMPDIR=$SINGULARITY_CACHEDIR/tmp\nexport SINGULARITY_LOCALCACHEDIR=$SINGULARITY_CACHEDIR/localcache\nexport SINGULARITY_PULLFOLDER=$SINGULARITY_CACHEDIR/pull\n\n# Bind your Scratch directory so it is accessible from inside the container\n#      and the temporary storage jobs are allocated\nexport SINGULARITY_BINDPATH=/scratch/scratch/$USER,/tmpdir\n

Different subdirectories are being set for each cache so you can tell which files came from where.

You probably want to add those export statements to your .bashrc under # User specific aliases and functions so those environment variables are always set when you log in.

For more information on these options, have a look at the Singularity documentation:

"},{"location":"Software_Guides/Singularity/#downloading-and-running-a-container","title":"Downloading and running a container","text":"

Assuming you want to run an existing container, first you need to pull it from somewhere online that provides it:

# make sure we set up singularity as above\nmodule load singularity-env\n\n# get image from location and call it hello-world.sif in our current directory\nsingularity pull hello-world.sif shub://vsoch/hello-world\n

Run the container.

singularity run hello-world.sif\n

Run a specific command within our container.

singularity exec hello-world.sif /bin/echo Hello World!\n

You can run containers inside jobscripts in the same way.

Useful links:

"},{"location":"Software_Guides/Singularity/#docker-containers","title":"Docker containers","text":"

You can use Singularity to run Docker containers. Docker itself is not suitable for use on a multi-user HPC system, but Singularity can convert and run Docker containers for you.

singularity pull python-3.9.6.sif docker://python:3.9.6-slim-buster\n

In this case, singularity pull is downloading a Docker image, and also converting it into a format that Singularity uses. You then use singularity run or singularity exec on the .sif image as above.

"},{"location":"Software_Guides/Singularity/#graphical-containers-in-interactive-jobs","title":"Graphical containers in interactive jobs","text":"

If you are trying to run a graphical application from inside a container in an interactive job and it is failing with errors about not being able to open a display, you will need to:

To do the bind mount, you could add it to your $SINGULARITY_BINDPATH

export SINGULARITY_BINDPATH=/scratch/scratch/$USER,/tmpdir,$HOME/.Xauthority\n

or you can pass it in with the --bind option to singularity shell or singularity exec.

"},{"location":"Supplementary/Compiling_Your_Code/","title":"Compiling Your Code","text":""},{"location":"Supplementary/Compiling_Your_Code/#download-your-code","title":"Download your code","text":"

Use wget or curl to download the source code for the software you want to install to your account. There might be binaries available, but they often won't work on our clusters because they were compiled for other machines with other library versions available. Use tar or unzip or similar depending on archive type to uncompress your source code.

wget\u00a0https://www.example.com/program.tar.gz\ntar\u00a0-xvf\u00a0program.tar.gz\n

You won't be able to use a package manager like yum, you'll need to follow the manual installation instructions for a user-space install (not using sudo).

"},{"location":"Supplementary/Compiling_Your_Code/#set-up-your-modules","title":"Set up your modules","text":"

Before you start compiling, you need to make sure you have the right compilers, libraries and other tools available for your software. If you haven't changed anything, you will have the default modules loaded.

Check what the instructions for your software tell you about compiling it. If the website doesn't say much, the source code will hopefully have a README or INSTALL file.

You may want to use a different compiler - the default is the Intel compiler.

module avail compilers will show you all the compiler modules available. Most Open Source software tends to assume you're using GCC and OpenMPI (if it uses MPI) and is most tested with that combination, so if it doesn't specify you may want to begin there (do check what the newest modules available are):

module\u00a0unload\u00a0compilers\u00a0mpi\u00a0mkl\nmodule\u00a0load\u00a0compilers/gnu/4.9.2  \nmodule\u00a0load\u00a0mpi/openmpi/1.10.1/gnu-4.9.2\n
"},{"location":"Supplementary/Compiling_Your_Code/#available-compilers","title":"Available compilers","text":"

The following compilers are available and supported on Legion:

We currently have a limited number of licenses for the Intel compilers so only a certain number of users can use them simultaneously. This means that your compilation may fail with an error complaining about not being able to obtain a valid license. If this happens, simply wait for a few minutes and try again.

In addition to the supported tools, there are a number of tools installed on Legion which are not supported (for example the PGI compilers) which were installed to build certain supported packages. Users who use the unsupported packages do so at their own risk.

"},{"location":"Supplementary/Compiling_Your_Code/#build-systems","title":"Build systems","text":"

Most software will use some kind of build system to manage how files are compiled and linked and in what order. Here are a few common ones.

"},{"location":"Supplementary/Compiling_Your_Code/#automake-configure","title":"Automake configure","text":"

Automake will generate the Makefile for you and hopefully pick up sensible options through configuration. You can give it an install prefix to tell it where to install (or you can build it in place and not use make install at all).

./configure\u00a0--prefix=/home/username/place/you/want/to/install  \nmake  \n#\u00a0if\u00a0it\u00a0has\u00a0a\u00a0test\u00a0suite,\u00a0good\u00a0idea\u00a0to\u00a0use\u00a0it  \nmake\u00a0test\u00a0  \nmake\u00a0install\n

If it has more configuration flags, you can use ./configure --help to view them.

Usually configure will create a config.log: you can look in there to find if any tests have failed or things you think should have been picked up haven't.

"},{"location":"Supplementary/Compiling_Your_Code/#cmake","title":"CMake","text":"

CMake is another build system. It will have a CMakeFile or the instructions will ask you to use cmake or ccmake rather than make. It also generates Makefiles for you. ccmake is a terminal-based interactive interface where you can see what variables are set to and change them, then repeatedly configure until everything is correct, generate the Makefile and quit. cmake is the commandline version. The process tends to go like this:

ccmake\u00a0CMakeLists.txt  \n#\u00a0press\u00a0c\u00a0to\u00a0configure\u00a0-\u00a0will\u00a0pick\u00a0up\u00a0some\u00a0options  \n#\u00a0press\u00a0t\u00a0to\u00a0toggle\u00a0advanced\u00a0options  \n#\u00a0keep\u00a0making\u00a0changes\u00a0and\u00a0configuring\u00a0until\u00a0no\u00a0more\u00a0errors\u00a0or\u00a0changes  \n#\u00a0press\u00a0g\u00a0to\u00a0generate\u00a0and\u00a0exit  \nmake  \n#\u00a0if\u00a0it\u00a0has\u00a0a\u00a0test\u00a0suite,\u00a0good\u00a0idea\u00a0to\u00a0use\u00a0it  \nmake\u00a0test\u00a0  \nmake\u00a0install\n

If you need to rerun ccmake and reconfigure, remember to delete the CMakeCache.txt file, or you'll be wondering why your changes haven't taken. Turning on verbose Makefiles in ccmake is also useful if your code didn't compile first time - you'll be able to see what flags the compiler or linker is actually being given when it fails.

"},{"location":"Supplementary/Compiling_Your_Code/#make","title":"Make","text":"

Your code may just come with a Makefile and have no configure, in which case the generic way to compile it is as follows:

make\u00a0targetname\n

There's usually a default target, which make on its own will use. If you need to change any configuration options, you'll need to edit those sections of the Makefile (at the top, where the variables/flags are defined).

Here are some typical variables you may want to change in a Makefile.

These are what compilers/mpi wrappers to use - these are also defined by the compiler modules, so you can see what they should be. Intel would be icc, icpc, ifort, for example. If it's a program that can be compiled using MPI and only has a variable for CC, then set that to mpicc.

CC=gcc  \nCXX=g++  \nFC=gfortran  \nMPICC=mpicc  \nMPICXX=mpicxx  \nMPIF90=mpif90\n

CFLAGS and LDFLAGS are flags for the compiler and linker respectively, and there might be LIBS or INCLUDE as well. When linking a library with the name libfoo, use -lfoo.

CFLAGS=\"-I/path/to/include\"  \nLDFLAGS=\"-L/path/to/foo/lib\u00a0-L/path/to/bar/lib\"  \nLDLIBS=\"-lfoo\u00a0-lbar\"\n

Remember to make clean first if you are recompiling with new options!

"},{"location":"Supplementary/Compiling_Your_Code/#blas-and-lapack","title":"BLAS and LAPACK","text":"

BLAS and LAPACK are provided as part of MKL, OpenBLAS or ATLAS. There are several different OpenBLAS and ATLAS modules on Legion for different compilers. MKL is available in the Intel compiler module.

Your code may try to link -lblas -llapack: this isn't the right way to use BLAS and LAPACK with MKL or ATLAS (our OpenBLAS now has symlinks that allow you to do this).

"},{"location":"Supplementary/Compiling_Your_Code/#set-your-path-and-other-environment-variables","title":"Set your PATH and other environment variables","text":"

After you have installed your software, you'll need to add it to your PATH environment variable so you can run it without having to give the full path to its location.

Put this in your ~/.bashrc file so it will set this with every new session you create. Replace username with your username and point to the directory your binary was built in (frequently program/bin). This adds it to the front of your PATH, so if you install a newer version of something, it will be found before the system one.

export\u00a0PATH=/home/username/location/of/software/binary:$PATH\n

If you built a library that you'll go on to compile other software with, you probably want to also add the lib directory to your LD_LIBRARY_PATH and LIBRARY_PATH, and the include directory to CPATH (add export statements as above). This may mean your configure step will pick your library up correctly without any further effort on your part.

To make these changes to your .bashrc take effect in your current session:

source\u00a0~/.bashrc\n
"},{"location":"Supplementary/Compiling_Your_Code/#python","title":"Python","text":"

There are python2/recommended and python3/recommended bundles. These use a virtualenv and have pip set up for you. They both have numpy and scipy available.

"},{"location":"Supplementary/Compiling_Your_Code/#set-compiler-module","title":"Set compiler module","text":"

The Python versions on Myriad were built with GCC. You can run them with the default Intel compilers loaded because everything depends on the gcc-libs/4.9.2 module. When you are building your own Python packages you should have the GCC compiler module loaded however, to avoid the situation where you build a package with the Intel compiler and then try to run it with GCC, in which case it will be unable to find Intel-specific instructions.

module\u00a0unload\u00a0compilers\nmodule\u00a0load\u00a0compilers/gnu/4.9.2\n

If you get an error like this when trying to run something, recheck what compiler you used.

undefined\u00a0symbol:\u00a0__intel_sse2_strrchr\n
"},{"location":"Supplementary/Compiling_Your_Code/#install-your-own-packages-in-the-same-virtualenv","title":"Install your own packages in the same virtualenv","text":"

This will use our central virtualenv, which contains a number of packages already installed.

pip\u00a0install\u00a0--user\u00a0<python2pkg>  \npip3\u00a0install\u00a0--user\u00a0<python3pkg>\n

These will install into .python2local or .python3local directories in your home directory, respectively.

To see what is already installed, the Python-shared list shows what is installed for both Python2 and 3, while the Python2 list and Python3 list show what is only installed for one or the other. (There may also be prereqs that aren't listed explicitly - pip will tell you if something is already installed as long as you have the recommended module bundle loaded).

"},{"location":"Supplementary/Compiling_Your_Code/#use-your-own-virtualenvs","title":"Use your own virtualenvs","text":"

If you need different packages that are not compatible with the central installs, you can create a new virtualenv and only yours will be available.

virtualenv\u00a0<DIR> \nsource\u00a0<DIR>/bin/activate\n

Your bash prompt will show you that a different virtualenv is active.

"},{"location":"Supplementary/Compiling_Your_Code/#installing-via-setuppy","title":"Installing via setup.py","text":"

If you need to install using setup.py, you can use the --user flag and as long as one of the python bundles is loaded, it will install into the same .python2local or .python3local as pip and you won't need to add any new paths to your environment.

python\u00a0setup.py\u00a0install\u00a0--user\n

You can alternatively use --prefix in which case you will have to set the install prefix to somewhere in your space, and also set PYTHONPATH and PATH to include your install location. Some installs won't create the prefix directory for you, in which case create it first. This is useful if you want to keep this package entirely separate and only in your paths on demand.

export\u00a0PYTHONPATH=/home/username/your/path/lib/python2.7/site-packages:$PYTHONPATH  \n#\u00a0if\u00a0necessary,\u00a0create\u00a0install\u00a0path  \nmkdir\u00a0-p\u00a0home/username/your/path/lib/python2.7/site-packages  \npython\u00a0setup.py\u00a0install\u00a0--prefix=/home/username/your/path\n\n#\u00a0add\u00a0these\u00a0to\u00a0your\u00a0.bashrc\u00a0or\u00a0jobscript  \nexport\u00a0PYTHONPATH=/home/username/your/path/lib/python2.7/site-packages:$PYTHONPATH  \nexport\u00a0PATH=/home/username/your/path/bin:$PATH\n

Check that the PATH is where your Python executables were installed, and the PYTHONPATH is correct. It will tend to tell you at install time if you need to change or create the PYTHONPATH directory.

"},{"location":"Supplementary/Compiling_Your_Code/#python-script-executable-paths","title":"Python script executable paths","text":"

If you have an executable python script giving the location of python like this, and it fails because that python doesn't exist in that location or isn't the one that has the additional packages installed:

#!/usr/bin/python2.7\n

You should change it so it uses the first python found in your environment. 

#!/usr/bin/env\u00a0python \n
"},{"location":"Supplementary/Compiling_Your_Code/#perl","title":"Perl","text":"

Perl modules will freqently have a Makefile.PL (especially if you download the tar files from CPAN.org yourself). You can install manually as:

perl\u00a0Makefile.PL\u00a0PREFIX=/home/username/your/perl/location\nmake\nmake\u00a0install\n
"},{"location":"Supplementary/Compiling_Your_Code/#cpan","title":"CPAN","text":"

You can use CPAN to download and install modules locally for you. The first time you run the cpan command, it will create a .cpan directory for you and ask you to give it configuration settings or allow it to set them automatically.

You need to tell it where you want your install prefix to be.

If it is automatically configured, you need to edit these lines in your .cpan/CPAN/MyConfig.pm, for example if you want it to be in a lib directory in your home (change username to your own username):

'make_install_arg'\u00a0=>\u00a0q[PREFIX=/home/username/lib],\n#\u00a0other\u00a0lines\u00a0in\u00a0here  \n'makepl_arg'\u00a0=>\u00a0q[PREFIX=/home/username/lib],\n'mbuild_install_arg'\u00a0=>\u00a0q[PREFIX=/home/username/lib],\n'mbuildpl_arg'\u00a0=>\u00a0q[--install_base\u00a0/home/username/lib],\n

It will download and build modules inside .cpan and install them where you specified.

"},{"location":"Supplementary/Compiling_Your_Code/#set-perl5lib-paths","title":"Set PERL5LIB paths","text":"

If you install your own Perl or Perl modules, you will need to append them to your PERL5LIB:

export\u00a0PERL5LIB=/home/username/your/perl/location:$PERL5LIB\n

If you installed with CPAN, you may need to add several paths to this based on the layout it creates inside your nominated Perl directory.

"},{"location":"Supplementary/Compiling_Your_Code/#errors-when-using-non-default-perl-versions","title":"Errors when using non-default Perl versions","text":""},{"location":"Supplementary/Compiling_Your_Code/#warningspm","title":"warnings.pm","text":"

If you are using a version of Perl that is not the default system Perl and get strange errors when trying to run a Perl script, particularly ones about warnings.pm:

Search\u00a0pattern\u00a0not\u00a0terminated\u00a0at\u00a0/shared/ucl/apps/perl/5.20.0/lib/5.20.0/warnings.pm\u00a0line\u00a01099\n

then you need to edit the script so that instead of beginning with #!/usr/bin/perl, it begins with #!/usr/bin/env perl. Otherwise it will try to use the old system Perl libraries with your newer Perl executable, which won't work.

"},{"location":"Supplementary/Compiling_Your_Code/#libperlso-not-found","title":"libperl.so not found","text":"

You probably built perl without telling it to build the shared library too. Add -Duseshrplib to your build flags.

"},{"location":"Supplementary/Compiling_Your_Code/#r","title":"R","text":"

There are instructions on installing and using local R packages in Using your own R packages.

"},{"location":"Supplementary/Compiling_Your_Code/#compiling-with-mpi","title":"Compiling with MPI","text":"

OpenMPI and Intel MPI are available. Certain programs do not work well with one or the other, so if you are having problems try the other one. Intel MPI is based on MPICH, so if the program you are compiling mentions that, try Intel MPI first.

The Intel MPI is threadsafe; some versions of OpenMPI aren't.

Note that OpenMPI 1.8.4 had a segv bug in non-blocking collectives that is fixed in OpenMPI 1.10.1.

"},{"location":"Supplementary/Compiling_Your_Code/#enabling-openmp","title":"Enabling OpenMP","text":"

To enable OpenMP with the Intel compilers, you simply need to add -openmp to your compile line. With the GNU compilers you need to add -fopenmp.

"},{"location":"Supplementary/Compiling_Your_Code/#problems","title":"Problems","text":"

If you experience problems building your applications, please contact your local IT support in the first instance. We are available at rc-support AT ucl.ac.uk to help you if you still cannot build your app or if you need to report a problem with our software stack.

"},{"location":"Supplementary/Compiling_and_Running_Matlab_Programs/","title":"Compiling and Running Matlab Programs","text":"

Although full Matlab is available on Myriad, you can also compile Matlab programs on an external machine and then run them on Myriad using the Matlab runtime.

There are some caveats, however:

"},{"location":"Supplementary/Compiling_and_Running_Matlab_Programs/#compiling-your-program","title":"Compiling your program:","text":"

The Matlab code is must be compiled using the mcc tool; this must be initially run as mcc -setup before anything is built. The mcc tool can actually be invoked from the interpreter command prompt and executing help mcc will give you quite a lot of information about how to use the tool, along with examples.

All .m files must be built into the compiled code with the first .m referenced in the build line acting as the main entry point for the built code. It may be useful to include data files in the built code which are handled in the build line using the -a <datafile> option. Please remember to make the .m file an actual function and all other dependencies sub-functions, otherwise the compiled code will not execute.

"},{"location":"Supplementary/Compiling_and_Running_Matlab_Programs/#some-important-mcc-options","title":"Some important mcc options:","text":""},{"location":"Supplementary/Compiling_and_Running_Matlab_Programs/#some-important-runtime-options","title":"Some important runtime options:","text":"

Once the application has been built, there should be an executable named after the prefix of the .m file, generally <app name>.m, and a shell script with the name run\\_<app name>.sh - both these files need to be transferred to Myriad.

If you have been given pre-compiled code by someone else, the application may not work as the Matlab runtime version must reasonably match that of the Matlab compiler that was used to build the application. The runtime is freely distributable and can be found in the installation directory of Matlab. The runtime has a GUI install interface and it can be installed at any location in your home directory.

For more information, please read the Matlab documentation.

"},{"location":"Supplementary/Compiling_and_Running_Matlab_Programs/#job-submission-scripts","title":"Job submission scripts:","text":"

There are three things that you must take into account:

  1. The location of the Matlab compiler runtime needs to be passed to the script used to run the compiled Matlab code as the first argument.
  2. The compiler runtime needs a directory (cache) to unpack files to when it is running. By default this directory is in the home folder. Since the Matlab runs will be single node jobs, the cache location should be in the storage on the compute nodes which is stored in TMPDIR.

For example, a multi-threaded serial script should look something like:

#!/bin/bash\u00a0-l\n#\u00a0Batch\u00a0script\u00a0to\u00a0run\u00a0a\u00a0serial\u00a0job\u00a0on\u00a0Legion\u00a0under\u00a0SGE.\n\n#\u00a0Force\u00a0bash\u00a0as\u00a0the\u00a0executing\u00a0shell.\n#$\u00a0-S\u00a0/bin/bash\n\n#\u00a0Request\u00a0ten\u00a0minutes\u00a0of\u00a0wallclock\u00a0time\u00a0(format\u00a0hours:minutes:seconds).\n#$\u00a0-l\u00a0h_rt=0:10:0\n\n#\u00a0Request\u00a01\u00a0gigabyte\u00a0of\u00a0RAM\u00a0\n#$\u00a0-l\u00a0mem=1G\n\n#\u00a0Request 36 cores\n#$\u00a0-pe smp 36\n\n#\u00a0Set\u00a0the\u00a0name\u00a0of\u00a0the\u00a0job.\n#$\u00a0-N\u00a0Matlab_Job_1\n\n#\u00a0Set\u00a0the\u00a0working\u00a0directory\u00a0to\u00a0somewhere\u00a0in\u00a0your\u00a0scratch\u00a0space.\n#\u00a0For\u00a0example:\n##$\u00a0-wd\u00a0/home//Scratch\n#\u00a0Alternatively,\u00a0you can automatically use the current working directory \n#  if you launch\u00a0your\u00a0job\u00a0from\u00a0anywhere\u00a0*within\u00a0~/Scratch*\n#$\u00a0-cwd\n\n#\u00a0store\u00a0the\u00a0MATLAB\u00a0runtime\u00a0path\u00a0in\u00a0a\u00a0global\u00a0environment\u00a0variable\u00a0(MCR_HOME)\nexport\u00a0MCR_HOME=/shared/ucl/apps/Matlab/R2011a/Runtime7.15/v715/\n\n#\u00a0the\u00a0path\u00a0to\u00a0the\u00a0Matlab\u00a0cache\u00a0is\u00a0stored\u00a0in\u00a0the\u00a0global\u00a0variable\u00a0MCR_CACHE_ROOT\u00a0\nexport\u00a0MCR_CACHE_ROOT=$TMPDIR/mcr_cache\n\n#\u00a0make\u00a0sure\u00a0the\u00a0directory\u00a0in\u00a0MCR_CACHE_ROOT\u00a0exists\nmkdir\u00a0-p\u00a0$MCR_CACHE_ROOT\n\n#\u00a0Run\u00a0the\u00a0executable,\u00a0passing\u00a0the\u00a0path\u00a0stored\u00a0in\u00a0MCR_HOME\u00a0as\u00a0the\u00a0first\u00a0argument.\n#\u00a0There\u00a0is\u00a0no\u00a0need\u00a0to\u00a0pass\u00a0the\u00a0content\u00a0of\u00a0MCR_CACHE_ROOT\u00a0as\u00a0an\u00a0argument\u00a0to\u00a0the\n#\u00a0to\u00a0the\u00a0run_appname.sh\u00a0script\u00a0since\u00a0it\u00a0is\u00a0a\u00a0variable\u00a0that\u00a0the\u00a0Matlab\u00a0runtime\u00a0is\u00a0aware\u00a0of.\n./run_appname.sh\u00a0$MCR_HOME\u00a0[arguments\u00a0list]\n\n#\u00a0Preferably,\u00a0tar-up\u00a0(archive)\u00a0all\u00a0output\u00a0files\u00a0onto\u00a0the\u00a0shared\u00a0scratch\u00a0area\ntar\u00a0zcvf\u00a0$HOME/Scratch/files_from_job_${JOB_ID}.tgz\u00a0$TMPDIR\n\n#\u00a0Make\u00a0sure\u00a0you\u00a0have\u00a0given\u00a0enough\u00a0time\u00a0for\u00a0the\u00a0copy\u00a0to\u00a0complete!\n

For any queries and problem reports, please contact rc-support@ucl.ac.uk.

"},{"location":"Supplementary/Connecting_to_RDSS/","title":"Connecting to the Research Data Storage Service","text":"

The Research Data Storage Service (RDSS) is a system run by the Research Data group in the Advanced Research Computing department, and is designed to help with data storage during and after a project. Several solutions for copying data between RDSS and the central UCL research computing platforms are presented below. Sections of the example code surrounded by angle brackets (\\<>) should be replaced by the information indicated (do not keep the angle brackets in).

"},{"location":"Supplementary/Connecting_to_RDSS/#between-myriad-and-rdss","title":"Between Myriad and RDSS","text":"

If you already have an account with the Research Data Storage Service, you can transfer data directly between Legion and Research Data Storage using the Secure Copy (scp) command.

"},{"location":"Supplementary/Connecting_to_RDSS/#from-rds-to-myriad","title":"From RDS to Myriad","text":"

If you are on an RDSS login node, you can transfer data to Myriad\u2019s Scratch area at the highest rate currently possible by running the command: 

scp\u00a0data_file.tgz\u00a0myriad.rc.ucl.ac.uk:~/Scratch/\n

Or from somewhere within Myriad (including compute nodes in running jobs) running the command: 

scp\u00a0ssh.rd.ucl.ac.uk:~/data_file.tgz\u00a0~/Scratch/\n
"},{"location":"Supplementary/Connecting_to_RDSS/#from-myriad-to-rdss","title":"From Myriad to RDSS","text":"

From Myriad, send data to your project space on RDSS by running the command:

scp\u00a0data_file.tgz\u00a0ccaaxyz@ssh.rd.ucl.ac.uk:<path_to_project_space>\n

The RDSS support pages provide more information:

https://www.ucl.ac.uk/isd/live-storage-access-guide

"},{"location":"Supplementary/GPU_Clusters/","title":"GPU clusters","text":"

UCL users may be able to access the following GPU clusters.

"},{"location":"Supplementary/GPU_Clusters/#national-gpu-clusters","title":"National GPU clusters","text":"

There are two nationally-accessible EPSRC Tier 2 HPC centres with GPUs.

Access is generally managed through calls to an EPSRC Resource Allocation Panel

There may also be pump-priming/proof of concept access available.

General information about machines with external access is available at HPC-UK.

"},{"location":"Supplementary/GPU_Clusters/#csd3","title":"CSD3","text":""},{"location":"Supplementary/GPU_Clusters/#jade","title":"JADE","text":""},{"location":"Supplementary/GPU_Nodes/","title":"GPU Nodes","text":""},{"location":"Supplementary/GPU_Nodes/#node-types","title":"Node Types","text":"

You can view the hardware specifications for GPU node types in Myriad.

There are several types of GPU nodes available in Myriad.

"},{"location":"Supplementary/GPU_Nodes/#available-modules","title":"Available modules","text":"

You can see all the available CUDA modules by typing

module load beta-modules\nmodule\u00a0avail\u00a0cuda\n

The ones that become visible once you load beta-modules have been built with newer compilers.

"},{"location":"Supplementary/GPU_Nodes/#sample-cuda-code","title":"Sample CUDA code","text":"

There are samples in some CUDA install locations, e.g. 

/shared/ucl/apps/cuda/7.5.18/gnu-4.9.2/samples\n/shared/ucl/apps/cuda/8.0.61/gnu-4.9.2/samples\n

which are further documented by NVIDIA here. In general, you should look at their CUDA docs: http://docs.nvidia.com/cuda/

"},{"location":"Supplementary/GPU_Nodes/#sample-jobscripts","title":"Sample jobscripts","text":"

You can see sample jobscripts here.

Use this in your script to request up to 2 GPUs. 

#$\u00a0-l\u00a0gpu=2\n

Load GCC and the relevant CUDA module. 

module\u00a0unload\u00a0compilers\u00a0mpi\nmodule\u00a0load\u00a0compilers/gnu/4.9.2\nmodule\u00a0load\u00a0cuda/7.5.18/gnu-4.9.2\n
"},{"location":"Supplementary/GPU_Nodes/#running-the-sample-code","title":"Running the sample code","text":"

To get started, here's how you would compile one of the CUDA samples and run it in an interactive session on a GPU node.

You can compile CUDA code on the login nodes like this (which do not have GPUs) if they do not require all the CUDA libraries to be present at compile time. If they do, you'll get an error saying it cannot link the CUDA libraries, and ERROR: CUDA could not be found on your system and you will need tro do your compiling on the GPU node as well.

1. Load the cuda module 

module\u00a0unload\u00a0compilers\u00a0mpi\nmodule\u00a0load\u00a0compilers/gnu/4.9.2 \nmodule\u00a0load\u00a0cuda/7.5.18/gnu-4.9.2\n

2. Copy the samples directory to somewhere in your home (or to Scratch if you're building on the GPU node or are going to want a job to write anything in the same directory).

cp\u00a0-r\u00a0/shared/ucl/apps/cuda/7.5.18/gnu-4.9.2/NVIDIA_CUDA-7.5_Samples/\u00a0~/cuda\n

3. Choose an example: eigenvalues in this case, and build using the provided makefile - if you have a look at it you can see it is using nvcc and g++. 

cd\u00a0NVIDIA_CUDA-7.5_Samples/6_Advanced/eigenvalues/\nmake\n

4. Request an interactive job with a GPU and wait to be given access to the node. You will see your prompt change to indicate that you are on a different node than the login node once your qrsh request has been scheduled, and you can then continue. Load the cuda module on the node and run the program. 

qrsh\u00a0-l\u00a0mem=1G,h_rt=0:30:0,gpu=1\u00a0-now\u00a0no\n\n# wait for interactive job to start\n\nmodule\u00a0unload\u00a0compilers\u00a0mpi\nmodule\u00a0load\u00a0compilers/gnu/4.9.2\nmodule\u00a0load\u00a0cuda/7.5.18\ncd\u00a0~/cuda/NVIDIA_CUDA-7.5_Samples/6_Advanced/eigenvalues/\n./eigenvalues\n

5. Your output should look something like this: 

Starting\u00a0eigenvalues  \nGPU\u00a0Device\u00a00:\u00a0\"Tesla\u00a0M2070\"\u00a0with\u00a0compute\u00a0capability\u00a02.0\n\nMatrix\u00a0size:\u00a02048\u00a0x\u00a02048\u00a0  \nPrecision:\u00a00.000010  \nIterations\u00a0to\u00a0be\u00a0timed:\u00a0100  \nResult\u00a0filename:\u00a0'eigenvalues.dat'  \nGerschgorin\u00a0interval:\u00a0-2.894310\u00a0/\u00a02.923303  \nAverage\u00a0time\u00a0step\u00a01:\u00a026.739325\u00a0ms  \nAverage\u00a0time\u00a0step\u00a02,\u00a0one\u00a0intervals:\u00a09.031162\u00a0ms  \nAverage\u00a0time\u00a0step\u00a02,\u00a0mult\u00a0intervals:\u00a00.004330\u00a0ms  \nAverage\u00a0time\u00a0TOTAL:\u00a035.806992\u00a0ms  \nTest\u00a0Succeeded!\n
"},{"location":"Supplementary/GPU_Nodes/#building-your-own-code","title":"Building your own code","text":"

As above, if the code you are trying to compile needs to link against libcuda, it must also be built on a GPU node because only the GPU nodes have the correct libraries.

The NVIDIA examples don't require this, but things like Tensorflow do.

"},{"location":"Supplementary/GPU_Nodes/#tensorflow","title":"Tensorflow","text":"

Tensorflow is installed: type module avail tensorflow to see the available versions.

Modules to load for the non-MKL GPU version: 

module\u00a0unload\u00a0compilers\u00a0mpi \nmodule\u00a0load\u00a0compilers/gnu/4.9.2  \nmodule\u00a0load\u00a0python3/3.7\nmodule\u00a0load\u00a0cuda/10.0.130/gnu-4.9.2  \nmodule\u00a0load\u00a0cudnn/7.4.2.24/cuda-10.0\nmodule\u00a0load\u00a0tensorflow/2.0.0/gpu-py37\n

Modules to load the most recent version we have installed with GPU support (2.11.0):

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load python/3.9.6-gnu-10.2.0\nmodule load cuda/11.2.0/gnu-10.2.0\nmodule load cudnn/8.1.0.77/cuda-11.2\nmodule load tensorflow/2.11.0/gpu\n
"},{"location":"Supplementary/GPU_Nodes/#pytorch","title":"PyTorch","text":"

PyTorch is installed: type module avail pytorch to see the versions available.

Modules to load the most recent release we have installed (May 2022) are:

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load python3/3.9-gnu-10.2.0\nmodule load cuda/11.3.1/gnu-10.2.0\nmodule load cudnn/8.2.1.32/cuda-11.3\nmodule load pytorch/1.11.0/gpu\n

If you want the CPU only version then use:

module -f unload compilers mpi gcc-libs\nmodule load beta-modules\nmodule load gcc-libs/10.2.0\nmodule load python3/3.9-gnu-10.2.0\nmodule load pytorch/1.11.0/cpu\n
"},{"location":"Supplementary/GPU_Nodes/#using-mpi-and-gpus","title":"Using MPI and GPUs","text":"

It is possible to run MPI programs that use GPUs but only within a single node, so you can request up to 4 GPUs and 36 cores on Myriad.

"},{"location":"Supplementary/GPU_Nodes/#looking-for-more-gpus","title":"Looking for more GPUs?","text":""},{"location":"Supplementary/Glossary/","title":"Research Computing Glossary","text":"Bash A shell and scripting language, which is the default command processor on most Linux operating systems. Cluster A cluster consists of a set of computer nodes connected together over a fast local area network. A message passing protocol such as MPI allows individual nodes to work together as a single system. Core A core refers to a processing unit within a node. A node may have multiple cores which can work in parallel on a single task, operating on the same data in memory. This kind of parallelism is coordinated using the OpenMP library. Alternatively, cores may work independently on different tasks. Cores may or may not also share cache. Interconnect The interconnect is the network which is used to transfer data between nodes in a cluster. Different types of interconnect operate at different bandwidths and with different amounts of latency, which affects the suitability of a collection of nodes for jobs which use message passing (MPI). Job In the context of Batch Processing, a job refers to a computational task to be performed such as a single simulation or analysis. Job Script A job script is essentially a special kind of script used to specify the parameters of a job. Users can specify the data to input, program to use, and the computing resources required. The job script is specified when a job is submitted to SGE, which reads lines starting with #$. MPI The Message Passing Interface (MPI) system is a set of portable libraries which can be incorporated into programs in order to control parallel computation. Specifically it coordinates effort between nodes which do not share the same memory address space cf. OpenMP. Node In cluster computing, a node refers to a computational unit which is capable of operating independently of other parts of the cluster. As a minimum it consists of one (or more) processing cores, has its own memory, and runs its own operating system. OpenMP Open Multi-Processing. OpenMP supports multithreading, a process whereby a master thread generates a number of slave threads to run a task which is divided among them. OpenMP applies to processes running on shared memory platforms, i.e. jobs running on a single node. Hybrid applications may make use of both OpenMP and MPI. Process A process is a single instance of a program that is running on a computer. A single process may consist of many threads acting concurrently, and there may multiple instances of a program running as separate processes. Script A shell script enables users to list commands to be run consecutively by typing them into a text file instead of typing them out live. The first line of the script uses the shebang notation #! to designate the scripting language interpreter program to be used to interpret the commands, e.g. bash. Shebang \"Shebang\" is a common abbreviation for \"hash-bang\" \u2014 the character sequence #! \u2014 which is placed at the start of a script to specify the interpreter that should be used. When the shebang is found in the first line of a script, the program loader reads the rest of the line as the path to the required interpreter (e.g. /bin/bash is the usual path to the bash shell). The specified interpreter is then run with the path to the script passed as an argument to it. Shell A command line interpreter which provides an interface for users to type instructions to be interpreted by the operating system and display output via the monitor. Users type specific shell commands in order to run processes, e.g. ls to list directory contents. Son of Grid Engine (SGE or SoGE) The queuing system used by many cluster computing systems (including, currently, all the ones we run) to organise and schedule jobs. Once jobs are submitted to SGE, it takes care of executing them when the required resources become available. Job priority is subject to the local fair use policy. Sun Grid Engine (SGE) The original software written by Sun Microsystems that was later modified to make Son of Grid Engine (among other products, like Univa Grid Engine). Documentation may refer to Sun Grid Engine instead of Son of Grid Engine, and for most user purposes, the terms are interchangeable. Thread A thread refers to a serial computational process which can run on a single core. The number of threads generated by a parallel job may exceed the number of cores available though, in which case cores may alternate between running different threads. Threads are a software concept whereas cores are physical hardware."},{"location":"Supplementary/Hostkeys/","title":"Hostkeys","text":"

These are the current hostkey fingerprints for our clusters. The MD5 or SHA256 at the front is letting you know what type of fingerprint it is - your ssh client may not include that part in its output.

"},{"location":"Supplementary/Hostkeys/#myriad","title":"Myriad","text":"
ED25519 key fingerprint is MD5:92:6d:97:46:eb:8d:0a:4b:8a:65:cb:0d:65:79:bb:7f\nECDSA key fingerprint is SHA256:7FTryal3mIhWr9CqM3EPPeXsfezNk8Mm8HPCCAGXiIA\nRSA key fingerprint is 29:a7:45:04:83:86:ec:95:fa:25:dc:7a:f4:93:78:c1\n
"},{"location":"Supplementary/Hostkeys/#kathleen","title":"Kathleen","text":"
ED25519 key fingerprint is MD5:92:6d:97:46:eb:8d:0a:4b:8a:65:cb:0d:65:79:bb:7f\nECDSA key fingerprint is SHA256:rCKAb0yOWXK8+GClKy/pdbwrUbrGMvFkMciZLVcbaTA\nRSA key fingerprint is 5a:cf:95:a2:e4:05:8a:36:46:dc:65:0a:f2:8b:ab:e1\n
"},{"location":"Supplementary/Hostkeys/#young","title":"Young","text":"
ED25519 key fingerprint MD5 is 92:6d:97:46:eb:8d:0a:4b:8a:65:cb:0d:65:79:bb:7f\nECDSA key fingerprint is SHA256:3zwMU9C8d9rgmYJ9qDElo15NnWyF2I4xy2X/VIAmFdo\nRSA key fingerprint is 06:17:f3:f2:0c:3e:0d:df:1d:04:fb:53:dc:77:60:56\n
"},{"location":"Supplementary/Hostkeys/#thomas","title":"Thomas","text":"
ED25519 key fingerprint is MD5:92:6d:97:46:eb:8d:0a:4b:8a:65:cb:0d:65:79:bb:7f\nECDSA key fingerprint is SHA256:r48udIRDfBEIJG+jiIJFs/56ZayaKUdusFd+JQ3jsO4\nRSA key fingerprint is SHA256:AZ88UVU3BfZkSBOsMw5VKgbDi47o3dpEabPlIB9GtcM\n
"},{"location":"Supplementary/Hostkeys/#michael","title":"Michael","text":"
ED25519 key fingerprint is MD5:b3:d9:60:a8:73:62:d5:91:ef:2b:ba:1e:d3:68:7c:ec\nECDSA key fingerprint is SHA256:3PMLXp6ny0dECycvx4D7+t0sNgsSsLvSO5QUYmzkbhs\nRSA key fingerprint is 85:31:4b:cf:1a:ec:64:e4:b2:98:28:4a:46:b2:c1:90\n
"},{"location":"Supplementary/Installing_PGI/","title":"Installing PGI","text":""},{"location":"Supplementary/Installing_PGI/#pgi-compiler-suite-installation-at-ucl","title":"PGI Compiler Suite Installation at UCL","text":"

UCL has two floating licences for PGI Fortran/C/C++ Server for Linux, purchased primarily for building Gaussian 03 and Gaussian 09 on UCL computers. To install follow the procedure below. If you are installing on a system outside the Institutional Firewall, please connect to the UCL VPN service first.

tar\u00a0xvzf\u00a0pgilinux-2013-139.tar.gz\n
./install`\n
export\u00a0LM_LICENSE_FILE=27000@lic-pgi.ucl.ac.uk\n\nexport\u00a0LM_LICENSE_FILE=$LM_LICENSE_FILE:27000@lic-pgi.ucl.ac.uk\n

The PGI compilers should now be installed and working.

"},{"location":"Supplementary/NVIDIA_Containers/","title":"Using NVIDIA Grid Cloud Containers","text":"

NVIDIA's NGC Container Registry stores a lot of containers1 with various applications pre-configured to be run with GPUs. Each container can be set up completely differently, so you'll need to read the instructions in the registry to work out how to use it.

Many of the instructions for using these containers assume you are using the Docker toolset to run the container, but this is not installed on our services2. The Singularity container runtime can be used instead for most workloads, but there are some limitations, that mostly affect containers that try to run web services.

In general, if a container's instructions do not have specific instructions for using Singularity, and would tell you to run, for example:

docker run --gpus all nvcr.io/some/container\n

You should instead use:

singularity run --nv https://nvcr.io/some/container\n

For jobs using MPI, this is more complicated, because the MPI traffic has to be passed between the container and the operating system underneath. Containers built to use MPI will usually contain instructions for using Singularity, however.

"},{"location":"Supplementary/NVIDIA_Containers/#worked-example-namd-3","title":"Worked Example: NAMD 3","text":"

The NAMD authors publish a NAMD container on the NGC Container Registry, and we should be able to download this and run it on our cluster inside a batch job.

The page about how to use the container is here: https://catalog.ngc.nvidia.com/orgs/hpc/containers/namd

Following it through, you can download the benchmark example to check later whether your container works:

mkdir ngc_namd_experiment\ncd ngc_namd_experiment\nwget -O - https://gitlab.com/NVHPC/ngc-examples/raw/master/namd/3.0/get_apoa1.sh | bash\n

There are a couple of typos in the instructions: you'll need to use the tag 3.0-alpha3-singlenode instead of 3.0_alpha3-singlenode:

export NAMD_TAG=\"3.0-alpha3-singlenode\"\n
"},{"location":"Supplementary/NVIDIA_Containers/#creating-the-container-image","title":"Creating the Container Image","text":"

Before you use Singularity to create the container image, you should load the Singularity module to set up some directories where things are stored. Not doing this can cause you problems, because the default places often do not have space to store the large files needed.

module load singularity-env\n

Once you've done that, you can download the container's files and build them into the usable container:

singularity build ${NAMD_TAG}.sif docker://nvcr.io/hpc/namd:${NAMD_TAG}\n

This can take a while: Singularity has to download quite a few file collections and assemble them into a single usable set. You may see some of the following warnings:

WARNING: 'nodev' mount option set on /lustre, it could be a source of failure during build process\n2022/02/03 14:06:28  warn xattr{var/log/apt/term.log} ignoring ENOTSUP on setxattr \"user.rootlesscontainers\"\n2022/02/03 14:06:28  warn xattr{/home/uccaiki/Scratch/.singularity/tmp/rootfs-5ac43e37-84fa-11ec-8784-0894ef553d4e/var/log/apt/term.log} destination filesystem does not support xattrs, further warnings will be suppressed\n

These indicate that various capabilities are not available because of how we're building the container. For HPC use, they don't present a problem, but they could be problematic if you were building a web server into a container.

When Singularity has finished, you should see the following message:

INFO:    Creating SIF file...\nINFO:    Build complete: 3.0-alpha3-singlenode.sif\n

This file is the container image, which contains the files needed to run NAMD. You can see what NAMD gets when running \"inside\" the container, by running ls with it:

singularity exec 3.0-alpha3-singlenode.sif ls /\n

gives:

WARNING: Bind mount '/home/uccaiki => /home/uccaiki' overlaps container CWD /home/uccaiki/ngc_namd_experiment, may not be available\nbin  boot  dev  environment  etc  home  host_pwd  lib  lib64  lustre  media  mnt  opt  proc  root  run  sbin  scratch  singularity  srv  sys  tmp  tmpdir  usr  var\n

The warning you get is telling you that your current working directory overlaps with a directory being \"bound\" into the container. Binding brings a directory into the container's view of the filesystem, so that, for example, programs can still access your home directory as usual. In this case it's not a problem, because it's warning you that your home directory is being bound into the container in the same place it would usually be, and that means the same files are visible.

By default, the clusters have Singularity configured to bind your home and Scratch directories into the container, as well as the per-job temporary storage allocated to jobs under $TMPDIR.

The NAMD instructions make an alternative suggestion when setting up this environment variable to use to run Singularity, binding your data directory into a fixed place in the container:

SINGULARITY=\"$(which singularity) exec --nv -B $(pwd):/host_pwd ${NAMD_TAG}.sif\"\n

The option -B $(pwd):/host_pwd handles this, binding wherever you run the command from to the fixed location /host_pwd inside the container.

So, for example, if you run:

ls\n$SINGULARITY ls /host_pwd\n

In both cases, you should see the same files, because you're looking at the same underlying directory.

"},{"location":"Supplementary/NVIDIA_Containers/#running-on-a-single-node","title":"Running on a Single Node","text":"

At this point you're ready to run NAMD inside the container, but you need a job script to submit to the scheduler which can set up the number of cores and GPUs correctly.

#!/bin/bash -l\n\n# Start with our resource requirements:\n#  1 hour's maximum runtime\n#$ -l h_rt=1:00:00\n\n# 2 GPUs\n#$ -l gpu=2\n\n# 36 processor cores\n#$ -pe smp 36\n\n# Start with current working directory the same as where we submitted the job from\n#$ -cwd\n\n# Make sure Singularity looks for our stored container data in the right places\nmodule load singularity-env\n\n# Set the variables we need for this example to run\nNAMD_TAG=\"3.0-alpha3-singlenode\"\nSINGULARITY=\"$(which singularity) exec --nv -B $(pwd):/host_pwd ${NAMD_TAG}.sif\"\n\n# This is where the benchmark's data ends up inside the container\nINPUT=\"/host_pwd/apoa1/apoa1_nve_cuda.namd\"\n\n# Show us some info we can refer to later\nprintf \"Running NAMD using:\\n Cores: %d\\n GPUs: %d\\n Container image: %s\\nWorking directory: %s\\n Input: %s\\n\" \\\n\"$NSLOTS\" \\\n\"$GPU\" \\\n\"${NAMD_TAG}.sif\" \\\n\"$(pwd)\" \\\n\"$INPUT\"\n\n# Run NAMD\n\"$SINGULARITY\" namd3  +ppn \"$NSLOTS\" +setcpuaffinity +idlepoll \"$INPUT\"\n

Copy this into a file, and submit it to the queue, e.g.:

qsub ngc_namd_experiment.sh\n

This should take about 5 minutes to run on a 36-core, 2-GPU node.

  1. A container is a way of bundling up a collection of files and instructions to run as a kind of \"altered view\" of the computer's files and systems.\u00a0\u21a9

  2. This is for a variety of reasons, but primarily that Docker presents a much larger range of security and misuse risks when used by inexperienced or hostile users. Singularity represents a much better fit for common HPC workloads and use-cases.\u00a0\u21a9

"},{"location":"Supplementary/Points_of_Contact/","title":"MMM Points of Contact","text":""},{"location":"Supplementary/Points_of_Contact/#user-management-tools","title":"User management tools","text":"

This page contains tools and information for the nominated Points of Contact.

Other system-specific information is at Thomas, Michael or Young.

These commands can all be run as thomas-command or michael-command or young-command: they run the same thing and the different names are for convenience.

"},{"location":"Supplementary/Points_of_Contact/#displaying-user-information","title":"Displaying user information","text":"

thomas-show, michael-show or young-show is a tool that enables you to find a lot of information about users. Access to the database is given to points of contact individually, contact rc-support@ucl.ac.uk if you try to use this and get an access denied.

At the top level, --user shows all information for one user, in multiple tables. --contacts shows all points of contact - useful for getting the IDs, and --institutes is the same. --allusers will show everyone's basic info. --getmmm will show the most recently used mmm username.

thomas-show\u00a0-h  \nusage:\u00a0thomas-show\u00a0[-h]\u00a0[--user\u00a0username]\u00a0[--contacts]\u00a0[--institutes]  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0[--allusers]\u00a0[--getmmm]  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0{recentusers,getusers,whois}\u00a0...\n\nShow\u00a0data\u00a0from\u00a0the\u00a0Thomas\u00a0database.\u00a0Use\u00a0[positional\u00a0argument\u00a0-h]\u00a0for\u00a0more  \nhelp.\n\npositional\u00a0arguments:  \n\u00a0{recentusers,getusers,whois}  \n\u00a0\u00a0\u00a0recentusers\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0the\u00a0n\u00a0newest\u00a0users\u00a0(5\u00a0by\u00a0default)  \n\u00a0\u00a0\u00a0getusers\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0all\u00a0users\u00a0with\u00a0this\u00a0project,\u00a0institute,\u00a0contact  \n\u00a0\u00a0\u00a0whois\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Search\u00a0for\u00a0users\u00a0matching\u00a0the\u00a0given\u00a0requirements\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit  \n\u00a0--user\u00a0username\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0all\u00a0current\u00a0info\u00a0for\u00a0this\u00a0user  \n\u00a0--contacts\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0all\u00a0allowed\u00a0values\u00a0for\u00a0contact  \n\u00a0--institutes\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0all\u00a0allowed\u00a0values\u00a0for\u00a0institute  \n\u00a0--allusers\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0all\u00a0current\u00a0users  \n\u00a0--getmmm\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0the\u00a0highest\u00a0mmm\u00a0username\u00a0used\n
"},{"location":"Supplementary/Points_of_Contact/#show-recent-users","title":"Show recent users","text":"

thomas-show recentusers shows you the most recently-added N users, default 5.

thomas-show\u00a0recentusers\u00a0-h  \nusage:\u00a0thomas-show\u00a0recentusers\u00a0[-h]\u00a0[-n\u00a0N]\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit  \n\u00a0-n\u00a0N\n
"},{"location":"Supplementary/Points_of_Contact/#show-users-with-a-given-project-institute-contact","title":"Show users with a given project, institute, contact","text":"

thomas-show getusers will search for exact matches to the given project, institute, contact combination.

thomas-show\u00a0getusers\u00a0-h  \nusage:\u00a0thomas-show\u00a0getusers\u00a0[-h]\u00a0[-p\u00a0PROJECT]\u00a0[-i\u00a0INST_ID]\u00a0[-c\u00a0POC_ID]\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit  \n\u00a0-p\u00a0PROJECT,\u00a0--project\u00a0PROJECT  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Project\u00a0name  \n\u00a0-i\u00a0INST_ID,\u00a0--institute\u00a0INST_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Institute\u00a0ID  \n\u00a0-c\u00a0POC_ID,\u00a0--contact\u00a0POC_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Point\u00a0of\u00a0Contact\u00a0ID\n
"},{"location":"Supplementary/Points_of_Contact/#search-for-users-based-on-partial-information","title":"Search for users based on partial information","text":"

thomas-show whois can be used to search for partial matches to username, name, email fragments, including all of those in combination.

thomas-show\u00a0whois\u00a0-h  \nusage:\u00a0thomas-show\u00a0whois\u00a0[-h]\u00a0[-u\u00a0USERNAME]\u00a0[-e\u00a0EMAIL]\u00a0[-n\u00a0GIVEN_NAME]  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0[-s\u00a0SURNAME]\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit  \n\u00a0-u\u00a0USERNAME,\u00a0--user\u00a0USERNAME  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0UCL\u00a0username\u00a0of\u00a0user\u00a0contains  \n\u00a0-e\u00a0EMAIL,\u00a0--email\u00a0EMAIL  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Email\u00a0address\u00a0of\u00a0user\u00a0contains  \n\u00a0-n\u00a0GIVEN_NAME,\u00a0--name\u00a0GIVEN_NAME  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Given\u00a0name\u00a0of\u00a0user\u00a0contains  \n\u00a0-s\u00a0SURNAME,\u00a0--surname\u00a0SURNAME  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Surname\u00a0of\u00a0user\u00a0contains\n
"},{"location":"Supplementary/Points_of_Contact/#adding-user-information-and-new-projects","title":"Adding user information and new projects","text":"

thomas-add will add information to the database. Access to the database is given to points of contact individually, contact rc-support@ucl.ac.uk if you try to use this and get an access denied.

Please note that all options have a --debug flag that will allow you to see the query generated without committing the changes to the database - double-check that the information you are adding is correct.

thomas-add\u00a0-h  \nusage:\u00a0thomas-add\u00a0[-h]\u00a0{user,project,projectuser,poc,institute}\u00a0...\n\nAdd\u00a0data\u00a0to\u00a0the\u00a0Thomas\u00a0database.\u00a0Use\u00a0[positional\u00a0argument\u00a0-h]\u00a0for\u00a0more\u00a0help.\n\npositional\u00a0arguments:  \n\u00a0{user,project,projectuser,poc,institute}  \n   csv                 Add all users from the provided CSV file\n\u00a0\u00a0\u00a0user\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Adding\u00a0a\u00a0new\u00a0user\u00a0with\u00a0their\u00a0initial\u00a0project  \n\u00a0\u00a0\u00a0project\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Adding\u00a0a\u00a0new\u00a0project  \n\u00a0\u00a0\u00a0projectuser\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Adding\u00a0a\u00a0new\u00a0user-project-contact\u00a0relationship  \n\u00a0\u00a0\u00a0poc\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Adding\u00a0a\u00a0new\u00a0Point\u00a0of\u00a0Contact  \n\u00a0\u00a0\u00a0institute\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Adding\u00a0a\u00a0new\u00a0institute/consortium\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit\n
"},{"location":"Supplementary/Points_of_Contact/#add-a-new-user","title":"Add a new user","text":"

thomas-add user allows you to add a new user, with their initial project and point of contact. As of 27 June 2022 this now goes ahead and creates their account automatically within 10 minutes - first prompting you that the information you have entered is correct. You do not need to email us separately about creating accounts unless something has gone wrong. The user's initial project must already exist (create with thomas-add project first).

The user will be allocated the next free mmmxxxx username - you should only specify username yourself if they are an existing UCL user, or on Young if they previously had a Thomas or Michael account you should give them the same username. If they already have an account on this cluster with a different institution, just add them as a projectuser instead using their existing username.

You can get your poc_id by looking at thomas-show --contacts.

thomas-add\u00a0user\u00a0-h  \nusage:\u00a0thomas-add\u00a0user\u00a0[-h]\u00a0-u\u00a0USERNAME\u00a0-n\u00a0GIVEN_NAME\u00a0[-s\u00a0SURNAME]\u00a0-e  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0EMAIL_ADDRESS\u00a0-k\u00a0\"SSH_KEY\"\u00a0-p\u00a0PROJECT_ID\u00a0-c\u00a0POC_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0[--debug]\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit  \n\u00a0-u\u00a0USERNAME,\u00a0--user\u00a0USERNAME  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0UCL\u00a0username\u00a0of\u00a0user  \n\u00a0-n\u00a0GIVEN_NAME,\u00a0--name\u00a0GIVEN_NAME  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Given\u00a0name\u00a0of\u00a0user  \n\u00a0-s\u00a0SURNAME,\u00a0--surname\u00a0SURNAME  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Surname\u00a0of\u00a0user\u00a0(optional)  \n\u00a0-e\u00a0EMAIL_ADDRESS,\u00a0--email\u00a0EMAIL_ADDRESS  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Institutional\u00a0email\u00a0address\u00a0of\u00a0user  \n\u00a0-k\u00a0\"SSH_KEY\",\u00a0--key\u00a0\"SSH_KEY\"  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0User's\u00a0public\u00a0ssh\u00a0key\u00a0(quotes\u00a0necessary)  \n\u00a0-p\u00a0PROJECT_ID,\u00a0--project\u00a0PROJECT_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Initial\u00a0project\u00a0the\u00a0user\u00a0belongs\u00a0to  \n\u00a0-c\u00a0POC_ID,\u00a0--contact\u00a0POC_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Short\u00a0ID\u00a0of\u00a0the\u00a0user's\u00a0Point\u00a0of\u00a0Contact  \n --noconfirm           Don't ask for confirmation on user account creation\n\u00a0--verbose\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0SQL\u00a0queries\u00a0that\u00a0are\u00a0being\u00a0submitted  \n\u00a0--debug\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0SQL\u00a0query\u00a0submitted\u00a0without\u00a0committing\u00a0the\u00a0change\n
"},{"location":"Supplementary/Points_of_Contact/#ssh-key-formats","title":"SSH key formats","text":"

It will verify the provided ssh key by default. Note that it has to be in the form ssh-xxx keystartshere. If someone has sent in a key which has line breaks and header items, make it into this format by adding the key type to the start and removing the line breaks from the key body.

This key:

----\u00a0BEGIN\u00a0SSH2\u00a0PUBLIC\u00a0KEY\u00a0----  \nComment:\u00a0\"comment\u00a0goes\u00a0here\"  \nAAAAB3NzaC1yc2EAAAABJQAAAQEAlLhFLr/4LGC3cM1xgRZVxfQ7JgoSvnVXly0K  \n7MNufZbUSUkKtVnBXAOIjtOYe7EPndyT/SAq1s9RGZ63qsaVc/05diLrgL0E0gW+  \n9VptTmiUh7OSsXkoKQn1RiACfH7sbKi6H373bmB5/TyXNZ5C5KVmdXxO+laT8IdW  \n7JdD/gwrBra9M9vAMfcxNYVCBcPQRhJ7vOeDZ+e30qapH4R/mfEyKorYxrvQerJW  \nOeLKjOH4rSnAAOLcEqPmJhkLL8k6nQAAK3P/E1PeOaB2xD7NNPqfIsjhAJLZ+2wV  \n3eUZATx9vnmVF0YafOjvzcoK2GqUrhNAvi7k0f+ihh8twkfthj==  \n----\u00a0END\u00a0SSH2\u00a0PUBLIC\u00a0KEY\u00a0----\n

should be converted into

ssh-rsa\u00a0AAAAB3NzaC1yc2EAAAABJQAAAQEAlLhFLr/4LGC3cM1xgRZVxfQ7JgoSvnVXly0K7MNufZbUSUkKtVnBXAOIjtOYe7EPndyT/SAq1s9RGZ63qsaVc/05diLrgL0E0gW+9VptTmiUh7OSsXkoKQn1RiACfH7sbKi6H373bmB5/TyXNZ5C5KVmdXxO+laT8IdW7JdD/gwrBra9M9vAMfcxNYVCBcPQRhJ7vOeDZ+e30qapH4R/mfEyKorYxrvQerJWOeLKjOH4rSnAAOLcEqPmJhkLL8k6nQAAK3P/E1PeOaB2xD7NNPqfIsjhAJLZ+2wV3eUZATx9vnmVF0YafOjvzcoK2GqUrhNAvi7k0f+ihh8twkfthj==\n

Other types of keys (ed25519 etc) will say what they are in the first line, and you should change the ssh-rsa appropriately. The guide linked at Creating an ssh key in Windows also shows where users can get the second format out of PuTTY.

"},{"location":"Supplementary/Points_of_Contact/#add-new-users-in-bulk-from-a-csv-file","title":"Add new users in bulk from a CSV file","text":"

young-add csv allows you to add users in bulk using a CSV file of specific format and headers. As of 27 June 2022 the accounts will be all created and activated automatically within 10 minutes.

The CSV is comma-separated with a header line of 

email,given_name,surname,username,project_ID,ssh_key\n

You can leave username empty for it to allocate them a new username, but if they have an existing mmm username you should fill it in. It may be useful to show users with a given institute on Thomas if you are migrating users from one service to another.

You can download a CSV template here. Replace the example data.

young-add csv will try to automatically get your Point of Contact ID based on your username. If it can't, or if you have more than one, it will give you a list to choose from. (All users in one CSV upload will be added using the same Point of Contact ID).

It will prompt you for confirmation on each user account creation unless you give the --noconfirm option.

The project you are adding the user to must already exist.

The SSH key must be formatted as shown in SSH key formats.

If you check your CSV file on the cluster with cat -v and it shows that it is beginning with M-oM-;M-? and ending with ^M you probably need to run dos2unix on it first.

"},{"location":"Supplementary/Points_of_Contact/#add-a-new-project","title":"Add a new project","text":"

thomas-add project will create a new project, associated with an institution. It will not show in Gold until it also has a user in it.

A project ID should begin with your institute ID, followed by an underscore and a project name.

thomas-add\u00a0project\u00a0-h  \nusage:\u00a0thomas-add\u00a0project\u00a0[-h]\u00a0-p\u00a0PROJECT_ID\u00a0-i\u00a0INST_ID\u00a0[--debug]\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit  \n\u00a0-p\u00a0PROJECT_ID,\u00a0--project\u00a0PROJECT_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0A\u00a0new\u00a0unique\u00a0project\u00a0ID  \n\u00a0-i\u00a0INST_ID,\u00a0--institute\u00a0INST_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Institute\u00a0ID\u00a0this\u00a0project\u00a0belongs\u00a0to  \n\u00a0--debug\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0SQL\u00a0query\u00a0submitted\u00a0without\u00a0committing\u00a0the\u00a0change\n
"},{"location":"Supplementary/Points_of_Contact/#add-a-new-projectuser-pairing","title":"Add a new project/user pairing","text":"

thomas-add projectuser will add an existing user to an existing project. Creating a new user for an existing project also creates this relationship. After a new project-user relationship is added, a cron job will pick that up within 15 minutes and create that project for that user in Gold, with no allocation.

thomas-add\u00a0projectuser\u00a0-h  \nusage:\u00a0thomas-add\u00a0projectuser\u00a0[-h]\u00a0-u\u00a0USERNAME\u00a0-p\u00a0PROJECT_ID\u00a0-c\u00a0POC_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0[--debug]\n\noptional\u00a0arguments:  \n\u00a0-h,\u00a0--help\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0show\u00a0this\u00a0help\u00a0message\u00a0and\u00a0exit  \n\u00a0-u\u00a0USERNAME,\u00a0--user\u00a0USERNAME  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0An\u00a0existing\u00a0UCL\u00a0username  \n\u00a0-p\u00a0PROJECT_ID,\u00a0--project\u00a0PROJECT_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0An\u00a0existing\u00a0project\u00a0ID  \n\u00a0-c\u00a0POC_ID,\u00a0--contact\u00a0POC_ID  \n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0An\u00a0existing\u00a0Point\u00a0of\u00a0Contact\u00a0ID  \n\u00a0--debug\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Show\u00a0SQL\u00a0query\u00a0submitted\u00a0without\u00a0committing\u00a0the\u00a0change\n
"},{"location":"Supplementary/Points_of_Contact/#deactivating-information","title":"Deactivating information","text":"

This tool is only partly functional at present. It allows you to deactivate (not delete) some entities that may no longer exist or may have been created in error.

"},{"location":"Supplementary/Points_of_Contact/#deactivate-a-projectuser","title":"Deactivate a projectuser","text":"

Use this when the user should no longer be a member of the given project. It does not deactivate the user account, just their membership in this project. You can confirm the change by looking at young-show --user - it will say 'deactivated' rather than 'active' next to their listing for this project.

young-deactivate projectuser -h\nusage: thomas_deactivate.py projectuser [-h] -u USERNAME -p PROJECT [--debug]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -u USERNAME, --user USERNAME\n                        An existing username\n  -p PROJECT, --project PROJECT\n                        An existing project ID\n  --debug               Show SQL query submitted without committing the change\n
"},{"location":"Supplementary/Points_of_Contact/#gold-resource-allocation","title":"Gold resource allocation","text":"

We are currently using Gold to manage allocations. Thomas and Michael share one Gold database, so all the projects exist on both, but they are only active on the correct cluster. Young has its own database.

"},{"location":"Supplementary/Points_of_Contact/#reporting-from-gold","title":"Reporting from Gold","text":"

There are wrapper scripts for a number of Gold commands (these exist in the userscripts module, loaded by default).

These are all set to report in cpu-hours with the -h flag, as that is our main unit. If you wish to change anything about the wrappers, they live in /shared/ucl/apps/cluster-scripts/ so you can take a copy and add your preferred options.

They all have a --man option to see the man pages for that command.

Here are some basic useful options and what they do. They can all be given more options for more specific searches.

gusage\u00a0-p\u00a0project_name\u00a0[-s\u00a0start_time]\u00a0\u00a0#\u00a0Show\u00a0the\u00a0Gold\u00a0usage\u00a0per\u00a0user\u00a0in\u00a0this\u00a0project,\u00a0in\u00a0the\u00a0given\u00a0timeframe\u00a0if\u00a0specified.  \ngbalance\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0#\u00a0Show\u00a0the\u00a0balance\u00a0for\u00a0every\u00a0project,\u00a0split\u00a0into\u00a0total,\u00a0reserved\u00a0and\u00a0available.  \nglsuser\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0#\u00a0Shows\u00a0all\u00a0the\u00a0users\u00a0in\u00a0Gold.  \nglsproject\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0#\u00a0Shows\u00a0all\u00a0the\u00a0projects\u00a0and\u00a0which\u00a0users\u00a0are\u00a0in\u00a0them.  \nglsres\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0#\u00a0Show\u00a0all\u00a0the\u00a0current\u00a0reservatioms,\u00a0inc\u00a0user\u00a0and\u00a0project.\u00a0The\u00a0Name\u00a0column\u00a0is\u00a0the\u00a0SGE\u00a0job\u00a0ID.  \ngstatement\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0#\u00a0Produce\u00a0a\u00a0reporting\u00a0statement\u00a0showing\u00a0beginning\u00a0and\u00a0end\u00a0balances,\u00a0credits\u00a0and\u00a0debits.\n\n#\u00a0Less\u00a0useful\u00a0commands  \nglstxn\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0#\u00a0Show\u00a0all\u00a0Gold\u00a0transactions.\u00a0Filter\u00a0or\u00a0it\u00a0will\u00a0take\u00a0forever\u00a0to\u00a0run.  \nglsalloc\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0#\u00a0Show\u00a0all\u00a0the\u00a0allocations.\n

These can be run by any user. The date format is YYYY-MM-DD.

Eg. gstatement -p PROJECT -s 2017-08-01 will show all credits and debits for the given project since the given date, saying which user and job ID each charge was associated with.

"},{"location":"Supplementary/Points_of_Contact/#transferring-gold","title":"Transferring Gold","text":"

As the point of contact, you can transfer Gold from your allocation account into other project accounts. As before, we've put -h in the wrapper so it is always working in cpu-hours.

gtransfer\u00a0--fromProject\u00a0xxx_allocation\u00a0--toProject\u00a0xxx_subproject\u00a0cpu_hours\n

You can also transfer in the opposite direction, from the subproject back into your allocation account.

Note that you are able to transfer your allocation into another institute's projects, but you cannot transfer it back again - only the other institute's point of contact (or rc-support) can give it back, so be careful which project you specify.

"},{"location":"Supplementary/Points_of_Contact/#when-two-allocations-are-active","title":"When two allocations are active","text":"

There is now an overlap period of a week when two allocations can be active. By default, gtransfer will transfer from active allocations in the order of earliest expiring first. To transfer from the new allocation only, you need to specify the allocation id.

gtransfer\u00a0-i\u00a0allocation_ID\u00a0--fromProject\u00a0xxx_allocation\u00a0--toProject\u00a0xxx_subproject\u00a0cpu_hours\n

glsalloc -p xxx_allocation shows you all allocations that ever existed for your institute, and the first column is the id.

Id\u00a0\u00a0Account\u00a0Projects\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0StartTime\u00a0\u00a0EndTime\u00a0\u00a0\u00a0\u00a0Amount\u00a0\u00a0\u00a0\u00a0\u00a0Deposited\u00a0\u00a0Description\u00a0\u00a0\u00a0\u00a0  \n---\u00a0-------\u00a0---------------------\u00a0----------\u00a0----------\u00a0----------\u00a0----------\u00a0--------------\u00a0  \n87\u00a0\u00a038\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0UKCP_allocation\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a02017-08-07\u00a02017-11-05\u00a0212800.00\u00a03712800.00  \n97\u00a0\u00a038\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0UKCP_allocation\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a02017-10-30\u00a02018-02-04\u00a03712800.00\u00a03712800.00\n
"},{"location":"Supplementary/Troubleshooting/","title":"Troubleshooting","text":"

This page lists some common problems encountered by users, with methods to investigate or solve them.

"},{"location":"Supplementary/Troubleshooting/#why-is-my-job-in-eqw-status","title":"Why is my job in Eqw status?","text":"

If your job goes straight into Eqw state, there was an error in your jobscript that meant your job couldn't be started. The standard qstat job information command will give you a truncated version of the error:

qstat\u00a0-j\u00a0<job_ID>\n

To see the full error instead:

qexplain\u00a0<job_ID>\n

The qexplain script is part of our userscripts set -- if you try to use it and get an error that it doesn't exist, load the userscripts module:

module load userscripts\n

The most common reason jobs go into this error state is that a file or directory your job is trying to use doesn't exist. Creating it after the job is in the Eqw state won't make the job run: it'll still have to be deleted and re-submitted.

"},{"location":"Supplementary/Troubleshooting/#binbash-invalid-option-error","title":"\"/bin/bash: invalid option\" error","text":"

This is a sign that your jobscript is a DOS-formatted text file and not a Unix one - the line break characters are different. Type dos2unix <yourscriptname> in your terminal to convert it.

Sometimes the offending characters will be visible in the error. You can see here it's trying to parse ^M as an option.

"},{"location":"Supplementary/Troubleshooting/#i-think-i-deleted-my-scratch-space-how-do-i-restore-it","title":"I think I deleted my Scratch space, how do I restore it?","text":"

You may have accidentally deleted or replaced the link to your Scratch space. Do an ls -al in your home - if set up correctly, it should look like this:

lrwxrwxrwx\u00a0\u00a0\u00a01\u00a0username\u00a0group\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a024\u00a0Apr\u00a014\u00a0\u00a02022\u00a0Scratch\u00a0->\u00a0/scratch/scratch/username\n

where username is your UCL user ID and group is your primary group.

If this link is not present, you can recreate it with

ln\u00a0-s\u00a0/scratch/scratch/$(whoami)\u00a0Scratch\n

If you have actually deleted the files stored in your Scratch space, there is unfortunately no way to restore them.

"},{"location":"Supplementary/Troubleshooting/#which-mkl-library-files-should-i-use-to-build-my-application","title":"Which MKL library files should I use to build my application?","text":"

Depending on which whether you wish to use BLAS/LAPACK/ScaLAPACK/etc... there is a specific set of libraries that you need to pass to your compilation command line. Fortunately, Intel have released a tool that allows you to determine which libraries to link and in which order for a number of compilers and operating systems:

http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/

"},{"location":"Supplementary/Troubleshooting/#unable-to-run-job-jsv-stderr-perl-warning-setting-locale-failed","title":"Unable to run job: JSV stderr: perl: warning: Setting locale failed.","text":"

This error is generally because your SSH client is passing LANG through as part of the SSH command, and is passing something that conflicts with what Myriad has it set to. You may be more likely to come across this with newer versions of macOS - if your client is different, have a look for an equivalent option.

In Mac OS X Terminal, click Settings and under International untick the box that says \"Set locale environment variables on startup\".

Per session, you can try LANG=C ssh userid@myriad.rc.ucl.ac.uk

"},{"location":"Supplementary/Troubleshooting/#what-can-i-do-to-minimise-the-time-i-need-to-wait-for-my-jobs-to-run","title":"What can I do to minimise the time I need to wait for my job(s) to run?","text":"
  1. Minimise the amount of wall clock time you request.
  2. Use job arrays instead of submitting large numbers of jobs (see our job script examples).
  3. Plan your work so that you can do other things while your jobs are being scheduled.
"},{"location":"Supplementary/Troubleshooting/#what-is-my-project-code-short-string-project-id","title":"What is my project code (short string) / project ID?","text":"

Prior to July 2014, every user had a project code. Now all users belong to the default project \"AllUsers\" and no longer have to specify this. If you see older job script examples mentioning a project ID, you can delete that section. Only projects with access to paid or specialised resources need to give a project code in order to use those resources. If you do not know yours, contact rc-support.

"},{"location":"Supplementary/X-Forwarding/","title":"X-Forwarding","text":"

X is a system and protocol that lets remote computers push interactive windows to your local computer over a network. We use a method known as X-Forwarding, together with an SSH client, to direct the network messages for X over the same connection you use for the command-line.

The setup steps for getting this working on Windows, Linux, and macOS are each different, and we've put them below.

"},{"location":"Supplementary/X-Forwarding/#windows","title":"Windows","text":"

Windows doesn't natively have the ability to receive X windows, so you need to install an X server separately.

There are a few choices; UCL has a site-wide license (covering UCL-owned and personal computers) for one called Exceed, which is pretty reliable and seems to handle 3D content well, but there's also Xming, which is free and open-source if that's a concern.

Exceed is installed on all UCL's centrally-managed Windows computers.

"},{"location":"Supplementary/X-Forwarding/#installing-exceed","title":"Installing Exceed","text":"

Exceed comes in two parts: the main package, and the add-on that handles certain types of rendered 3D content.

You can download both parts from the Exceed page in the UCL Software Database. First, install the main package, then the 3D package.

"},{"location":"Supplementary/X-Forwarding/#using-putty-with-exceed","title":"Using PuTTY with Exceed","text":"

When you run the Exceed program, it will automatically wait in the background until you run a remote application, so you only have to configure PuTTY to pass the data through.

To do this, first fill in the details for your PuTTY connection as normal with the hostname and any other details you'd normally use to connect, and then, in the pane labelled \"Category\" on the left, click the \"+\" next to \"SSH\" and then select \"X11\".

The main pane should change to one labelled \"Options controlling SSH X11 forwarding\". Check the box labelled \"Enable X11 Forwarding\".

You can now click \"Open\" to start the connection, or you can return to the \"Session\" options to save this setup.

"},{"location":"Supplementary/X-Forwarding/#macos","title":"macOS","text":"

Like Windows, macOS doesn't come with an X server to receive X windows. The most commonly used X server for macOS is XQuartz. If you download and install that, you can follow the Linux instructions below. When you connect with X-Forwarding enabled, the XQuartz server program should start automatically, ready to present remote windows.

"},{"location":"Supplementary/X-Forwarding/#linux","title":"Linux","text":"

Almost all Linux versions that have a graphical desktop use an X server to provide it, so you don't have to install a separate one.

You still have to set up your SSH client's connection to \"tunnel\" the X windows from the remote computer, though. You can do this by simply adding the -X option to your ssh command line, so for example to connect to Myriad with X-Forwarding:

ssh -X ccaaxyz@myriad.rc.ucl.ac.uk\n

To use X-Forwarding from outside UCL, you must either use the VPN, or use the SSH Gateway system, with the appropriate flags for both ssh steps, for example:

[me@my_computer ~]$ ssh\u00a0-X\u00a0ccaaxyz@ssh-gateway.ucl.ac.uk\n[...]\n[ccaaxyz@ejp-gateway-01 ~]$ ssh -X ccaaxyz@myriad.rc.ucl.ac.uk\n

Note

This assumes you use a Linux distribution that uses Xorg as its display server. If your distribution uses Wayland instead, and you aren't sure how to make this work, please contact us, letting us know what version of which distribution you're using.

"},{"location":"Supplementary/X-Forwarding/#checking-your-setup","title":"Checking your Setup","text":"

There are some simple programs on the system that use X, which can be used to check whether your setup is working correctly.

If these work, you have a working X connection. If not, you should see an error when you try to run them that may look something like:

xterm: Xt error: Can't open display:\n

Or:

Error: couldn't open display (null)\n

If you see these, please check you have followed all the appropriate steps above, and if you still have problems, contact rc-support@ucl.ac.uk for assistance.

"},{"location":"Supplementary/Young_GPU_Nodes/","title":"Young GPU Nodes","text":""},{"location":"Supplementary/Young_GPU_Nodes/#pilot-access","title":"Pilot access","text":"

A group of nominated pilot users had access to these nodes during the pilot, which lasted from 15 July to 5 September 2022. They are now available to all.

"},{"location":"Supplementary/Young_GPU_Nodes/#gpu-specs","title":"GPU specs","text":"

The nodes are listed in Young's node types table

There are 6 nodes which each have 64 AMD EPYC CPU cores and 8 Nvidia A100-SXM4-40GB GPUs. They have 1T RAM and 200G local disk is available to request as tmpfs unlike the rest of Young. The AMD equivalent of hyperthreading is not enabled.

"},{"location":"Supplementary/Young_GPU_Nodes/#request-gpus","title":"Request GPUs","text":"
# Submit a job to the GPU nodes by adding a request for a number of GPUs per node\n#$ -l gpu=8\n\n# Only Free jobs are available at present. Use your normal projects\n#$ -P Free\n#$ -A Inst_Project\n

At the start of the pilot, jobs did not share nodes and users always had access to all GPUs on each node. This has since been altered and device cgroups are implemented (as of 10 Aug 2022) so jobs can share nodes on the GPU nodes and each only have access to the number of GPUs they requested.

For example, 8 separate single-GPU jobs from different users can be running on one node, or 2 4-GPU jobs. Multi-node parallel GPU jobs do not share nodes, so a job asking for two nodes and some number of GPUs per node over those two nodes will block out the entire 16 GPUs even if using fewer than that.

"},{"location":"Supplementary/Young_GPU_Nodes/#exclusive-use-of-nodes","title":"Exclusive use of nodes","text":"

If you are using fewer than 8 GPUs but want to make sure nothing else is running on the same node as you, add this to your jobscript:

#$ -ac exclusive\n

This would generally only be done if you are benchmarking or investigating speeds and want to rule out anything else running on the rest of the node as possibly affecting your timings.

"},{"location":"Supplementary/Young_GPU_Nodes/#cuda-versions","title":"CUDA versions","text":"

The newer CUDA installs we have are made visible by first loading beta-modules but can then be used alongside any other compiler.

module load beta-modules\nmodule avail cuda\n\n# pick one of the 11.x CUDA installs\nmodule load cuda/11.3.1/gnu-10.2.0\n# or\nmodule load cuda/11.2.0/gnu-10.2.0\n
"},{"location":"Supplementary/Young_GPU_Nodes/#choosing-a-cuda-version","title":"Choosing a CUDA version","text":"

The drivers we have installed on the GPU nodes are version 460.27.03 which is CUDA 11.2. CUDA 11 has minor version compatibility so in most cases you can use the 11.3.1 runtime, but not all functionality is available.

If your code builds but when running it you get an error like this:

CUDA RUNTIME API error: Free failed with error cudaErrorUnsupportedPtxVersion \n

then use the cuda/11.2.0/gnu-10.2.0 module to build and run your program instead.

"},{"location":"Supplementary/Young_GPU_Nodes/#building-with-cuda","title":"Building with CUDA","text":"

If the code you are trying to build only needs to link to the CUDA runtime libraries, libcudart.so then you can build it on the login nodes which do not have GPUs.

If it needs the full libcuda.so to be available, you need to build it on a GPU node. You can submit it as a job or request an interactive session with qrsh. Eg:

qrsh -l gpu=8,h_rt=2:0:0,tmpfs=10G,mem=1G -pe smp 4 -P Free -A Inst_Project -now no\n
"},{"location":"Supplementary/Young_GPU_Nodes/#nvidia-documentation","title":"NVIDIA documentation","text":"

NVIDIA has some useful information at these locations:

"},{"location":"Supplementary/Young_GPU_Nodes/#run-on-a-specific-device-or-limit-the-number-visible","title":"Run on a specific device or limit the number visible","text":"

If you want to tell your code to run on a specific device or devices, you can set CUDA_VISIBLE_DEVICES to the ids between 0 and 7. If the code only uses one GPU it will usually default to running on device 0, but if it is running on all GPUs that belong to your job and you don't want it to, you can limit it.

# run on gpu 1\nexport CUDA_VISIBLE_DEVICES=1\n\n# run on gpus 0 and 4\nexport CUDA_VISIBLE_DEVICES=0,4\n
"},{"location":"Supplementary/Young_GPU_Nodes/#cuda-utility-devicequery","title":"CUDA utility deviceQuery","text":"

CUDA has a number of small utilities that come with its examples which can be useful: you can take a copy of the samples directory from the corresponding CUDA version - for example /shared/ucl/apps/cuda/11.3.1/gnu-10.2.0/samples/ and build those utilities with their corresponding CUDA module loaded.

samples/1_Utilities/deviceQuery will give you a small utility that will confirm that setting CUDA_VISIBLE_DEVICES is working - you can run it before and after setting it. The devices will have been renamed as 0 and 1 in its output, but the location IDs will be the same as when you could see all of them.

"},{"location":"Supplementary/Young_GPU_Nodes/#setting-ppn","title":"Setting PPN","text":"

You will also be able to set the number of cpu slots per node that you want. Instead of -pe smp or -pe mpi, you would request:

-pe ppn=<slots per node> <total slots>\n
# this would give you 8 slots per node and 16 slots total (so is using 2 nodes)\n# along with 8 GPUs per node (16 GPUs in total).\n#$ -pe ppn=8 16\n#$ -l gpu=8\n

Like -pe mpi this will also create a suitable machinefile for you so MPI will know how many cores on which nodes it can use. gerun (our mpirun wrapper) will use it automatically for Intel MPI as usual and our OpenMPI modules shouldn't need it since they have scheduler integration, but you can find it in $TMPDIR/machines if you are using mpirun and need it.

"},{"location":"Supplementary/Young_GPU_Nodes/#request-tmpfs","title":"Request tmpfs","text":"

The GPU nodes do have local disks and you can request an amount of tmpfs up to the maximum 200G like this:

# Request a $TMPDIR of 20G\n#$ -l tmpfs=20G\n

In the job, you refer to this using $TMPDIR. Many programs will use this environment variable for temporary files automatically, or you may need to tell them to do it explicitly with a command line argument.

$TMPDIR is deleted at the end of the job, so if you need any data that is being written to there, copy it back to your Scratch at the end of the job.

"},{"location":"Supplementary/Young_GPU_Nodes/#software-of-interest","title":"Software of interest","text":"

GPU software we have installed that may be of particular interest to users of Young.

You can also use NVIDIA Grid Cloud Containers via Singularity which provide pre-configured GPU applications. Our page gives an example of using the NAMD 3 container.

"},{"location":"Walkthroughs/Logging_In/","title":"Logging In","text":"

Here are walkthroughs showing all the steps to get logged in to one of our clusters from computers with different operating systems.

These instructions assume you have already applied for an account and received the email saying it has been created and that you can log in.

We use Myriad as the cluster we are logging in to. The same steps also apply to Kathleen.

To log in, you need an SSH client to be installed on the computer you are logging in from.

"},{"location":"Walkthroughs/Logging_In/#logging-in-from-windows","title":"Logging in from Windows","text":"

There are several choices of SSH client for Windows, with one now included in the Windows Command Prompt on Windows 10 or later. You can also log in via Desktop@UCL Anywhere, which provides a Windows environment inside the UCL network.

"},{"location":"Walkthroughs/Logging_In/#putty","title":"PuTTY","text":"

PuTTY will provide you with a graphical interface to configure your SSH connection and then open a terminal window you can type into and press return to submit.

"},{"location":"Walkthroughs/Logging_In/#windows-command-prompt","title":"Windows Command Prompt","text":"

Launch the Command Prompt from the Windows Start menu. It will give you a prompt that you can type commands into.

Replace \"uccacxx\" with your own central UCL username.

ssh uccacxx@ssh-gateway.ucl.ac.uk\n

If your computer has never connected to the Gateway before, it has no existing record of the host fingerprint which identifies it, so it will ask if you want to accept it and continue.

Type \"yes\" to accept the fingerprint. It will save it and check it next time. If the fingerprint is different, it can be an indication that something else is pretending to be the Gateway (or that it has changed after a major update). If concerned, contact rc-support@ucl.ac.uk or the main Service Desk.

It now informs you that it has added the Gateway to your list of known hosts.

You have now contacted the Gateway and it displays a small splash screen and asks for your UCL password. Nothing will show up when typing in this box - no placeholders or bullet point characters. Press return at the end to submit.

If you have a typo in your password or have changed it within the last couple of hours and the new one hasn't propagated yet, it will ask again.

Once the correct password has been entered, it will show you a longer message about the system.

From the Gateway, we want to ssh in to Myriad:

ssh uccacxx@myriad.rc.ucl.ac.uk\n

Your user on the Gateway will also not have connected to Myriad before, so you will get a similar prompt about Myriad's host fingerprint. You can check this against our current key fingerprints.

After saying \"yes\" you will be prompted for your password again, and after typing it in you will be logged in to Myriad and see Myriad's message containing information about the system and where to get help.

At the bottom you can see that the prompt on Myriad looks like

[uccacxx@login13 ~]$\n

It shows you your username, which Myriad login node you are on, and where you are (~ is a short way to reference your home directory). You can now look at what software is available and write jobscripts.

Helpful resources:

After you have successfully logged in for the first time, visit the remote access for further resources on accessing UCL services from outside the UCL firewall and creating an SSH key pair to help with logging in.

"},{"location":"Walkthroughs/Logging_In/#desktopucl-anywhere","title":"Desktop@UCL Anywhere","text":"

You can log in to this from a web browser or download a Citrix client. Once logged in, Desktop@UCL is inside the UCL network so we can log straight into a cluster with no need for a gateway machine or VPN.

You can use PuTTY or the Windows Command Prompt on Desktop@UCL Anywhere to log in to Myriad. It has a version of the Command Prompt called \"SSH Command Prompt\" that may set up some additional configuration for SSH usage - either should work.

If you want to use SSH keys for your connection, you need to use PuTTY and PuTTYgen. The OpenSSH Authentication Agent is not running as a service on Desktop@UCL Anywhere or on UCL managed laptops, so you cannot use ssh-add in the Windows Command Prompt.

"},{"location":"Walkthroughs/Machine_Learning/","title":"Machine Learning","text":"

Myriad is a traditional HPC system with a batch scheduler. This means that there are some additional steps to think about when running Machine Learning toolkits. Here are some videos that walk through installing and using various machine learning toolkits on Myriad.

"}]} \ No newline at end of file diff --git a/sitemap.xml.gz b/sitemap.xml.gz index b412478b8ef0baf2a00e8adc850a5bee7693b64c..3ecf50b256f4ff7812287c18c176c42313da6adc 100644 GIT binary patch delta 15 WcmZ3*wu+5SzMF%CC2=F03Nrv7Gy`M+ delta 15 WcmZ3*wu+5SzMF%?K7J#c3Nrv8j02qj