UC Davis DataLab
Winter 2024
Instructors & Authors: Nick Ulle, Oliver Kreylos, Tyler Shoemaker
Maintainer: Nick Ulle <[email protected]>
This workshop series provides an introduction to accessing and computing on remote servers such as UC Davis's "Farm" cluster. The series covers everything you need to know to get started: how to set up and use SSH to log in and transfer files, how to install software with conda, how to reserve computing time and run programs with SLURM, and shell commands that are especially useful for working with servers.
After this workshop series, learners should be able to:
- Use SSH to log in to a server
- Transfer files to and from a server
- Set up and use conda/mamba to install software on a server
- Use SLURM to run interactive and non-interactive software on a server
- Explain etiquette for using a server cluster such as Farm
Participants must have taken DataLab’s "Overview of Remote and High Performance Computing (HPC)” workshop and "Introduction to the Command Line" workshop series, or have equivalent prior experience. Participants must be comfortable with basic Linux shell syntax.
The course reader is a live webpage, hosted through GitHub, where you can enter curriculum content and post it to a public-facing site for learners.
To make alterations to the reader:
-
Check in with the reader's current maintainer and notify them about your intended changes. Maintainers might ask you to open an issue, use pull requests, tag your commits with versions, etc.
-
Run
git pull
, or if it's your first time contributing, see Setup. -
Edit an existing chapter file or create a new one. Chapter files may be either Markdown files (
.md
) or Jupyter Notebook files (.ipynb
). Either is fine, but you must remain consistent across the reader (i.e. don't mix and match filetypes). Put all chapter filess in thechapters/
directory. Enter your text, code, and other information directly into the file. Make sure your file:- Follows the naming scheme
##_topic-of-chapter.md/ipynb
(the only exception isindex.md/ipynb
, which contains the reader's front page). - Begins with a first-level header (like
# This
). This will be the title of your chapter. Subsequent section headers should be second-level headers (like## This
) or below.
Put any supporting resources in
data/
orimg/
. - Follows the naming scheme
-
Run the command
jupyter-book build .
in a shell at the top level of the repo to regenerate the HTML files in the_build/
. -
When you're finished,
git add
:- Any files you edited directly
- Any supporting media you added to
img/
Then
git commit
andgit push
. This updates themain
branch of the repo, which contains source materials for the web page (but not the web page itself). -
Run the following command in a shell at the top level of the repo to update the
gh-pages
branch:ghp-import -n -p -f _build/html
This uses the
ghp-import
Python package, which you will need to install first (pip install ghp-import
). The live web page will update automatically after 1-10 minutes.
We recommend using mamba to manage Python dependencies. The env.yml
file
in this repo contains a list of packages necessary to build the reader. You can
create a new conda environment with all of the packages listed in that file
with this shell command:
mamba env create --file env.yaml