Skip to content

Software: Environment Modules

Environment modules enable multiple versions of software to coexist on Statistics servers. These modules are optimized for our hardware, providing performance up to four times faster than their non-module counterparts.

Many modules include additional libraries commonly used for scientific computing, eliminating the need for manual installation. For example, the R module includes libraries such as Rcpp and promises.

To use software packages like R and Python, you must load the appropriate version into your environment using the module command.

Quick Reference

Command Description
module avail Lists all available modules
module list Lists all currently lodaded modules
module load <mod> Loads the specified module
module unload <mod> Unloads the specified module
module purge Unloads all currently loaded modules
module help <mod> Shows help information for the specified module.

Commonly Used Modules

Name Description
R-bundle-CRAN R and a large collection of R libaries for scientific computing. Recommended for most R use cases.
R R and a minimal set of libraries.
Python Recommended for most Python use cases.
Anaconda3 Not recommended due to very poor performance. If you need help finding or installing a specific library not available in the regular Python module, contact help@stat.washington.edu
Java
MATLAB Limited availability due to licensing. Contact help@stat.washington.edu for details.
MOSEK The MOSEK solver requires a free personal academic license. See https://www.mosek.com/license/request/personal-academic/

Tutorial

In this example, the R module will be loaded.

  • If you're using the cluster, start a Slurm interactive session with srun. For details, see Slurm Commands
  • List available modules with the module avail command. Once modules are loaded, module avail will show a (L) next to the loaded modules.
    • For a detailed description of all available modules, run module spider
  • Show module details. Notice that this will list all the extra packages included with this R verison: module help R
  • Load the R module: module load R-bundle-CRAN. By default, the module command will load the newest version available. You can select a specific version if needed - for example, module load R-bundle-CRAN/2024.06-foss-2023b.
    • Dependent modules will be loaded automatically. For example, the default R-bundle-CRAN module is linked to the OpenBLAS library. So the OpenBLAS module will load automatically when the R-bundle-CRAN module is loaded.
  • List the loaded modules: module list
  • R can now be run. To verify that you're using the module version of R (or any other package), run which R.
  • To unload modules, use module unload <module name>, or unload all modules with module purge

Using Modules with Cluster Jobs

To use environment modules with Slurm, a module load line must be included in your sbatch script before lines running your code. For example:

 #!/bin/bash
#SBATCH --job-name myjob              # Set a name for your job. This is especially useful if you have multiple jobs queued.
#SBATCH --partition short             # Slurm partition to use
#SBATCH --ntasks 1                    # Number of tasks to run. By default, one CPU core will be allocated per task
#SBATCH --time 0-02:00                # Time limit in D-HH:MM
#SBATCH --mem-per-cpu=300             # Memory limit for each tasks (in MB)
#SBATCH --array=1-123                 # The number of iterations for this job
module load R-bundle-CRAN
Rscript myscript.R inputfile$SLURM_ARRAY_TASK_ID

Troubleshooting

module unable to load or list any packages

Ensure you are in an interactive session when using the cluster; the module command on the login node will not display or load any packages. Start an srun session (Slurm Commands), then run module avail.

How do I install R packages? R packages are outdated?

The R-bundle-CRAN module is updated approximately every 6-18 months.

If you need newer R packages, you can install them in your home directory:

  1. Start an srun session on the "build" partition (see Slurm Commands)
  2. Load the R module: module load R-bundle-CRAN
  3. Start R and install packages from CRAN: install.packages("package_name")

How do I install Python packages?

You can install Python libraries in your home directory using virtualenv and pip:

  1. If you are on the cluster, start an srun session on the "build" partition (see Slurm Commands)
  2. Load the Python module: module load Python
  3. Create and load a Python virtual environment, then install with pip (see Pip and Virtualenv)