Conda

Conda is an open-source package management system and environment management system that runs on Windows, macOS, and Linux. Conda quickly installs, runs, and updates packages and their dependencies. Conda as a package manager helps you find and install packages.

Info

Full Conda Documentation

Module

module load conda/latest

Create

Conda enviornments must first be created and the conda packages can be installed within them

conda create -n <environment_name>

Note

All conda environments and packages will be installed to ~/.conda by default

Custom Path

If you wish to create a conda environment to share with your group, create the conda environment in a custom path

mkdir -p $WORK/conda/envs
conda create --prefix $WORK/conda/envs/<environment_name>

This will create the environment in your $WORK directory. To activate this environment, you must specify the exact path since it is not in the standard location for Conda environments

conda activate /panfs/pfs.local/work/<groupname>/<username>/conda/envs/<environment_name>

Activate

The above will create an empty conda environment. Next step is to activate the conda environment.

conda activate <environment_name>

Install Packages

Install packages into the fresh conda environment. This will install the latest version of python and matplotlib that are compatible with each other.

conda install python matplotlib

Specific version of package

You can specify the version of package you wish to install.

conda install python=3.8

Search for versions

Search for specific package named 'python':

conda search python

Search for a specific version of a package:

conda search 'python>=3.8'

Deactivate

Deactivating the conda environment returns you back to your normal shell.

conda deactivate

Channels

Conda channels are the locations where packages are stored. The default Anaconda channel has most packages, but sometimes an installation will need a new channel to look for packages in. Popular channels that are used are conda-forge and bioconda.

conda install cutadapt -c bioconda conda-forge

Conda File Usage

Users often run into the 100K file limit in their $HOME folder and most of the time it is due to conda. Conda downloads and unpackages all files for the environment in ~/.conda/pkgs. This uses a lot of files. One of the ways to circumvent using $HOME for the .conda folder is to symlink it to a folder in your $WORK directory.

mkdir $WORK/conda
mv ~/.conda/* $WORK/conda/
rmdir ~/.conda
ln -s $WORK/conda ~/.conda

Example Install

Putting it all together to create and install a conda enviornment

module load conda/latest
conda create cutadapt-env
conda activate cutadapt-env
conda install cutadapt -c bioconda conda-forge

Example Submit Sciprt

How to use a Conda environment in a submit script

module load conda/latest
conda activate cutadapt-env
cutadapt --version
conda deactivate

Hint

If you have a long workflow, you can insert the above anywhere in your submit script. Activate and deactivate multiple conda environments throughout a single submit script