Analytics/Systems/Anaconda

Anaconda is a prepackaged conda distribution for mostly python based analytics and research purposes. WMF maintains a custom debian package of Anaconda that includes some extra packages, but also has scripts for creating 'stacked' conda user environments. These conda user environments allow users to install packages into their on conda environment without modifying the base anaconda environment.

Usage

Anaconda base environment

To use the readonly Anaconda base environment, you can simply run python or other executables directly out of /usr/lib/anaconda-wmf/bin. If you prefer to activate the environment, run source /usr/lib/anaconda-wmf/bin/activate.

Creating a new conda user environment

Run

 anaconda-create-stacked-env

and a new conda environment will be created for you in ~/.conda/envs. When used, this environment will automatically append the base conda environment Python load paths to its own. If the same package is installed in both environments, your user conda environment's package will take precedence.

Activating a conda user environment

Running

 source /usr/bin/anaconda-activate-stacked-env

Will activate the most recently created user conda environment. You may also choose to activate a specific user conda environment by name or path. Run /usr/lib/anaconda-wmf/bin/conda info --envs to get a list of available conda environments. E.g.

 source /usr/bin/anaconda-activate-stacked-env otto_2020-08-03T17.28.44

Installing packages into your user conda environment

After activating your user conda environment, you can set http proxy env vars and install conda and pip packages. E.g.

 export http_proxy=http://webproxy.eqiad.wmnet:8080
 export https_proxy=http://webproxy.eqiad.wmnet:8080
 conda install -c conda-forge <desired_conda_package>
 pip install <desired_pip_package>

These packages will be installed into the currently activated conda user environment.

Deactivating your user conda environment

 source /usr/bin/anaconda-deactivate-stacked-env