You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Analytics/Systems/Anaconda: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Ottomata
mNo edit summary
imported>Ottomata
Line 3: Line 3:
= Usage =
= Usage =
== Anaconda base environment ==
== Anaconda base environment ==
To use the readonly Anaconda base environment, you can simply run python or other executables directly out of <code>/usr/lib/anaconda-wmf/bin</code>.  If you prefer to activate the environment, run <code>source /usr/lib/anaconda-wmf/bin/activate</code>.
To use the readonly Anaconda base environment, you can simply run python or other executables directly out of <code>/usr/lib/anaconda-wmf/bin</code>.  If you prefer to activate the anaconda base environment, run <code>source /usr/lib/anaconda-wmf/bin/activate</code>.


== Creating a new conda user environment ==
== Creating a new conda user environment ==
Run
Run


   anaconda-create-stacked-env
   /usr/lib/anaconda-wmf/bin/conda-create-stacked


and a new conda environment will be created for you in ~/.conda/envs.  When used, this environment will automatically append the base conda environment Python load paths to its own.  If the same package is installed in both environments, your user conda environment's package will take precedence.
and a new conda environment will be created for you in ~/.conda/envs.  When used, this environment will automatically append the base conda environment Python load paths to its own.  If the same package is installed in both environments, your user conda environment's package will take precedence.
If you prefer, you can name your conda environment
  /usr/lib/anaconda-wmf/bin/conda-create-stacked my-cool-env


== Activating a conda user environment ==
== Activating a conda user environment ==
There are several ways to activate a conda user environment.  Just running
  source /usr/lib/anaconda-wmf/bin/conda-create-stacked
On its own will attempt to guess at the most recent conda environment to activate.  If you only have one conda environment, this will work.


Running
You can also specify the name of the conda env to activate. Run <code>/usr/lib/anaconda-wmf/bin/conda info --envs</code> to get a list of available conda environments. E.g.


   source /usr/bin/anaconda-activate-stacked-env
   source /usr/lib/anaconda-wmf/bin/conda-create-stacked otto_2020-08-17T20.52.02


Will activate the most recently created user conda environment.  You may also choose to activate a specific user conda environment by name or path. Run <code>/usr/lib/anaconda-wmf/bin/conda info --envs</code> to get a list of available conda environments.  E.g.
Or, you can run the 'activate' script out if your conda environment path:


   source /usr/bin/anaconda-activate-stacked-env otto_2020-08-03T17.28.44
   source ~/.conda/envs/otto_2020-08-17T20.52.02/bin/activate


== Installing packages into your user conda environment ==
== Installing packages into your user conda environment ==
Line 33: Line 42:
== Deactivating your user conda environment ==
== Deactivating your user conda environment ==


   source /usr/bin/anaconda-deactivate-stacked-env
   source /usr/lib/anaconda-wmf/bin/conda-deactivate-stacked
 
Or, since the user conda env's bin dir has been added to your path, you should also be able to just run
 
  source deactivate

Revision as of 21:03, 17 August 2020

Anaconda is a prepackaged conda distribution for mostly python based analytics and research purposes. WMF maintains a custom debian package of Anaconda that includes some extra packages, but also has scripts for creating 'stacked' conda user environments. These conda user environments allow users to install packages into their own conda environment without modifying the base anaconda environment.

Usage

Anaconda base environment

To use the readonly Anaconda base environment, you can simply run python or other executables directly out of /usr/lib/anaconda-wmf/bin. If you prefer to activate the anaconda base environment, run source /usr/lib/anaconda-wmf/bin/activate.

Creating a new conda user environment

Run

 /usr/lib/anaconda-wmf/bin/conda-create-stacked

and a new conda environment will be created for you in ~/.conda/envs. When used, this environment will automatically append the base conda environment Python load paths to its own. If the same package is installed in both environments, your user conda environment's package will take precedence.

If you prefer, you can name your conda environment

 /usr/lib/anaconda-wmf/bin/conda-create-stacked my-cool-env

Activating a conda user environment

There are several ways to activate a conda user environment. Just running

 source /usr/lib/anaconda-wmf/bin/conda-create-stacked

On its own will attempt to guess at the most recent conda environment to activate. If you only have one conda environment, this will work.

You can also specify the name of the conda env to activate. Run /usr/lib/anaconda-wmf/bin/conda info --envs to get a list of available conda environments. E.g.

 source /usr/lib/anaconda-wmf/bin/conda-create-stacked otto_2020-08-17T20.52.02

Or, you can run the 'activate' script out if your conda environment path:

 source ~/.conda/envs/otto_2020-08-17T20.52.02/bin/activate

Installing packages into your user conda environment

After activating your user conda environment, you can set http proxy env vars and install conda and pip packages. E.g.

 export http_proxy=http://webproxy.eqiad.wmnet:8080
 export https_proxy=http://webproxy.eqiad.wmnet:8080
 conda install -c conda-forge <desired_conda_package>
 pip install <desired_pip_package>

These packages will be installed into the currently activated conda user environment.

Deactivating your user conda environment

 source /usr/lib/anaconda-wmf/bin/conda-deactivate-stacked

Or, since the user conda env's bin dir has been added to your path, you should also be able to just run

 source deactivate