Edit

Share via

Run pipelines with Anaconda environments

Azure DevOps Services

Set up and use Anaconda with Python in your pipeline. Anaconda is a Python distribution for data science and machine learning.

Prerequisites

Python is preinstalled on Microsoft-hosted agents for Linux, macOS, and Windows. You don't have to set up anything more to build Python projects. To see which Python versions are preinstalled, see Software.

Product Requirements
Azure DevOps - An Azure DevOps project.
- An ability to run pipelines on Microsoft-hosted agents. You can either purchase a parallel job or you can request a free tier.
- Basic knowledge of YAML and Azure Pipelines. For more information, see Create your first pipeline.
- Permissions:
     - To create a pipeline: you must be in the Contributors group and the group needs to have Create build pipeline permission set to Allow. Members of the Project Administrators group can manage pipelines.
    - To create service connections: You must have the Administrator or Creator role for service connections.
GitHub - A GitHub account.
- A GitHub service connection to authorize Azure Pipelines.

Get started

Follow these instructions to set up a pipeline for a sample Python app with Anaconda environment.

  1. Sign in to your Azure DevOps organization and navigate to your project.

  2. In your project, navigate to the Pipelines page. Then choose the action to create a new pipeline.

  3. Walk through the steps of the wizard by first selecting GitHub as the ___location of your source code.

  4. You might be redirected to GitHub to sign in. If so, enter your GitHub credentials.

  5. When the list of repositories appears, select your Anaconda sample repository.

  6. Azure Pipelines analyzes the code in your repository and detects an existing azure-pipelines.yml file.

  7. Select Run.

  8. A new run is started. Wait for the run to finish.

Tip

To make changes to the YAML file as described in this article, select the pipeline in the Pipelines page, and then Edit the azure-pipelines.yml file.

Add conda to your system path

On hosted agents, conda isn't included in PATH by default to prevent its Python version from conflicting with other installed versions. Use the task.prependpath agent command to make conda available to all later steps.

- bash: echo "##vso[task.prependpath]$CONDA/bin"
  displayName: Add conda to PATH

Create an environment

From command-line arguments

The conda create command creates an environment with the arguments you pass.

- bash: conda create --yes --quiet --name myEnvironment
  displayName: Create Anaconda environment

From YAML

You can check in an environment.yml file to your repo that defines the configuration for an Anaconda environment.

- script: conda env create --quiet --file environment.yml
  displayName: Create Anaconda environment

Note

If you use a self-hosted agent and don't remove the environment at the end, you get an error on the next build because the environment already exists. To fix this, remove the environment before creating a new one with conda env remove --name your-env-name.

Note

If you use self-hosted agents that share storage and run jobs in parallel using the same Anaconda environments, there can be clashes between those environments. To fix this, use the --name argument and a unique identifier as the argument value, like a concatenation with the $(Build.BuildNumber) build variable.

Install packages from Anaconda

The following YAML installs the scipy package in the conda environment named myEnvironment.

- bash: |
    conda activate myEnvironment
    conda install --yes --quiet --name myEnvironment scipy
  displayName: Install Anaconda packages

Run pipeline steps in an Anaconda environment

Note

Each build step runs in its own process. When you activate an Anaconda environment, it edits PATH and makes other changes to its current process. So, activate an Anaconda environment separately for each step.

- bash: |
    conda activate myEnvironment
    python -m pytest --junitxml=junit/unit-test.xml
  displayName: pytest

- task: PublishTestResults@2
  inputs:
    testResultsFiles: 'junit/*.xml'
  condition: succeededOrFailed()

FAQ

Why am I getting a "Permission denied" error?

On hosted macOS, the agent user doesn't own the directory where Miniconda is installed. To fix this issue, go to the "Hosted macOS" tab under Add conda to your system path.

Why does my build stop responding on a conda create or conda install step?

If you don't pass --yes, conda stops and waits for user input.

Why is my script on Windows stopping after it activates the environment?

On Windows, activate is a Batch script. You must use the call command to resume running your script after activating. See examples of using call in a pipeline.

How can I run my tests with multiple versions of Python?

Go to Build Python apps in Azure Pipelines.