Multiple python versions in one environment

Some tools are designed to have access to multiple python interpreters, and then create their own virtualenvs. Tox is a common example (although there are other ways to use tox in Actions). Less common, but my use case, is maturin https://github.com/PyO3/maturin , which is used to build rust crates with python bindings. It's a lot quicker to do them all at once rather than in different jobs, as most of the compiled code is shared.

It would be helpful if setup-python allowed users to pass an array of python versions and installed all of them (which one takes python is up for debate, but python3.7, python3.8 etc should be fine).

I'd like to use tox --parallel for linting py2 and py3 as it's faster than separate machines. So I'd also like to be able to install both at once

I would go for installing x86 and x64 python's at same time as well, expecially if it means generating python c extension modules depending on the platform.

Any practical workarounds? I need this in order to make use of combined coverage reporting and I really want to avoid going towards collecting artifacts and combining them later, it would add a lot of complexity to the pipelines and increase the lock-in effect.

My goal is to read the python versions I want to test against from a config file, so I don't have to update the workflow(s) later on, when I only want to add or remove a python version. As I currently cannot just pass the list of python versions to actions/setup-python action I separate them in another job beforehand and put them in different variables like so:

jobs:
  pyproject-config:
    name: Get config from pyproject.toml
    runs-on: ubuntu-latest
    outputs:
      test_python_version_1: ${{ steps.get-config.outputs.test_python_version_1 }}
      test_python_version_2: ${{ steps.get-config.outputs.test_python_version_2 }}
      ...
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
      - name: Install dependencies
        run: |
          python -m pip install -U pip
          pip install -U tomlkit
      - name: Get config
        id: get-config
        shell: python
        run: |
          import tomlkit

          with open("pyproject.toml") as pyproject_file:
              PYPROJECT = tomlkit.parse(pyproject_file.read())

          test_ver = PYPROJECT["tool"]["_testing"]["ci_test_python_versions"]
          if len(test_ver) >= 1:
              print(f"::set-output name=test_python_version_1::{test_ver[0]}")
          if len(test_ver) >= 2:
              print(f"::set-output name=test_python_version_2::{test_ver[1]}")
          ...

The test job than depends on the upper job and initializes the python versions like so:

jobs:
  test-code:
    needs: pyproject-config
    ...
    steps:
      - uses: actions/setup-python@v2
        with:
      - uses: actions/setup-python@v2
        with:
          python-version: ${{ needs.pyproject-config.outputs.test_python_version_1 }}
        if: needs.pyproject-config.outputs.test_python_version_1
      - uses: actions/setup-python@v2
        with:
          python-version: ${{ needs.pyproject-config.outputs.test_python_version_2 }}
        if: needs.pyproject-config.outputs.test_python_version_2
      ...

I currently does that for 10 possible python versions and have 7 specified (incl. PyPy) so 3 steps are skipped.

I was able to use the Miniconda action as shown below to accomplish the OP's mention for tox with multiple Pythons in one invocation, where I am testing a Python library for 3.7, 3.8, and 3.9. Note the bash -l {0} is required for the Miniconda action to work properly across steps. Below, the testenv environment is automatically active at each step. Essentially, I just create the extra Python environments using conda and append the paths for the Python binaries in those environments to the PATH when running tox in the testing environment.

name: build_tests
on: [push]
jobs:
  build:
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
    name: Run tox build tests
    runs-on: ${{ matrix.os }}
    defaults:
      run:
        shell: bash -l {0}
    steps:
      - name: Setup repo
        uses: actions/checkout@v2
      - uses: conda-incubator/setup-miniconda@v2
        with:
          miniconda-version: latest
          python-version: 3.8
          activate-environment: testenv
          channels: conda-forge
      - name: Setup conda python environments
        run: |
          conda config --set always_yes yes --set changeps1 no
          conda install -q -c conda-forge tox
          conda create -q -n py37 python=3.7
          conda create -q -n py39 python=3.9
      - name: Run pytest
        env:
          TOXENV: py37,py38,py39
        run: |
          export PY37_PATH=$CONDA/envs/py37/bin
          export PY39_PATH=$CONDA/envs/py39/bin
          export PATH=$PATH:$PY37_PATH:$PY39_PATH
          tox
      - name: Run code checks
        if: ${{ matrix.os == 'ubuntu-latest' }}
        env:
          TOXENV: typing,pypi-description,manifest,precom
        run: tox

Hello everyone. We released a new version with support to install multiple python versions and updated the major tag.

For now I'm going to close the issue. If you have any concerns feel free to ping us.

Hello everyone. We released a new version with support to install multiple python versions and updated the major tag.

For now I'm going to close the issue. If you have any concerns feel free to ping us.

Does this include the ability to specify multiple architectures as well in it too? like x86 and x64 where it will install each version with each architecture listed?

Navigation Menu

Repository navigation

Multiple python versions in one environment #98

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Multiple python versions in one environment #98

Description

Activity

graingert commented on Aug 2, 2020

AraHaan commented on Oct 20, 2020

ssbarnea commented on Jan 11, 2021

Cielquan commented on Jan 12, 2021

btcross26 commented on May 24, 2021

dmitry-shibanov commented on Jan 9, 2023

AraHaan commented on Jan 11, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions