Python poetry review

Development Code
, ,
1

poetry reviewEdd

I installed and tested poetry for the past couple of days, it works much better at a repo/project level (building the environment is much faster than conda). The initializer for a project is very nice and if they can incorporate the data science cookie cutter template with git initialization that would be a super handy feature.

The only thing keeping it from fully replacing miniforge for me is that it doesn’t include a python distribution (so if you install 3.9 on the system but your teammate needs 3.11, it won’t build a new venv with 3.11) and the instructions to install on windows are not super beginner friendly.

My ideal workflow is having conda manage the python versions (as it creates a completely new python directory), and install poetry to each python version environment (3.10, 3.11, 3.12). As I work on projects, I make local repo venvs which make the process of sharing environments with collaborators easier. Also with local venvs, vs code is able to pick it up when selecting the interpreter and will actually default to the venv rather than the conda environments.

A potential use case for poetry is when working company laptops that restrict admin access and only allows python to be installed from a company app store/portal, as you can install python and then point poetry to the python version from the windows path

installation woes

The curl python method for whatever reason used an older version of python on my system, and no amount of environment variables wrangling allowed me to point poetry to use a different distribution.

The suggested pipx method on windows also didn’t work either, so instead I used pip to install pipx

If you are also on windows and use miniconda/miniforge, these steps below worked for me

how to recreate my env

  1. install pipx https://github.com/pypa/pipx?tab=readme-ov-file#on-windows
    1. pip install pipx
    2. ensure the path using pipx ensurepath
  2. install poetry with pipx
    1. pipx install poetry
  3. have poetry always build in your local repo
    1. poetry config virtualenvs.in-project true
  4. build the poetry environment
    1. navigate to the project directory
    2. initialize poetry using poetry init in the directory you want to build the environment
    3. run poetry install to build the environment
  5. launch the poetry shell within the poetry environment
    1. poetry shell
1 Like
2

I did some further exploration of this ideal case while setting it up on my fedora desktop, I found that you don’t necessarily need to use pipx (which will isolate poetry from the base environment).

The original instructions are still valid, these new instructions are for those who want conda as the python version manager and poetry as the package manager. Using a pyproject.toml in conjunction with a local venv is very handy as the workflow for installing packages using poetry add, is very intentional and logs packages you need.

In an ideal python development environment, you would create a new conda environment for each project. In practice, I’ve found that I neglect my conda environments because switching between environments and projects is too cumbersome. Conda environments will duplicate the full python installation, whereas .venvs will just include the custom packages that get installed on top of an existing python install. Furthermore, using vs code/codium, loading the python kernel for jupyter notebooks using local .venvs is much faster than waiting for the conda environment

I wanted to use conda environments for each python version I use

  1. install miniforge, miniconda or anaconda
  2. create a conda environment for your python version you want to use. Replace $env_name and $version with the python version you want
    conda create --name $env_name python=$version -c conda-forge
  3. activate your newly created conda environment, again replace $env_name with the name you gave to your environment
    conda activate $env_name
  4. install poetry with pip
    python -m pip install poetry
    • note: python -m ensures that you will use the pip local to your environment, you shouldn’t need it by default, but I have it that way to avoid using the system’s global pip
  5. install poetry-conda to have poetry install to your local repo
    • note: as of the current moment, if poetry is installed to an conda environment, it will default to the venv directory within the conda environment. There have been numerous proposals for how to resolve this. Ranging from variable flags, to documentation revisions, and even a poetry-conda plugin as seen below
    • Until a decision is made from the poetry maintainers, this guide will use the plugin, as it is the current solution that works.
    python -m pip install poetry-conda
  6. Configure poetry to create virtual environments in the project
    poetry config virtualenvs.in-project true

With these instructions you should be ready to do work with your poetry environment

3

The package/venv management in python is such a shitshow… tbh xD

Maybe this poetry thing is worth checking out the next time I need python for something.

2 Likes
4

Yeah it’s definitely a tool you want in your toolbox if you want to keep track of your project’s dependencies or if you want to build python packages.

I found that poetry will append new packages to the end of the dependencies section, so it’s easier to track as I work on a project.

side tangent expanding the note in step 5

I just wish that poetry had a way to manage python versions better (the documentation recommends having poetry isolated, but doing so with conda really screws with the local project level venv.

There’s been an ongoing discussion of how poetry should dump .venvs within conda environments

Hopefully they can implement the proposals suggested in the thread