Loading...
Package Management
Python Package Installer
★ 4.7
Python dependency management and packaging in one tool
★ 4.5
N/A — bundled with Pythonpip install poetryN/A — bundled with Pythonpip install poetryPip is the foundational tool for managing Python dependencies in data engineering projects. Engineers pin exact package versions in `requirements.txt` for reproducible pipeline environments, use `pip install -e .` to install local library packages during development, and use pip in Dockerfile builds to create consistent container images for Airflow workers and Spark executors.
Python data engineers use Poetry to standardize project setup across the team — running `poetry install` from a cloned repo installs all pinned dependencies in an isolated virtual environment in one step, eliminating environment drift between machines. The `poetry add pandas==2.0` command adds a package, resolves its transitive dependencies against the full graph, and updates `poetry.lock` atomically, preventing the 'it works on my machine' problem. For pipeline projects with separate concerns, `poetry add --group dev pytest great-expectations` keeps test and quality tooling out of the production install. Teams publishing internal data libraries to a private PyPI registry use `poetry publish --repository private-pypi` to handle build and upload in a single authenticated step.
Individual Tool Pages