Package Management
Python Package Installer
★ 4.7
Extremely fast Python package manager written in Rust
★ 4.8
N/A — bundled with Pythoncurl -LsSf https://astral.sh/uv/install.sh | shN/A — bundled with Pythoncurl -LsSf https://astral.sh/uv/install.sh | shPip is the foundational tool for managing Python dependencies in data engineering projects. Engineers pin exact package versions in `requirements.txt` for reproducible pipeline environments, use `pip install -e .` to install local library packages during development, and use pip in Dockerfile builds to create consistent container images for Airflow workers and Spark executors.
Python data engineers use uv to set up isolated project environments faster than any alternative — running `uv sync` installs all dependencies from a lockfile in seconds rather than minutes, even in fresh CI containers. The `uv run` command executes scripts or tools in ad-hoc virtual environments without manually activating anything, making one-off data processing scripts trivial to share. For teams managing multiple Python versions across projects, `uv python install 3.11 3.12` and per-project `.python-version` files eliminate the need for pyenv entirely. The global cache layer means that once a package version is downloaded, every subsequent project or CI job that needs it installs from disk at near-zero cost.
Individual Tool Pages