Package Management
Python Package Installer
★ 4.7
Virtual Environment Manager
★ 4.6
N/A — bundled with Pythonpip install virtualenvN/A — bundled with Pythonpip install virtualenvPip is the foundational tool for managing Python dependencies in data engineering projects. Engineers pin exact package versions in `requirements.txt` for reproducible pipeline environments, use `pip install -e .` to install local library packages during development, and use pip in Dockerfile builds to create consistent container images for Airflow workers and Spark executors.
Python data engineers use virtualenv to isolate pipeline dependencies on shared servers and development machines — each project gets its own environment with pinned library versions, preventing conflicts between pipelines that depend on different versions of pandas or SQLAlchemy. Most modern projects use `python -m venv` (built-in) or Poetry instead.
Individual Tool Pages