Kedro

About Kedro

An open-source Python framework for creating reproducible, maintainable, and modular data science code. Kedro applies software engineering best practices to data pipelines with built-in data catalog, pipeline visualization, and experiment tracking.

Key Features

• 1Modular pipeline framework with a catalog-based data management layer
• 2Node and Pipeline abstractions separate data logic from orchestration
• 3DataCatalog decouples dataset reading/writing from pipeline code
• 4Built-in support for Jupyter, Airflow, Prefect, and Kubeflow runners
• 5Project template enforces consistent structure across data science projects

How Python Data Engineers Use Kedro

Data engineering teams use Kedro to structure ML and analytics pipelines as modular, testable Python functions. The DataCatalog allows engineers to define data sources (S3 Parquet files, SQL tables, local CSVs) in a YAML config — switching environments just changes the catalog config, not the pipeline code. Nodes are pure Python functions that are easy to unit test.