Loading...
ETL Frameworks
Transform Data in Your Warehouse
★ 4.9
Fast DataFrame library for Python and Rust
★ 4.8
pip install dbt-corepip install polarspip install dbt-corepip install polarsData engineers use dbt to manage all transformation logic inside the warehouse — writing SELECT statements as `.sql` model files that dbt compiles and runs in the right order. Python engineers also write custom dbt tests and macros in Python, and use dbt's Python models feature to run pandas or Spark logic alongside SQL in the same project.
Python data engineers use Polars as a high-performance pandas replacement for processing large datasets on a single machine. Its lazy API lets you chain transformations like .filter(), .group_by(), and .join() that Polars optimizes before executing. Common in data pipelines where pandas runs out of memory or becomes too slow, but where spinning up a Spark cluster is overkill.
Individual Tool Pages