Loading...
ETL Frameworks
Transform Data in Your Warehouse
★ 4.9
Database-to-Database CLI Tool
★ 4.1
pip install dbt-corepip install ingestrpip install dbt-corepip install ingestrData engineers use dbt to manage all transformation logic inside the warehouse — writing SELECT statements as `.sql` model files that dbt compiles and runs in the right order. Python engineers also write custom dbt tests and macros in Python, and use dbt's Python models feature to run pandas or Spark logic alongside SQL in the same project.
Python data engineers use ingestr as a fast way to set up one-off or recurring data copies between systems — running `ingestr ingest --source-uri pg://... --dest-uri bq://... --table schema.table` from Python subprocess calls or shell scripts. For more complex transformations or pipeline logic, ingestr handles the raw ingestion while Python handles the business logic.
Individual Tool Pages