Loading...
ETL Frameworks
Transform Data in Your Warehouse
★ 4.9
Bulk Data Loader
★ 3.9
pip install dbt-coreN/A — Java-based, install via embulk.orgpip install dbt-coreN/A — Java-based, install via embulk.orgData engineers use dbt to manage all transformation logic inside the warehouse — writing SELECT statements as `.sql` model files that dbt compiles and runs in the right order. Python engineers also write custom dbt tests and macros in Python, and use dbt's Python models feature to run pandas or Spark logic alongside SQL in the same project.
Python data engineers invoke Embulk from Python subprocess calls or Airflow BashOperator tasks — generating the YAML config file programmatically from a Python template, then running `embulk run config.yml`. Embulk's parallel file loading is used for bulk data migrations from legacy systems to modern warehouses where Python-native libraries are too slow.
Individual Tool Pages