Loading...
ETL Frameworks
Database-to-Database CLI Tool
★ 4.1
Python API for Apache Spark
★ 4.8
pip install ingestrpip install pysparkpip install ingestrpip install pysparkPython data engineers use ingestr as a fast way to set up one-off or recurring data copies between systems — running `ingestr ingest --source-uri pg://... --dest-uri bq://... --table schema.table` from Python subprocess calls or shell scripts. For more complex transformations or pipeline logic, ingestr handles the raw ingestion while Python handles the business logic.
PySpark is the standard Python interface for large-scale ETL on Hadoop and cloud clusters. Data engineers write transformation logic using the DataFrame API — reading from S3 or Hive, applying joins and aggregations, then writing to Delta Lake or a data warehouse — with Spark distributing the work across hundreds of nodes.
Individual Tool Pages