db2lake

About db2lake

A lightweight Node.js ETL framework for moving data from databases to data lakes and data warehouses. db2lake provides simple configuration-driven extraction with support for incremental loads and multiple output formats.

Key Features

• 1Tool for migrating relational database data to data lake formats (Parquet, Delta)
• 2Reads from PostgreSQL, MySQL, Oracle, and SQL Server
• 3Writes Parquet files with correct schema mapping and partitioning
• 4Supports full and incremental extraction modes
• 5Configurable via YAML for repeatable, version-controlled migrations

How Python Data Engineers Use db2lake

Python data engineers use db2lake to bootstrap data lake migration projects — extracting historical data from relational databases and writing it as partitioned Parquet files to S3 or HDFS. Once the initial migration is done, incremental extractions keep the lake in sync, and Python-based PySpark or DuckDB pipelines take over for ongoing processing.