Loading...
Unified Logging Layer
An open-source data collector for building a unified logging layer. FluentD structures data as JSON and provides 500+ community-contributed plugins for connecting various data sources and outputs, widely used for log aggregation and forwarding.
Explore similar tools in the Data Ingestion category that complement FluentD for your data engineering projects.
Open Source Message Broker
A robust, open-source message broker that supports multiple messaging protocols including AMQP, MQTT, and STOMP. RabbitMQ provides reliable message delivery with flexible routing, clustering, and federation for distributed data ingestion pipelines.
Distributed Pub-Sub Messaging
An open-source distributed pub-sub messaging system originally created by Yahoo. Pulsar provides multi-tenancy, geo-replication, and unified messaging and streaming with a serverless compute framework for lightweight processing.
Hadoop-RDBMS Data Transfer
A tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases. Sqoop uses MapReduce for parallel data transfer with support for incremental imports and direct connector APIs.