Loading...
File Systems & Storage
Fault-Tolerant Distributed File System
★ 3.7
Simple Distributed File System
★ 4.2
N/A — system package, install via package managerN/A — Go binary, see seaweedfs.comN/A — system package, install via package managerN/A — Go binary, see seaweedfs.comPython data engineers in on-premise environments use LizardFS as a shared POSIX file system mounted across pipeline worker nodes. Python scripts write output files to the LizardFS mount and those files are immediately visible to all other nodes in the cluster — enabling simple shared-nothing pipeline patterns where workers write outputs that other workers consume without message queue coordination.
Python data engineers use SeaweedFS's S3-compatible API with boto3 to store and retrieve pipeline artifacts, model binaries, and intermediate data files. Its optimized handling of billions of small files makes it a good fit for storing ML training sample files or pipeline checkpoint files that would create excessive metadata overhead in traditional distributed file systems.
Individual Tool Pages