Loading...
File Systems & Storage
Scalable Network File System
★ 4.0
Fault-Tolerant Distributed File System
★ 3.7
N/A — system package, install via package managerN/A — system package, install via package managerN/A — system package, install via package managerN/A — system package, install via package managerPython data engineers in HPC and on-premise environments use GlusterFS as a shared storage layer accessible by multiple pipeline worker nodes simultaneously. Python jobs write output files to a GlusterFS mount point, and other nodes in the cluster can immediately read those files without data movement — simplifying distributed batch processing without object storage dependencies.
Python data engineers in on-premise environments use LizardFS as a shared POSIX file system mounted across pipeline worker nodes. Python scripts write output files to the LizardFS mount and those files are immediately visible to all other nodes in the cluster — enabling simple shared-nothing pipeline patterns where workers write outputs that other workers consume without message queue coordination.
Individual Tool Pages