Weed-FS

Weed-FS is a simple and highly scalable distributed file system. There are two objectives: to store billions of files, and to serve the files fast! Instead of supporting full POSIX file system semantics, it implements only a key-file mapping. Instead of managing all file metadata in a central master, it manages file volumes in the central master and lets volume servers manage files and the metadata. This relieves concurrency pressure from the central master and spreads file metadata into volume servers' memories, allowing faster file access with just one disk read operation. It is modelled on Facebook's Haystack design paper. Only 40 bytes of disk storage are required for each file's metadata, and disk reads are O(1).