seaweedfs

A very good alternative is seaweedfs https://github.com/chrislusf/seaweedfs/ based on facebook haystack paper (efficient small files) & more.

Besides storage cost, S3 API access cost can also be high if frequently accessed. And latency is unpredictable. You can use SeaweedFS Remote Object Store Gateway to cache S3 (or any S3 API compatible vendors) to local servers, and access them at local network speed, and asynchronously sync back to S3. https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remot...

You can host your own S3 API-compatible object storage service on some EC2 instances (exercise left to the reader to figure out how to make that reliable). Zero PUT cost, higher operational overhead. Minio: https://github.com/minio/minio SeaweedFS: https://github.com/chrislusf/seaweedfs Ceph: https://ceph.com/en/discover/technology/

Maybe they added it since you looked or did something not match your needs? https://github.com/chrislusf/seaweedfs/wiki/Erasure-coding-f...

If you're still talking about SeaweedFS, the answer seems to be that it's not a "raft-based object store", hence it's not as chatty as the parent comment described. That proxy node is a volume server itself, and uses simple replication to mirror its volume on another server. Raft consensus is not used for the writes. Upon replication failure, the data becomes read-only [1], thus giving up partition tolerance. These are not really comparable. [1] https://github.com/chrislusf/seaweedfs/wiki/Replication

I am in the process of migrating off of Rook Ceph after using it in production for two years. Setting it up is easy thanks to Rook, but wait until Ceph gets under load, then the real fun begins. If you only need object storage, I suggest looking into SeaweedFS[0]. It's a far more lightweight and performant solution. [0]: https://github.com/chrislusf/seaweedfs/

This is a fair complaint. :) For filer metadata, you should just pick the one you are most familiar with. There is a wiki page for production setup. https://github.com/chrislusf/seaweedfs/wiki/Production-Setup

Usually you would benchmark on difficult tasks, not the easiest one. For computers, batch IO operations are much faster than random IO and can easily saturate the network. This benchmark uses large batch size, 64MB, to test. There is nothing new here. Most common file systems can easily do the same. The difficult task is to read and write lots of small files. There is a term for it, LOSF. I work on SeaweedFS, https://github.com/chrislusf/seaweedfs , which is designed to handle LOSF. And of course, no problem with large files at all.

Interesting that SeaweedFS also has a similar named "Gateway to Remote Object Storage" https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remot... The difference is that SeaweedFS can support both read and write, with asynchronous write back. Nginx can support read only caching with 1 hour TTL

There are simply too many object store vendors, with different storage pricing, egress pricing, network speed, and reliabilities. What about using SeaweedFS gateway to remote object store? https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remot... Basically you can think it is a cache. You operate with normal S3 APIs, and the data will be uploaded to remote storage asynchronously (most vendors have free uploading). It's not a magic pill for all users, but it should be very fast, and cut down the API costs, egress costs.

Most cloud object storage can be a good off-site backup. But accessing your data fast and cheap is not easy. That is why SeaweedFS added a gateway to remote object store. https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remot... , which asynchronously write local changes to the cloud. If there are enough local capacity, there should be no egress cost. Hopefully, this can change the common pattern, to really treat the cloud object store as a backup.

As a main contributor to an open source project (https://github.com/chrislusf/seaweedfs), I can confirm that this finding is so true. However, seems this research did not look into Apache projects, which basically maintain a different culture to encourage more contributors, so much as to encourage the main contributors to refrain from jumping to solve an issue until another person steps in first.

Hey, I was trying out Seaweed yesterday. I ran into an issue with HTTPS, it seems to be not very well supported at the moment. I managed to get it running mostly secure, but replication didn't work because the volumes were calling eachother with HTTP instead of HTTPS. The HTTP request is hardcoded here: https://github.com/chrislusf/seaweedfs/blob/43fd11278ef81185... and probably in a lot of other places. I even tried setting the address of the volume to https in the startup script, but then it makes a request to http://https://volume1.example.com and it still fails. I also noticed that the master API is still available over HTTP even when HTTPS is enabled. I can make an issue for these things if you want. These issues are currently the only showstopper for me. I need to have every endpoint on TLS with peer verification enabled. If you can

> That would make it impossible to link directly to a single storage server as a single file would also be distributed. Check out https://github.com/chrislusf/seaweedfs/ implementation of reed solomon. Small files can still be served from 1 server. It's also efficient for small files, which a image store requires.