arrow-rs

It's ever discussed in https://github.com/apache/arrow-rs/issues/3888 for comparing object_store in Apache Arrow to the APIs provided by Apache OpenDAL. Briefly, Apache OpenDAL is a library providing FS-like APIs over multiple storage backends, including S3 and many other cloud storage. A few database systems, such as GreptimeDB and Databend, use OpenDAL as a better S3 SDK to access data on cloud storage. Other solutions exist to manage filesystem-like interfaces over S3, including Alluxio and JuiceFS. Unlike Apache OpenDAL, Alluxio and JuiceFS need to be deployed standalone and have a dedicated internal metadata service.

rust_serialization_benchmark: https://github.com/djkoloski/rust_serialization_benchmark apache/arrow-rs: https://github.com/apache/arrow-rs From https://arrow.apache.org/faq/ : > How does Arrow relate to Flatbuffers? > Flatbuffers is a low-level building block for binary data serialization. It is not adapted to the representation of large, structured, homogenous data, and does not sit at the right abstraction layer for data analysis tasks. > Arrow is a data layer aimed directly at the needs of data analysis, providing a comprehensive collection of data types required to analytics, built-in support for “null” values (representing missing data), and an expanding toolbox of I/O and computing facilities. > The Arrow file format does use Flatbuffers under the hood to serialize schemas and other metadata needed to implement the Arrow binary I