TOtosh·Mar 23, 2024Submission
S3 Benchmark: Measure Amazon S3's performance from any location
2 points · Open discussiondvassallo
s3-benchmark
Go2019–2024
dvassallo / s3-benchmark
s3-benchmark
Measure Amazon S3's performance from any location.
6Distinct people
11Mentions
0Explicit recommenders
910GitHub stars
What to know before exploring
What it is
Measure Amazon S3's performance from any location.
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year11 dated mentions
Observed mentions: 2019-04-04 15:18:57 UTC to 2024-03-23 14:51:11 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2019 | 2019-01-01 00:00:00 UTC | 2020-01-01 00:00:00 UTC | 6 | 6 | 0 |
| 2020 | 2020-01-01 00:00:00 UTC | 2021-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2021 | 2021-01-01 00:00:00 UTC | 2022-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2022 | 2022-01-01 00:00:00 UTC | 2023-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2023 | 2023-01-01 00:00:00 UTC | 2024-01-01 00:00:00 UTC | 1 | 1 | 0 |
| 2024 | 2024-01-01 00:00:00 UTC | 2025-01-01 00:00:00 UTC | 1 | 1 | 0 |
What Hacker News says
11 sourcesThere's another benchmark somewhere showing S3 can max out a 100Gbps instance.
https://github.com/dvassallo/s3-benchmark
Another potential issue is ListBucket rate limiting. If you have lots of small objects, you'll spend most of the time waiting to discover the names than transferring data
In Ask HN: Have you ever switched cloud?TTFB in S3 is 20-30ms around the 50th percentile. it can go much higher at p99 [1]. In any case, rotational latency for HDD drives is an order of magnitude lower (typically 2-5ms for a seek operation).
S3 is great for higher throughput workloads where TTFB is amortized across larger downloads (this is why it's very common to use S3 as a "data lake" where larger columnar files are stored, usually at the order of hundreds of MiB).
I think it's an interesting project but perhaps explaining the use cases where this solution is beneficial would go a long way here.
[1] https://github.com/dvassallo/s3-benchmark
In A distributed Posix file system built on top of Redis and S3If your data is in S3, my experience is that you can push ~20-40MB/core/sec on most instances.
OP is probably talking about an x1e.32xlarge. According to Daniel Vassalo's S3 benchmark [1], it can do about 2.7GB/sec.
So your 4TB DB might take ~30min to fetch.
Bandwidth is free, you'd pay $2 for the 30 min of compute, and some fractions of pennies for the few hundred S3 requests.
[1]: https://github.com/dvassallo/s3-benchmark
In Does my data fit in RAM?S3 Benchmark: Measure Amazon S3's Performance from Any Location
1 points · Open discussionS3 performance from every EC2 instance type
2 points · Open discussionMeasure Amazon S3's performance from any location
3 points · Open discussionS3 Bandwidth from Every EC2 Instance
2 points · Open discussion