POpolyrand·Jun 11, 2025Submission
S5cmd: Parallel S3 and local filesystem execution tool
62 points · Open discussionpeak
s5cmd
Go2020–2025
peak / s5cmd
s5cmd
Parallel S3 and local filesystem execution tool.
9Distinct people
9Mentions
0Explicit recommenders
4.2KGitHub stars
What to know before exploring
What it is
Parallel S3 and local filesystem execution tool.
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 1:49:46 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year9 dated mentions
Observed mentions: 2020-09-13 22:44:30 UTC to 2025-06-11 13:38:00 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2020 | 2020-01-01 00:00:00 UTC | 2021-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2021 | 2021-01-01 00:00:00 UTC | 2022-01-01 00:00:00 UTC | 3 | 0 | 3 |
| 2022 | 2022-01-01 00:00:00 UTC | 2023-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2023 | 2023-01-01 00:00:00 UTC | 2024-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2024 | 2024-01-01 00:00:00 UTC | 2025-01-01 00:00:00 UTC | 2 | 0 | 2 |
| 2025 | 2025-01-01 00:00:00 UTC | 2026-01-01 00:00:00 UTC | 2 | 1 | 1 |
What Hacker News says
9 sourcesSomewhat related, I just came across s5cmd[1] which is mainly focused on performance and fast upload/download and sync of s3 buckets.
> 32x faster than s3cmd and 12x faster than aws-cli. For downloads, s5cmd can saturate a 40Gbps link (~4.3 GB/s), whereas s3cmd and aws-cli can only reach 85 MB/s and 375 MB/s respectively.
[1] https://github.com/peak/s5cmd
In Show HN: S3mini – Tiny and fast S3-compatible client, no-deps, edge-readyHow does it compare with s5cmd [1]? s5cmd is my goto tool for fast s3 sync and they have the following at the top of their Github page:
> For uploads, s5cmd is 32x faster than s3cmd and 12x faster than aws-cli. For downloads, s5cmd can saturate a 40Gbps link (~4.3 GB/s), whereas s3cmd and aws-cli can only reach 85 MB/s and 375 MB/s respectively.
[1] https://github.com/peak/s5cmd
In Show HN: S3HyperSync – Faster S3 sync tool – iterating with up to 100k files/sSwitch your blind faith to Go (s5cmd) and get ~1200 MB/s upload speed!
zfs send tank/pgdata@snapshot | pbzip2 > mybackup.zfs.bz2
s5cmd cp mybackup.zfs.bz2 s3://mygooglebucket/
https://github.com/peak/s5cmd/blob/master/README.md#Benchmar...
In We migrated from AWS to GCP with minimal downtimeI use s5cmd, it’s a fast parallel s3 cli client: https://github.com/peak/s5cmd
Haven’t heard of rclone before so can’t compare yet but s5cmd is supposed to work against GCS and other s3 compatible apis.
In Gcloud storage: Faster data transfers for Cloud Storage> you can also get an export of all objects in a bucket using S3 Inventory and run the output through AWS Batch in order to delete those objects
"S3 Batch Operations" sends S3 requests based on a csv file, which can but does not have to be from S3 Inventory. But S3 Batch Operations supports only a subset of APIs and this does not include DeleteObject(s). [0]
An AWS Batch job could run a container which sends DeleteObjects requests but only when triggered by a job queue which seems redundant here.
If I can't use an expiration lifecycle policy because I need a selection of objects not matching a prefix or object tags, I would run something with `s5cmd rm` [1]. Alternatively roll your own golang which parses the CSV and sends many DeleteObjects requests in parallel goroutines.
0. https://docs.aws.amazon.com/AmazonS3/latest/userguide/batch-..
Read the rest
s5cmd[0], written in Go, performs cp in parallel to/from s3 too. I've been using it for some time and it's plenty fast.
https://github.com/peak/s5cmd
In Downloading files from S3 with multithreading and Boto3Excellent walkthrough, love boto. We’ve recently been using s5cmd which we’ve found is ridiculously faster than boto without any extra boto tricks.
https://github.com/peak/s5cmd
In Downloading files from S3 with multithreading and Boto3or even faster https://github.com/peak/s5cmd
In Moving 25TB data from one S3 bucket to another took 7 engineers and 2 full days