HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
cr-marcstevens
sha1collisiondetection
C2017–2025
cr-marcstevens / sha1collisiondetection

sha1collisiondetection

Library and command line tool to detect SHA-1 collision in a file

Command LineLibraries & SDKs
Explore on GitHubRead Hacker News discussions
17Distinct people
19Mentions
2Explicit recommenders
1.4KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Library and command line tool to detect SHA-1 collision in a file

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

Each new _unrelated_ SHA1 attack will need an update in SHA1DC. But it has to be truly unrelated, as the collision detection method is fairly robust. I recommend reading the…

theamk used a recommendation phrase. 2 indexed comments match. This is a phrase match, not a verdict.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 5:04:57 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year19 dated mentions

Observed mentions: 2017-02-23 14:39:53 UTC to 2025-03-26 14:05:46 UTC.

02462017-01-01 00:00:00 UTC to 2018-01-01 00:00:00 UTC (exclusive): 7 mentions; 1 submissions, 6 comments20172018-01-01 00:00:00 UTC to 2019-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-01-01 00:00:00 UTC to 2020-01-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2020-01-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 4 mentions; 0 submissions, 4 comments2021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments20212022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 3 mentions; 1 submissions, 2 comments2023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20172017-01-01 00:00:00 UTC2018-01-01 00:00:00 UTC716
20182018-01-01 00:00:00 UTC2019-01-01 00:00:00 UTC000
20192019-01-01 00:00:00 UTC2020-01-01 00:00:00 UTC202
20202020-01-01 00:00:00 UTC2021-01-01 00:00:00 UTC404
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC101
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC312
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC000
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC101
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC101

What Hacker News says

19 sources
kbolino·Mar 26, 2025
GitHub Actions could use lock files, and they would be useful specifically for composite actions, but they don't really solve the full problem at hand. First, many GitHub Actions use a Docker image to do their work. This runs into basically the exact same problem in container land. So even if I pin the commit hash of an action, if that action doesn't pin the digest hash of its container image, then it's still ultimately a mutable reference. Second, lock files don't protect against officially distributed malware when you intentionally update your dependencies. The lock file is designed to produce a repeatable build, and that does protect against a lot of supply-chain attacks when you never change anything, but you generally can't get away with never changing anything forever. Third, commit hashes are SHA-1, and while it is exceedingly diff
Read the rest
icult to produce SHA-1 collisions through Git, it isn't impossible and it will only get easier over time. Git already technically has SHA-256 support but it's basically never used in the wild. This needs to be addressed sooner rather than later. EDIT: The third point may not be as practically pressing as I thought. While SHA-1 is broken and shouldn't be used in new protocols, the techniques to break it faster than brute-force can be detected [1], and GitHub already rejects commits that are flagged by the detector [2]. I still think Git should more aggressively use a better hash, and should also be designed to migrate to new hash algorithms more easily in the future, but at least there are mitigations against attacks. [1]: https://github.com/cr-marcstevens/sha1collisiondetection [2]: https://github.blog/news-insights/company-news/sha-1-collisi...
In Whose code am I running in GitHub Actions?
theamk·Dec 31, 2024Recommendation phrase
Each new _unrelated_ SHA1 attack will need an update in SHA1DC. But it has to be truly unrelated, as the collision detection method is fairly robust. I recommend reading the original "Counter-cryptanalysis" paper [0] for details on how attacks work and how they can be mitigated (there is certain internal state in SHA1 that is used in all known attacks). BTW, this paper has an interesting anecdote: apparently Flame malware had exploited MD5 collisions using novel unpublished attack method... and yet it was detected by collision detector (section 3.2). Another example is that SHA-mbles attack, published 3 years after "Counter-cryptanalysis", was detected as well, with no required code changes. No, there is nothing "fundamentally incompatible" in the new SHA1DC method. After all, git came out in 2005, 11 years before SHA1 attacks were known
Read the rest
, so it used regular SHA1. The collision detector was added in 2017 and nothing broke, because false positive chance is 2^-90 [1]. I have not heard of any new SHA1 collision results, but if they are based on no-difference differential paths, git has nothing to worry about. And if they are not, it may be possible to extend DC detector to seamlessly detect and prevent those attacks, and then only upgrade git clients, keeping backward and forward compatibility for data. Of course there is always a chance that someone will come out with all-new SHA1 preimage attack that cannot be detected without high rate of false positive, so it's prudent to switch git to sha256. There is a lot of work being done: git's sha256 mode went out of beta in 2.42 (2023), but neither github nor gitlab support it. But since the current state of git's sha is that there is nothing broken, and git git commit hash _is_ "enough to know you are distributing/executing the legitimate code, as opposed to a malicious doppelganger", there is no real pressure. [0] https://marc-stevens.nl/research/papers/C13-S.pdf [1] https://github.com/cr-marcstevens/sha1collisiondetection
In Colliding with the SHA prefix of Linux's initial Git commit
dwheeler·Dec 15, 2022
Git no longer uses SHA-1. It instead uses a variant called SHA-1DC that detects some known problems, and in those cases returns a different answer. More info: <https://github.com/cr-marcstevens/sha1collisiondetection>. Git switched to SHA-1DC in its version 2.13 release in 2017. It's a decent stopgap but not a grrat long term solution. There is also work to support SHA-256, though that seems to have stalled: https://lwn.net/Articles/898522/ The fundamental problem is that get developers assumed that hash algorithms would never be changed, and that was a ridiculous assumption. It's much wiser to implement crypto agility.
In NIST is announcing that SHA-1 should be phased out by Dec. 31, 2030
bawolff·Sep 20, 2022
For sha1, people made a system where you can detect the patterns that lead to a collision, and (for example) replace it with a different hash only for inputs that would be a problem. https://github.com/cr-marcstevens/sha1collisiondetection i think git does this to eek more life out of sha1. I imagine you could take a similar counter-cryptnalysis approach to md5. (I am out of my depth here, so there could be reasons this doesnt work for md5 im unaware of)
In Hash collisions and exploitations – Instant MD5 collision
est31·Jan 8, 2020
> it's very hard to make a "bad" variant of the code [...] looking like sane code That is very hard, but not what was quoted above. The length has no part in it. The core part needed for the shattered collision attacks involves basically binary data. $ curl -s https://shattered.io/static/shattered-1.pdf | hexdump -C > s1 $ curl -s https://shattered.io/static/shattered-2.pdf | hexdump -C > s2 $ diff s1 s2 13,20c13,20 < 000000c0 73 46 dc 91 66 b6 7e 11 8f 02 9a b6 21 b2 56 0f |sF..f.~.....!.V.| < 000000d0 f9 ca 67 cc a8 c7 f8 5b a8 4c 79 03 0c 2b 3d e2 |..g....[.Ly..+=.| < 000000e0 18 f8 6d b3 a9 09 01 d5 df 45 c1 4f 26 fe df b3 |..m......E.O&...| < 000000f0 dc 38 e9 6a c2 2f e7 bd 72 8f 0e 45 bc e0 46 d2 |.8.j./..r..E..F.| < 00000100 3c 57 0f eb 14 13 98 bb 55 2e f5 a0 a8 2b e3 31 |<W.....
Read the rest
.U....+.1| < 00000110 fe a4 80 37 b8 b5 d7 1f 0e 33 2e df 93 ac 35 00 |...7.....3....5.| < 00000120 eb 4d dc 0d ec c1 a8 64 79 0c 78 2c 76 21 56 60 |.M.....dy.x,v!V`| < 00000130 dd 30 97 91 d0 6b d0 af 3f 98 cd a4 bc 46 29 b1 |.0...k..?....F).| --- > 000000c0 7f 46 dc 93 a6 b6 7e 01 3b 02 9a aa 1d b2 56 0b |.F....~.;.....V.| > 000000d0 45 ca 67 d6 88 c7 f8 4b 8c 4c 79 1f e0 2b 3d f6 |E.g....K.Ly..+=.| > 000000e0 14 f8 6d b1 69 09 01 c5 6b 45 c1 53 0a fe df b7 |..m.i...kE.S....| > 000000f0 60 38 e9 72 72 2f e7 ad 72 8f 0e 49 04 e0 46 c2 |`8.rr/..r..I..F.| > 00000100 30 57 0f e9 d4 13 98 ab e1 2e f5 bc 94 2b e3 35 |0W...........+.5| > 00000110 42 a4 80 2d 98 b5 d7 0f 2a 33 2e c3 7f ac 35 14 |B..-....*3....5.| > 00000120 e7 4d dc 0f 2c c1 a8 74 cd 0c 78 30 5a 21 56 64 |.M..,..t..x0Z!Vd| > 00000130 61 30 97 89 60 6b d0 bf 3f 98 cd a8 04 46 29 a1 |a0..`k..?....F).| An ASCII formatted file only has text data. Also, with the shattered attack you can't choose what the two versions should be so you are required to cross reference the different looking binary data to turn on/turn off some functionality. So the attack is mostly interesting when you include binary data. With the chosen prefix attack, you can have two arbitrary components, even textual ones, but they still have to be followed by such a binary component. Also now git has collision detection code from sha1collisiondetection [1], making attacks even harder. [1]: https://github.com/cr-marcstevens/sha1collisiondetection
In The first chosen-prefix collision for SHA-1
Thorrez·Jan 8, 2020
No, you and joeyh are incorrect about the test (but correct about the result). As can be seen in the output, SHA1(bar)= f1d2d2f924e986ac86fdf7b36c94bcdf32beec15 but git_SHA1(bar) = 257cc5642cb1a054f08cc83f2d943e56fd3ebe99 . Why is there a difference? Not because of hardened SHA1. Hardened SHA1 essentially always produces identical outputs to SHA1 > git doesn't really use SHA-1 anymore, it uses Hardened-SHA-1 (they just so happen to produce the same outputs 99.99999999999...% of the time).[1] https://stackoverflow.com/questions/10434326/hash-collision-... There's essentially no chance that the string "foo\n" fell into that tiny probability of difference. The reason there's a difference is because before git hashes something, git will do various processing to it (maybe appending and prepending various things) and those things broke the care
Read the rest
fully created collision. But a chosen-prefix attack might mean those various things can be accounted for, and a collision could still be found. So we need to directly run hardened SHA1 on the data, which I believe is located at https://github.com/cr-marcstevens/sha1collisiondetection As seen in https://github.com/git/git/blob/master/sha1dc_git.c So I tested that one: $ sha1collisiondetection-master/bin/sha1dcsum bar baz messageA messageB shattered-1.pdf shattered-2.pdf f1d2d2f924e986ac86fdf7b36c94bcdf32beec15 bar f1d2d2f924e986ac86fdf7b36c94bcdf32beec15 baz 4f3d9be4a472c4dae83c6314aa6c36a064c1fd14 *coll* messageA 9ed5d77a4f48be1dbf3e9e15650733eb850897f2 *coll* messageB 16e96b70000dd1e7c85b8368ee197754400e58ec *coll* shattered-1.pdf e1761773e6a35916d99f891b77663e6405313587 *coll* shattered-2.pdf So it does protect against the new attack.
In The first chosen-prefix collision for SHA-1
est31·Jan 7, 2020
The shattered prefix was chosen as well, see my other comment in the thread: https://news.ycombinator.com/item?id=21980759 The only thing that prefixing the length makes difficult is using the same prefix multiple times: you basically have to make up your mind about the type and length before mounting the shattered attack. Also, the prefix means you have to do your own shattered attack and can't use the PDFs that google provided as proof of their project's success. Price tag for that seems to be 11k. [1]: https://github.com/cr-marcstevens/sha1collisiondetection
In The first chosen-prefix collision for SHA-1
avar·May 14, 2019
> SHAtter was waived by many because the threat model didn't convincingly apply to them. Example: git. Git quickly switched to the sha1collisiondetection library[1] by default after the SHAttered attack was published. It's a SHA-1 library written by the authors of the paper which the attack. Edit: Marc Stevens saying that existing library will mitigate this new attack: https://twitter.com/realhashbreaker/status/11284190295369236... 1. https://github.com/cr-marcstevens/sha1collisiondetection
In A SHA-1 chosen-prefix collision attack
peff·Mar 20, 2017
It is possible; the researchers estimate the likelihood of a false positive at 2^-90 (which puts us back in "Sun engulfs the Earth" territory). There are metrics that will alert GitHub's infrastructure team if a collision is found (to confirm that we aren't seeing any false positives). Those metrics were quietly shipped (without the matching "die") for a week before flipping the final switch. If you want to know more about the patterns, see the sha1collisiondetection project: https://github.com/cr-marcstevens/sha1collisiondetection There's a research paper linked in the README.
In SHA-1 Collision Detection on GitHub.com

Showing the 15 most recent of 19 sources. Enable JavaScript to search and browse every indexed mention.

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.