noseyparker

ing app using the excellent Textual TUI framework [2] and DuckDB [3] to build fast faceted search. A couple weeks later I had Nosey Parker Explorer. Nosey Parker Explorer proved very effective at the dataset construction task. My team of a few people used it to label about 15k examples collected from 2TB of input. However, beyond dataset creation, Nosey Parker Explorer was a huge boon for security engineers on engagements. Once you are dealing with more than a few dozen possible findings, it is _hugely_ helpful to be able to interactively slice-and-dice the data. The largest-scale use of it that I'm aware of was an assumed-breach engagement where we had tens of thousands of potential findings from 20TB of scanned inputs. Nosey Parker Explorer was far too useful to be left as a proof-of-concept or only used internally. A couple weeks ago I released it under the Apache 2 license. If you want to try it, It's a Python application (not yet on PyPI). There are prebuilt zipapp releases, but you can also clone the repository and `pip install`. You will first need to use Nosey Parker to scan something. See the project's README for details. Happy to answer questions. [1] Nosey Parker: https://github.com/praetorian-inc/noseyparker [2] Textual: https://textual.textualize.io [3] DuckDB: https://duckdb.org

a public Git repo, and saw that numerous IPs started trying those credentials less than 5 minutes after I pushed. > any theoretical gains from forcing login are dangerous to count on as vulnerable projects are still vulnerable Indeed, requiring login is not going to _solve_ the problem. But it could lessen the impact of leaked credentials at large scale, by making it more difficult for automated systems to harvest them. Perhaps more significantly, requiring login could give better audit trails in incidence response situations, as the logs would indicate which accounts were searching for secrets. > I think the solution to projects with hard coded credentials are to make the credentials easier to find and exploit so they fixed more quickly after being created Yes, this can help! There are several other companies that specialize in secret detection. I've also written Nosey Parker, a fast regex-based detection tool that has higher-precision rules than similar tools [4]. GitHub also has its own offering in Advanced Security to address this problem. [1] https://www.reuters.com/article/uk-uber-tech-lyft-hacking-ex... [2] https://www.securonix.com/blog/securonix-threat-research-ube... [3] https://news.ycombinator.com/item?id=28770590 [4] https://github.com/praetorian-inc/noseyparker

lots of the public content available on GitHub. Also, to put some color on this use case: in offensive security engagements (aka "red team" engagements) at Praetorian, we frequently find leaked credentials or tokens on GitHub or elsewhere, which allow us deeper access into the client's systems. It's a significant problem.

pared to similar tools. Nosey Parker is fast: it can scan 100GB of Linux kernel commit history in just over a minute on my laptop. This speed comes from several factors, but most significantly from using the amazing Hyperscan library for simultaneous matching of all regex rules in a single pass. In comparison with similar tools on large inputs, Nosey Parker is usually 1-3 orders of magnitude faster. Nosey Parker was originally created to help construct a labeled dataset of secrets for machine learning purposes. But it proved surprisingly useful on its own. In the past year, an internal, proprietary version (with added machine learning capabilities) has been regularly used in security engagements at Praetorian. In late 2022, Nosey Parker was reimplemented in Rust, released as open-source, and presented at Black Hat Arsenal. It now supports enumeration and scanning of GitHub repositories by providing just usernames or organization names. It also recently got support for SARIF output, which several other tools understand.