NUnullpt_rs·Nov 30, 2024
yep, but it can get tricky.
some projects worth checking out:
https://github.com/refraction-networking/utls
https://github.com/berstend/puppeteer-extra
In Breaking the 4Chan CAPTCHAberstend
puppeteer-extra
JavaScript2020–2024
berstend / puppeteer-extra
puppeteer-extra
💯 Teach puppeteer new tricks through plugins.
8Distinct people
8Mentions
0Explicit recommenders
7.4KGitHub stars
What to know before exploring
What it is
💯 Teach puppeteer new tricks through plugins.
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 5:04:57 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year8 dated mentions
Observed mentions: 2020-09-09 15:39:12 UTC to 2024-11-30 06:21:09 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2020 | 2020-01-01 00:00:00 UTC | 2021-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2021 | 2021-01-01 00:00:00 UTC | 2022-01-01 00:00:00 UTC | 2 | 0 | 2 |
| 2022 | 2022-01-01 00:00:00 UTC | 2023-01-01 00:00:00 UTC | 3 | 1 | 2 |
| 2023 | 2023-01-01 00:00:00 UTC | 2024-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2024 | 2024-01-01 00:00:00 UTC | 2025-01-01 00:00:00 UTC | 1 | 0 | 1 |
What Hacker News says
8 sourcesThere are even Puppeteer plugins that will do it for you. [^1]
The best detection I've come across so far (i.e. before this release) has just required I run headless Chrome in headed mode. Granted, I don't do a ton of scraping -- mostly just pulling data out of websites so that I can play with it in aggregate using more civilized tools.
[1]: https://github.com/berstend/puppeteer-extra/tree/master/pack...
In New headless Chrome has been released and has a near-perfect browser fingerprintPuppeteer-extra-plugin-stealth – plugin for puppeteer-extra to prevent detection
2 points · Open discussionThis extension takes an interesting approach to spoofing data, which is nice!
In my case, I’m interested in doing the same thing inside of Puppeteer for web scraping, unfortunately it seems like the only possible approach is similar to content scripts (for example https://github.com/berstend/puppeteer-extra/tree/master/pack...) which leads to it being easily detected. Are there any similar approaches that can be used for Puppeteer?
In Show HN: Browser extension that spoofs your location data to match your VPNThere are plugins for Puppeteer: https://github.com/berstend/puppeteer-extra/tree/master/pack...
In The State of Web Scraping 2022Unfortunately, the days of reliable non-JavaScript capable scraping are over.
Fortunately there are plenty of tools to handle this, and at a hobby level not particularly resource intensive. Something like this is simple and reliable in many cases: https://github.com/berstend/puppeteer-extra/tree/master/pack...
In Avoiding bot detection: How to scrape the web without getting blocked?The stealth plugin for Puppeteer Extra gives a pretty good idea of what you need to cover today. Maybe it's not rocket science, but it's not trivial either.
https://github.com/berstend/puppeteer-extra/tree/master/pack...
In Mimicking a device is becoming almost impossibleThe beautiful thing about web scraping is that it automates so much tedious work that the benefits in saved time offsets the brittleness inherent in scraping data.
And with Puppeteer (also Playwright) it's never been easier. Recaptcha solving, Ad blocking etc. in just a few lines of code[1].
I've built a business on the back of Puppeteer - https://simplescraper.io. Ten months in and we've just passed 100 customers so there's mucho opportunities in solving these kind of problems.
[1] https://github.com/berstend/puppeteer-extra/tree/master/pack...
In How we learnt to stop worrying and love web scraping