puppeteer-lambda-starter-kit

Starter Kit for running Headless-Chrome by Puppeteer on AWS Lambda.

Explore on GitHub Read Hacker News discussions

2Distinct people

2Mentions

1Explicit recommenders

576GitHub stars

REPOSITORY AT A GLANCE

What to know before exploring

What it is

Starter Kit for running Headless-Chrome by Puppeteer on AWS Lambda.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

I've just finished my research on web scraping for my company (took me about 7 days). I started with import.io and scrapinghub.com for point and click scraping to see if I…

deathemperor used a recommendation phrase. One indexed comment matches. This is a phrase match, not a verdict.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by week2 dated mentions

Observed mentions: 2017-11-15 03:17:08 UTC to 2018-01-05 00:20:26 UTC. Weeks start Monday, UTC.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges

UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
Period	Start (UTC)	End (exclusive, UTC)	Mentions	Comments
Nov 13	2017-11-13 00:00:00 UTC	2017-11-20 00:00:00 UTC	1	1
Nov 20	2017-11-20 00:00:00 UTC	2017-11-27 00:00:00 UTC	0	0
Nov 27	2017-11-27 00:00:00 UTC	2017-12-04 00:00:00 UTC	0	0
Dec 4	2017-12-04 00:00:00 UTC	2017-12-11 00:00:00 UTC	0	0
Dec 11	2017-12-11 00:00:00 UTC	2017-12-18 00:00:00 UTC	0	0
Dec 18	2017-12-18 00:00:00 UTC	2017-12-25 00:00:00 UTC	0	0
Dec 25	2017-12-25 00:00:00 UTC	2018-01-01 00:00:00 UTC	0	0
Jan 1	2018-01-01 00:00:00 UTC	2018-01-08 00:00:00 UTC	1	1

What Hacker News says

2 sources

ThatOtherPerson·Jan 5, 2018

That's already possible - Apex[0] provides a shim for running golang and Rust, and I'm running headless chrome on Lambda to take screenshots right now (see [1] for an example). [0]: http://apex.run/#runtime [1]: https://github.com/sambaiz/puppeteer-lambda-starter-kit

In Announcing Rust 1.23

deathemperor·Nov 15, 2017Recommendation phrase

I've just finished my research on web scraping for my company (took me about 7 days). I started with import.io and scrapinghub.com for point and click scraping to see if I could do it without writing codes. Ultimately, UI point and click scraping is for none-technical. There are many data you would find it hard to scrape. For example, lazada.com.my stores the product's SKU inside an attribute that looks like <div data-sku-simple="SKU11111"></div> which I couldn't get. import.io's pricing is also something. I need to pay $999 a month for accessing API data is just too high. So I decided to use scrapy, the core of scrapinghub.com. I haven't written much python before but scrapy was very easy to learn. I wrote 2 spiders and run on scrapinghub (their serverless cloud). Scrapinghub support jobs scheduling and many other things at a cost. I pre

Read the rest

fer scrapinghub because in my team we don't have DevOps. It also supports Crawlera to prevent IP banning, Portia for point and click (still in beta, it was still hard to use), and Splash for SPA websites but it's buggy and the github repo is not under active maintenance. For DOM query I use BeautifulSoup4. I love it. It's jQuery for python. For SPA websites I wrote a scrapy middleware which uses puppeteer. The puppeteer is deployed on Amazon Lambda (1m free request first 365 days, more than enough for scraping) using this https://github.com/sambaiz/puppeteer-lambda-starter-kit I am planning to use Amazon RDS to store scraped data.

In Ask HN: What are best tools for web scraping?