HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
mishushakov
llm-scraper
TypeScript2024–2025
mishushakov / llm-scraper

llm-scraper

Turn any webpage into structured data using LLMs

AI & Machine LearningWeb DevelopmentAutomation & Workflows
Explore on GitHubRead Hacker News discussions
3Distinct people
5Mentions
0Explicit recommenders
7KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Turn any webpage into structured data using LLMs

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month5 dated mentions

Observed mentions: 2024-04-20 20:37:36 UTC to 2025-05-12 00:37:26 UTC.

0122024-04-01 00:00:00 UTC to 2024-05-01 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 commentsApr 20242024-05-01 00:00:00 UTC to 2024-06-01 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 comments2024-06-01 00:00:00 UTC to 2024-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-07-01 00:00:00 UTC to 2024-08-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-08-01 00:00:00 UTC to 2024-09-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-09-01 00:00:00 UTC to 2024-10-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-10-01 00:00:00 UTC to 2024-11-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsOct 20242024-11-01 00:00:00 UTC to 2024-12-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-12-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-01-01 00:00:00 UTC to 2025-02-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025-02-01 00:00:00 UTC to 2025-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-03-01 00:00:00 UTC to 2025-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-04-01 00:00:00 UTC to 2025-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-05-01 00:00:00 UTC to 2025-06-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsMay 2025

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Apr 20242024-04-01 00:00:00 UTC2024-05-01 00:00:00 UTC110
May 20242024-05-01 00:00:00 UTC2024-06-01 00:00:00 UTC211
Jun 20242024-06-01 00:00:00 UTC2024-07-01 00:00:00 UTC000
Jul 20242024-07-01 00:00:00 UTC2024-08-01 00:00:00 UTC000
Aug 20242024-08-01 00:00:00 UTC2024-09-01 00:00:00 UTC000
Sep 20242024-09-01 00:00:00 UTC2024-10-01 00:00:00 UTC000
Oct 20242024-10-01 00:00:00 UTC2024-11-01 00:00:00 UTC000
Nov 20242024-11-01 00:00:00 UTC2024-12-01 00:00:00 UTC000
Dec 20242024-12-01 00:00:00 UTC2025-01-01 00:00:00 UTC000
Jan 20252025-01-01 00:00:00 UTC2025-02-01 00:00:00 UTC101
Feb 20252025-02-01 00:00:00 UTC2025-03-01 00:00:00 UTC000
Mar 20252025-03-01 00:00:00 UTC2025-04-01 00:00:00 UTC000
Apr 20252025-04-01 00:00:00 UTC2025-05-01 00:00:00 UTC000
May 20252025-05-01 00:00:00 UTC2025-06-01 00:00:00 UTC101

What Hacker News says

5 sources
ericciarla·May 1, 2024Submission

Show HN: Extracting structured data from the web with LLMs

Hey HN! Caleb, Nick, Garrett, and I from Mendable (YC S22) are excited to launch Intelligent Extraction for FireCrawl, the developer platform for scraping, search, and extraction. After a successful twitter launch last week, FireCrawl skyrocketed to over 2k stars and we have been getting a ton of feature requests [1]. One that stood out to us in particular was using the data we scrape to extract different types of structured metadata. Think querying “Is this company open source?” to a list of URLs and getting structured JSON back. Here’s a taste of what the request format / response looks like what scraping and extracting data from Firecrawl.dev Request format: { "company_mission": { "type": "string" }, "supports_sso": { "type": "boolean" }, "is_open_source": { "type": "boolean" } }, "required":
Read the rest
[ "company_mission", "supports_sso", "is_open_source" ] } Response format: { "company_mission":"transform any website into clean, LLM-ready markdown", "Supports_sso":false, "Is_open_source":true } The technical implementation for Intelligent Extraction involved: 1. Use Firecrawl to gather content as markdown 2. Use gpt-4 function calling to boil content down into a structured format. Inspiration was drawn from Simon Willison [2] and Mish Ushakov of llm-scraper [3]: This is just the beginning as we launched FireCrawl about a week ago so we expect a great deal of work will be required to make this as reliable and extendable as we envision! Any feedback would be highly appreciated [4]. [1] https://github.com/mendableai/firecrawl [2] https://til.simonwillison.net/gpt3/openai-python-functions-d... [3] https://github.com/mishushakov/llm-scraper/ [4] https://console.algora.io/org/mendableai
4 points · Open discussion

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.