SIsimonpure·Sep 5, 2020Submission
Ferret – Declarative Web Scraping
2 points · Open discussionMontFerret
ferret
Go2018–2020
MontFerret / ferret
ferret
Declarative data automation language and Go runtime for structured extraction workflows.
2Distinct people
5Mentions
0Explicit recommenders
6KGitHub stars
What to know before exploring
What it is
Declarative data automation language and Go runtime for structured extraction workflows.
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 5:04:57 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by month5 dated mentions
Observed mentions: 2018-10-02 15:59:34 UTC to 2020-09-05 03:17:50 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| Oct 2018 | 2018-10-01 00:00:00 UTC | 2018-11-01 00:00:00 UTC | 4 | 1 | 3 |
| Nov 2018 | 2018-11-01 00:00:00 UTC | 2018-12-01 00:00:00 UTC | 0 | 0 | 0 |
| Dec 2018 | 2018-12-01 00:00:00 UTC | 2019-01-01 00:00:00 UTC | 0 | 0 | 0 |
| Jan 2019 | 2019-01-01 00:00:00 UTC | 2019-02-01 00:00:00 UTC | 0 | 0 | 0 |
| Feb 2019 | 2019-02-01 00:00:00 UTC | 2019-03-01 00:00:00 UTC | 0 | 0 | 0 |
| Mar 2019 | 2019-03-01 00:00:00 UTC | 2019-04-01 00:00:00 UTC | 0 | 0 | 0 |
| Apr 2019 | 2019-04-01 00:00:00 UTC | 2019-05-01 00:00:00 UTC | 0 | 0 | 0 |
| May 2019 | 2019-05-01 00:00:00 UTC | 2019-06-01 00:00:00 UTC | 0 | 0 | 0 |
| Jun 2019 | 2019-06-01 00:00:00 UTC | 2019-07-01 00:00:00 UTC | 0 | 0 | 0 |
| Jul 2019 | 2019-07-01 00:00:00 UTC | 2019-08-01 00:00:00 UTC | 0 | 0 | 0 |
| Aug 2019 | 2019-08-01 00:00:00 UTC | 2019-09-01 00:00:00 UTC | 0 | 0 | 0 |
| Sep 2019 | 2019-09-01 00:00:00 UTC | 2019-10-01 00:00:00 UTC | 0 | 0 | 0 |
| Oct 2019 | 2019-10-01 00:00:00 UTC | 2019-11-01 00:00:00 UTC | 0 | 0 | 0 |
| Nov 2019 | 2019-11-01 00:00:00 UTC | 2019-12-01 00:00:00 UTC | 0 | 0 | 0 |
| Dec 2019 | 2019-12-01 00:00:00 UTC | 2020-01-01 00:00:00 UTC | 0 | 0 | 0 |
| Jan 2020 | 2020-01-01 00:00:00 UTC | 2020-02-01 00:00:00 UTC | 0 | 0 | 0 |
| Feb 2020 | 2020-02-01 00:00:00 UTC | 2020-03-01 00:00:00 UTC | 0 | 0 | 0 |
| Mar 2020 | 2020-03-01 00:00:00 UTC | 2020-04-01 00:00:00 UTC | 0 | 0 | 0 |
| Apr 2020 | 2020-04-01 00:00:00 UTC | 2020-05-01 00:00:00 UTC | 0 | 0 | 0 |
| May 2020 | 2020-05-01 00:00:00 UTC | 2020-06-01 00:00:00 UTC | 0 | 0 | 0 |
| Jun 2020 | 2020-06-01 00:00:00 UTC | 2020-07-01 00:00:00 UTC | 0 | 0 | 0 |
| Jul 2020 | 2020-07-01 00:00:00 UTC | 2020-08-01 00:00:00 UTC | 0 | 0 | 0 |
| Aug 2020 | 2020-08-01 00:00:00 UTC | 2020-09-01 00:00:00 UTC | 0 | 0 | 0 |
| Sep 2020 | 2020-09-01 00:00:00 UTC | 2020-10-01 00:00:00 UTC | 1 | 1 | 0 |
What Hacker News says
5 sourcesThat's what you can do right now :)
https://github.com/MontFerret/ferret/blob/master/docs/exampl...
Document, returned form DOCUMENT() function, represents an open browser tab which allows you to do all interactions with the page.
In Ferret – Declarative web scrapingIt can! :)
Even more - it can interact with these pages!
Here is an example of use of Google Search page:
https://github.com/MontFerret/ferret/blob/master/docs/exampl...
In Ferret – Declarative web scrapingYou are fine, I totally understand your scepticism. And you are right, there are definitely issues in communication.
First of all, I built it for myself. I needed a high level representation of scraping logic, which would run an isolated and safe environment.
Second, I needed to be able easily scrape dynamic pages.
So, what I got is:
- high level, declarative-is language, that hides all infrastructural details, which helps you to focus on the logic itself. that helps you to describe what you want without worring about underlying technology. Today, I'm using headless Chrome, tomorrow I will use something else, but the change should not affect your code.
- full support of dynamic pages. You can get data from dynamically rendered page, emulate user's actions and etc. Heck, you can even write bots with it.
- embeddable. now, I have only CLI,
Read the rest
there are plans to write a web server where you can save your scripts, schedule them and set up output streams.
But the main idea is to provide high level declarative way of scraping the web. I'm not saying you can't do that with other tools. I'm just trying to come up with something more easy to work with.
Regarding examples, the project is still WIP, so as more complex features I get, more complex examples I get.
Here is more or less complex, getting data from Google Search. It's not that difficult, but it showcases the core feature of work with dynamic pages.
https://github.com/MontFerret/ferret/blob/master/docs/exampl...
Ferret – Declarative web scraping
260 points · Open discussion