go-readability

is would be good in a healthy ecosystem, but would only serve to further entrench Google's position in the current situation.) By the way, using a filtering/rewriting proxy has other merits, especially on said older hardware; you can rewrite the entire web page to make it more lightweight and accessible. Check out miniwebproxy[1] and medium-rare[2]. It's also quite simple to write one; you need maybe a hundred lines of Go to start getting results. I've been experimenting with integrating Readability[3][4]; and I think there's more potential to this approach. [1]: https://humungus.tedunangst.com/r/miniwebproxy [2]: https://humungus.tedunangst.com/r/medium-rare [3]: https://github.com/go-shiori/go-readability [4]: https://github.com/mozilla/readability

roject. Next, DomDistiller is extractor that used in Chromium. It uses Java language with whopping 14,000+ lines of code and can only be used as part of Chromium browser, so you can't exactly use it as standalone library or CLI. Finally, Trafilatura is a Python package released under GPLv3 license. Created in order to build a text databases[5] for NLP research, it mainly intended for German web pages. However, as development continues, it works really great with other languages. It's a bit slow though compared to Readability.js. All of those three work in similar way: extract metadata, remove unneeded contents, and finally returns the cleaned up content. Their differences (that I remembered) are: - In Readability, they insist to make no special rules for any website, while DomDistiller and Trafilatura give a small exception for popular sites like Wikipedia. Thanks to this, if you use Readability.js in Wikipedia pages, it will shows `[edit]` button thorough the extracted content. - Readability has a small function to detect whether a web page can be converted to reader mode. While it's not really accurate, it's quite convenient to have. - In DomDistiller, the metadata extraction is more thorough than the others. It supports OpenGraph, Schema.org, and even the old IE Reading View mark up tags. - Since DomDistiller is only usable within Chromium, it has the advantage to be able to use CSS styling to determine if an element is important or not. If an element is styled to be invisible (e.g. `display: none`) then it will be deemed unimportant. However, according to a research[6] this step is actually doesn't really affect the extraction result. - DomDistiller also has an experimental feature to find and extract next page in sites that separated its article to several partial pages. - For Trafilatura, since it was created for collecting web corpus, it main ability is extracting text and the publication date of a web page. For the latter, they've created a Python package named htmldate[7] whose only purpose is to extract the publication or modification date for a web page. - Trafilatura also has an experimental feature to remove elements that repeated too often. The idea is if the element occured too often, then it's not important to the reader. I've found benchmark[8] that compare the performance between the extractors, and it said that Trafilatura has the best accuracy compared to the others. However, before you start rushing to use Trafilatura, you should remember that Trafilatura is intended for gathering web corpus, so it's really great for extracting text content, but IIRC is not as good as Readability.js and DomDistiller for extracting a proper article with images and embedded iframes (depending on how you look, it could be a feature though). By the way, if you are using Go and need to use a web extractor, I already ported all three of them to Go[9][10][11] including their dependencies[12][13], so have fun with it. [1]: https://github.com/mozilla/readability [2]: https://github.com/chromium/dom-distiller [3]: https://github.com/adbar/trafilatura [4]: https://www.bbaw.de/en/ [5]: https://www.dwds.de/d/k-web [6]: https://arxiv.org/abs/1811.03661 [7]: https://github.com/adbar/htmldate [8]: https://github.com/scrapinghub/article-extraction-benchmark [9]: https://github.com/go-shiori/go-readability [10]: https://github.com/markusmobius/go-domdistiller [11]: https://github.com/markusmobius/go-trafilatura [12]: https://github.com/markusmobius/go-htmldate [13]: https://github.com/markusmobius/go-dateparser

line-mode browser that supports JavaScript via Duktape. Originally written for blind users, it's great for sighted users as well. => https://edbrowse.org/ edbrowse If you want to go the extra mile and ensure that your website works with the most basic HTML parsers, try sending it through an HTML-to-Markdown converter that doesn't include inline html in the Markdown output. I test with pandoc's strict markdown output and make sure the resulting markdown is readable: pandoc -f html -t markdown_strict https://example.com None of this is a replacement for real a11y testing; most a11y technology leverages mainstream browsers (Chromium, FF, Safari, et cetera).