translators

Since the formal semantic failed to become massively used, everyone is trying to make sense of the Web using scraping. It's something most viable sites should want to get behind, from ecommerce where you can find the best price faster to education. But I suspect if a very good shared scraping system were created we'd see countermeasures pretty fast, from timid content producers, as well as the Googles who want to have exclusive access to this level of information. Yet it's a natural layer for a shared commons, and what can be built on that, when finally get past the mess of arbitrary representations, is very exciting. Anyone know of any standard to create interoperable scrapers? In Javascript, of course. Here's one submission: https://github.com/zotero/translators

> I've been building a library of ScraperJSON definitions for academic publisher sites, and I've converged on some formats that work for a majority of publishers with no modification (because they silently follow undocumented standards like the HighWire metadata). We've got a growing community of volunteers who will keep the definitions up to date for hundreds or thousands of journals. If you also use our scraper definitions for your metadata you'll get all the publishers for free. My approach has been to use the Zotero translators, since their 200,000 users have been alright at responding to publisher site changes. Unfortunately they are trapped in the Firefox ecosystem until someone converts their translators to generic js. Then Zotero could be a downstream consumer of the same scrapers, but also maybe maintain them as well. zotero's sc