nokogiri

I agree, and to go further, even _if_ a non-developer did on github (maybe higher in the search rankings), for many popular projects you can generally find the consumer facing url via the "homepage" link. Example: https://github.com/rack/rack homepage url points to http://rack.rubyforge.org/ https://github.com/tenderlove/nokogiri => http://nokogiri.org/ https://github.com/mxcl/homebrew => http://mxcl.github.com/homebrew/ https://github.com/joyent/node => http://nodejs.org/ etc, etc. It also seems fairly common to put the 'consumer' url near the top or bottom of the README.

Go take a look at the source code for ruby-rmagick, ruby-pg, nokogiri, or mongrel. https://github.com/rmagick/rmagick https://github.com/ged/ruby-pg https://github.com/tenderlove/nokogiri

Wow, this made my day. I've been using FireQuark (a custom firebug that provides CSS selectors - http://www.quarkruby.com/2007/9/5/firequark-quick-html-scree...) and while it was very helpful, SelectorGadget will remove sooo much more of the pain of scraping. Major kudos. As an aside, I'll mention the nokogiri gem (almost identical to hpricot, but under much more active development), that's what I've been using and it's excellent. http://github.com/tenderlove/nokogiri/tree/master

hpricot is great, but I hear lately that Nokogiri is faster and just as capable: http://github.com/tenderlove/nokogiri/tree/master