awesome-web-scraping

>I can scrape linkedin with a python script. That doesn't mean linkedin can shut down python. Well said! Also, what about copy-and-paste? The last time I checked, data could be highlighted in the browser, copied, and pasted to another application... Does that mean that LinkedIn can shut down the copy-and-paste capability of someone's browser and/or operating system? What about the "Save Page As..." functionality (the ability of a browser to save a page offline?) Can LinkedIn shut down "Save Page As..." ? Also, what about the Print Screen (take a screen snapshot) capabilities of someone's operating system? Can LinkedIn shut down that? Finally, there's literally oodles of software that can be used for web scraping; what follows below is just one non-canonical list: https://github.com/lorien/awesome-web-scraping Is LinkedIn going to shut do

This popped in my GitHub feed. There's an "awesome web-scraping" repo with ~5K stars. A few days ago the maintainer added the Russian flag at the top of README.md Someone raised this as an issue [0] "Just a suggestion, having a Russian flag at the top of the page might not be the best idea?" to which the maintainer simply replied "Why not? I am Russian citizen and I support my country." Seems trivial, but got me thinking of the implications. E.g. how would a Ukrainian dev feel having his project showcased in this list, under the Russian flag? [0] https://github.com/lorien/awesome-web-scraping/issues/136

Here is a place to start: https://github.com/lorien/awesome-web-scraping/blob/master/j... You might want to check out the Nightmare project (https://github.com/segmentio/nightmare) which makes automating an Electron instance pretty simple.

And an "awesome list" of web scraping open source tools: https://github.com/lorien/awesome-web-scraping