pipe2py

`riko` is pure python stream processing library for analyzing and processing streams of structured data. It's modeled after Yahoo! Pipes [1] and was originally a fork of pipe2py [2]. It has both synchronous and asynchronous (via twisted) APIs, and supports parallel execution (via multiprocessing). Out of the box, `riko` can read csv/xml/json/html files; create text and data based flows via modular pipes; parse and extract RSS/ATOM feeds; and bunch of other neat things. You can think of `riko` as a poor man's Spark/Storm... stream processing made easy! Feedback welcome so let me know what you think! Resources: FAQ [3], cookbook [4], and ipython notebook [5] Quickie Demo: >>> from riko.modules import fetch >>> >>> stream = fetch.pipe(conf={'url': 'https://news.ycombinator.com/rss'}) >>> item = next(stream) >>> item['tit

Hey gang! As some of you may have noticed, Yahoo! Pipes recently went dark (RIP). A speed-reading service I developed and use daily, Glance.wtf, was built on top of Yahoo! Pipes, and now I'm looking for a Pipes-compatible replacement. Fortunately, I was able to extract the Pipe-bodies before Pipes died, and those are available here: https://github.com/GlanceApps/PipeIDs/tree/master/pipes There are two interesting solutions I can see: pipe2py (https://github.com/ggaughan/pipe2py) + Google App Engine and pipes2js (https://github.com/neyric/pipes2js) + AWS SWF/Lambda. But really, I'd just like to get my service back up again with as little time and money being required as possible. I'm hoping that somebody else out there has already come up with a hosted Pipes replacement that I can simply piggy-back on. Does anybody know of anything like th

I would first look at zapier (as mentioned first by robgolding) and IFFT ( http://ifttt.com/ ), they can help integrate just about every service under the sun... if you want to host your own yahoo pipes clone here are a few: http://github.com/cantino/huginn http://github.com/ggaughan/pipe2py http://github.com/neyric/pipes2js https://github.com/olviko/RssPercolator http://github.com/superfeedr/superpipes Here also is a MEGA-LIST (and i wouldn't be using caps nor the word mega if I didn't mean it) of yahoo pipe replacements.... not my site, not my lists but clearly this guy did his homework (over 200+ options). http://meta-guide.com/integration/alternatives-to-yahoo-pipe... GL

Yahoo pipes was obviously influenced by UNIX pipes and as such is one of the best demonstrations of flow based programming to date. I really loved Yahoo pipes and it influenced the way I think. Pipes was a basic web "agent". It made basic programming available to the everyman much like HyperCard. Perhaps is just needed a runtime UI that matched the excellence of its design time interface. Yahoo is focussing on mobile yet here they had this custom agent building tool that could easily be re-purposed to mobile to make a killer platform for Yahoo users. Hopefully they will opensource the first generation perl version. Some of my favourite flow based programming links follow. Surprised nobody has mentioned noflow node.js workflows: https://github.com/noflo/noflo Also this (untested) will convert your yahoo pipes into Node.js https://g

Does everybody trust into these hosted services? (I do not.) How about to run your own one? - http://pipes.deri.org - still alive or dead? - https://github.com/ggaughan/pipe2py - looks alive - http://neyric.github.com/webhookit/docs/index.html - Alpha and dead? - swiftly.org - started as a open source project, but nothing released and dead? Did I forgot something/somebody? Just to be complete, GUI as a JS lib: - http://jsplumb.org/jquery/demo.html - http://neyric.github.com/wireit And additionally two commercial hosted projects: - http://www.wewiredweb.com (based on http://www.mashablelogic.com) - http://tarpipe.com/