parkour

The big goal of this project seems to be generating Pig Code for Hadoop from Idiomatic Clojure. This seems quite similar to Parkour (Hadoop MapReduce in idiomatic Clojure) https://github.com/damballa/parkour/ The other Elephant in the room is Cascalog (generating Cascading Mapreduce queries for Hadoop using a Datalog language) http://cascalog.org/ It is interesting to note that the argument here "this makes writing Pig code so much easier" was also used by Nathan Marz when he was announcing Cascalog three years ago: http://www.slideshare.net/nathanmarz/clojure-at-backtype (at slide 42) The main distinguishing feature of this library compared to the other two seems to be that this one actually generates Pig code (although not for human consumption) as opposed to calling Hadoop or Cascading libraries so the benefit is 'runs where Pig runs'.

If your query is in Clojure there are some great tools in addition to those mentioned for data science that leverage Hadoop - in particular - Parkour https://github.com/damballa/parkour/ (Hadoop MapReduce in idiomatic Clojure) If your query is in R then you can convert to a PMML query and then Cascading to Run on Hadoop: http://blog.cloudera.com/blog/2013/11/how-to-use-cascading-p...

One of the really powerful things about functional programming is Higher Order Functions, like Map, Reduce, Filter and ZipWith. This style of programming is inherently scalable. This is seen in Google naming its distributed processing framework MapReduce. You can see a presentation about Clojure Reducers by Leonardo Borges here: http://www.slideshare.net/borgesleonardo/clojure-reducers-cl... The application of these ideas is used in Big Data scenarios. In Clojure we can leverage frameworks that use Hadoop to apply ideas about Higher Order Functions: * Cascalog http://cascalog.org/ (Expressive queries in datalog over the Cascading library) * Parkour https://github.com/damballa/parkour/ (Hadoop MapReduce in idiomatic Clojure)