flintrock

> * AWS has a managed Spark offering called EMR There is also my rinky-dink open source project, Flintrock [0], that will launch open source Spark clusters on AWS for you. It's probably not the right tool for production use (and you would be right to wonder why Flintrock exists when we have EMR [1]), but I know of several companies that have used Flintrock at one point or other in production at large scale (like, 400+ node clusters). [0]: https://github.com/nchammas/flintrock [1]: https://github.com/nchammas/flintrock#why-build-flintrock-wh...

Quite a few out there but the problem is that of fragmentation. There have been too many API changes making following online resources tricky. Having said that, I got great value out of these resources. Make sure you are working on the right version of Spark. https://jaceklaskowski.gitbooks.io/mastering-apache-spark/ https://pages.databricks.com/definitive-guide-spark.html Databricks cloud workspace (community edition) offers free interactive spark clusters where you can practice or develop your skills https://community.cloud.databricks.com/ If you want a bit of flexibility, you can launch your own cluster using flinktrock and a pair of free tier t2.micro EC2 instances https://github.com/nchammas/flintrock

> as long as setting up a cluster is a bit easier! If you're on AWS, it's already quite easy to set up a cluster today, no? There's EMR of course, and there are tools like spark-ec2 [0] and Flintrock [1]. There are a few more tools listed on Spark Packages that target different cloud providers [2], too. Disclaimer: I am the primary author of Flintrock and am a contributor to spark-ec2. [0] https://github.com/amplab/spark-ec2 [1] https://github.com/nchammas/flintrock [2] https://spark-packages.org/?q=tags%3Adeployment