asgard

Asgard from Netflix[1] basically did what you are asking for. But it has been deprecated[2] and replaced with Spinnaker[3], which has a bit of a different focus. But really, with couple dozen VMs, you don't have a cloud management problem. I would take a step back and figure out what the problem is, and where you want to be. Why are people creating VMs? What are the problems with them creating VMs? How do you mitigate those problems? If it is just tracking, then tags + education can help solve the issue. If it is security and standardization, an opinionated deployment system could help. With a couple dozen VMs, I'd say you probably shouldn't be running any VMs you manage yourself, and you should be using fully manage solutions. Then focus on the tooling that makes those solutions efficient to use for your users, while meeting your securit

Over the years, I've heard a few podcasts stating that Netflix relies heavily on Grails/Groovy for infrastructure management tools (https://github.com/Netflix/asgard, etc). Wonder why that isn't mentioned?

Link to asguard: https://github.com/Netflix/asgard

Use Asgard from NetflixOSS (https://github.com/Netflix/asgard). Shameless project plug: for anyone using Hubot, easily get Asgard up and running, then interface with Asgard via Hubot (https://github.com/imperialwicket/hubot-asgard). * EDIT: that is, use Asgard because it does a great job of managing autoscaling groups.

I'll go too: I've been building a Hubot script for interfacing with Asgard from NetflixOSS. https://github.com/imperialwicket/hubot-asgard I think the chatbot mentality is a good one, as it forces you to script/automate things so that a chatbot can accomplish them. That level of automation is a good target imo. It's also a great way to be certain that you can solve many issues remotely - since you can generally provide a meaningful chatbot command via mobile device. In terms of scaling and cloud (aws) management, Asgard is awesome (seriously - if you're on aws and haven't checked it out, go now: https://github.com/Netflix/asgard/wiki). The biggest complaint I have about Asgard as a tool, is it feels like more than most organizations need; because it is. My goal with the hubot-asgard scripts is to wrap some of the more essential features,

Reliability: Amazon makes no guarantee as to the availability / down time / safety of EC2 EC2 Does have SLAs - http://aws.amazon.com/ec2-sla/ Security: Amazon does not makes any guarantee as to whom it will disclose your data Please look at the Security and Data privacy section of the agreement - http://aws.amazon.com/agreement/ Persistence: ensuring persistence of your data (that includes, effort to set up the system) is complicated over EC2 Not sure what this pertains to. Ephemeral storage ? Permanent storage being backed up to S3 ? Management: there are very few integrated management tools for a cloud deployed on EC2 Amazon's web management tool for EC2 is pretty good. Netflix open sourced their tool Asgard which does precisely what this - https://github.com/Netflix/asgard. Perhaps Asgard was not available at the time this comment was

The baked AMI comes up and pulls, so it doesn't need to be baked each time. If you take the route you just mentioned, I highly recommend using Netflix's Asgard (https://github.com/Netflix/asgard) and check out their recently released Aminator (https://github.com/Netflix/aminator). Asgard specifically makes AMI based deploys outrageously easy.