bootkube

Running K8S on other environments isn't a difficult job, but you do need a certain level of experience running systems. You need to set up EtcD, in prod you have 3 or 5 of them - there are a couple of topologies for running EtcD and peer discovery (you will need to RTFM). This is all your K8S state, so you need to set up backups for these. Each node you are going to include in your cluster needs to have docker and kubelet running. You will need to start kubelet with your init system (systemd, etc). Then all you need is to bootstrap a control plane. The control plane has 3 components. The API server needs to communicate to etcd, everything else just talks to the API server. The API server is stateless and can (should) have multiple instances running. The other two parts are the Controller and Scheduler. Also stateless. The tool Bootkube[1]

running into bugs and that I need to ssh into the nodes ;) (serious answer) I can basically automate nearly everything, os updates (coreos), deployment updates (ci/cd), however updating etcd or k8s is a manual operation on bare metal, either I'm on bootkube than I need to login and update kubectl, than I need to update all daemonsets/deployments in kube-system (it's not always easy and bootkube has strange bugs, https://github.com/kubernetes-incubator/bootkube/issues/977) So one should actually use kubeadm, once again I still need to update kubelet, so I need to ssh into the machine (anyway) I also need to update kubeadm and run it, again a manual operation. *ssh means I need to do it on my own, for small clusters this can be done manually for bigger clusters it can be scripted via ansible or even better create an own software that does i

The thing about k8s is it sort of forces you into best practices, immutable images are built and run instead of ansible or config management duct-taping them together. So he replaced his ansible playbooks that build his apps with Dockerfiles How are you deploying and maintaining your k8s clusters? There is some bootstrapping for nodes but k8s is distro and cloud agnostic... Stateless golang binaries plus etcd. It seems like you should be singing it's praises in that regard on the provisioning front? Funny enough there has been some work on "self hosted kubernetes" where even the components themselves are inside k8s. Pretty cool will probably be the future of cluster bootstrapping: https://github.com/kubernetes-incubator/bootkube I'm curious for your reasons for saying k8s is overvalued. You are the first person I have seen that has worked

What would your ideal setup look like on baremetal? The Kubernetes on baremetal setup above is for full automation. You can do simpler manual installs with bootkube[1] or kubeadm[2]; and this will further simplifying over the next few months. [1] https://github.com/kubernetes-incubator/bootkube [2] http://kubernetes.io/docs/getting-started-guides/kubeadm/