buildpacks-ci

> If you deal with Heroku, OpenStack, Cloud Foundry, etc. there is a ton of shell all over the place. With buildpacks, Travis CI, etc. Cloud Foundry Buildpacks were rewritten in Go and the (quite elaborate) automation for manufacturing them[0] is mostly test-driven Ruby these days. I can't claim credit for the former but I had a hand in the latter. Bash is not a sane production language. [0] https://github.com/cloudfoundry/buildpacks-ci (though I note someone snuck in some Crystal and I suspect it was Dave).

We (buildpacks team) get some of the way by ensuring that all secrets in our logs are redacted -- we actually wrote a rough-and-ready tool (concourse-filter[0]) for this purpose. It works on a whitelist principle. Any environment variable emitted to stdout or stderr is redacted unless it appears on a whitelist[1]. You're right that in the longer run, providing per-test keys will be the safest option. It's on our radar as part of the overall "3 Rs" effort[2]. [0] https://github.com/pivotal-cf-experimental/concourse-filter [1] https://github.com/cloudfoundry/buildpacks-ci/blob/1c345c30e... [2] https://medium.com/built-to-adapt/the-three-r-s-of-enterpris...

IBM sponsor a fleet of POWER-based systems at the OSU Open Source Lab[1]. Edit: You already said this ("Or, they could, like IBM provide SSH access to interested projects."), and if you'll excuse me, I'm going to go hide in shame. In my day job we've interacted with an IBM team who are porting our entire buildpacks pipeline[2][3] (which uses Concourse) to run on ppc64le. We fall under the Cloud Foundry heading the list of projects. The eventual goal is that we will be able to run x86 workers (on a regular commercial cloud) and some POWER workers at OSU-OSL or SoftLayer, and build both kinds of binaries from the same pipeline. I believe the eventual eventual goal is that all Cloud Foundry pipelines and products will be fully available across both x86 and ppc64le, including first-class integration with any pipeline producing binaries. Given

IBM are also providing hardware and engineering to get Cloud Foundry running on POWER8. I work for Pivotal, we donate the majority of engineering on CF, but IBM is the next biggest contributor. They've started assigning engineers to work alongside multiple teams on porting to POWER8 and have started setting up test clusters as well. Just this week I was in a remote tri-pairing session with an IBM engineer working on getting our buildpacks pipelines[0] running on POWER8. I personally can't wait to see how it performs. [0] https://github.com/cloudfoundry/buildpacks-ci

Surely. An index of all our pipelines are located here: https://github.com/cloudfoundry/buildpacks-ci#pipelines. Of note, the PR resource which JT built: https://github.com/jtarchie/pullrequest-resource

Full Disclosure: I also work at Pivotal on a team that has done a large amount of automation using Concourse (check out our pipelines [1]) Things that we've done so far with Concourse: * A dependency check notifier that based on an RSS feed from various language maintainers: Sends an email, updates our pivotal tracker account, sends a message to our slack channel * A job, that based on the completion of a build will upload the resulting artifact to Github and tag the release with a new version, auto generated release notes and checksums * Travis.ci style github pull request analysis * A security feed monitor that notifies us when a part of our application is vulnerable and when it is, kicks off a build and sends an email notification * Automatically updates submodules and gems for git repos and runs specs and tests against the updated art