cromwell

A very clever way to run genomic analysis on GCP is via WDL ( https://software.broadinstitute.org/wdl/ ) workflows executed on the Cromwell ( https://github.com/broadinstitute/cromwell ) service. Using WDL you define tasks, and compose those into workflows, then Cromwell executes those tasks on GCP using the pipelines api ( https://cloud.google.com/genomics/reference/rest/v1alpha2/pi... ). So you don't need to set up any instances at all, each stage of your pipeline, a task, is executed in its own docker instance, with files from google cloud storage being materialized on that instance for you. When that task is done, outputs are sent to cloud storage buckets, and everything is torn down. Picard can pull reads directly out of the google genomics API, and more of the Broad tools are adopting the GA4GH api, so this means an increasing amoun