Disclosure: I work on Google Cloud (but this applies generally).
There seem to be a lot of arguments about disk space (arguably free in 2020), memory (free-ish because as tytso points out, they won’t get backed) and then bandwidth.
I think most people saying “bandwidth” might mean “time to fetch”, because GCS, GCR, S3, etc. all have free egress on the transfer from them to GCE/EC2. If you have a self-hosted Docker Hub or something on an EC2 instance, that’s not the case (you may pay Zone to Zone egress of $.01/GB).
If you were paying a penny per gigabyte egress, a 100MB-ish binary is only .1 GB and therefore at best 1/1000th of a dollar. On the 16 vCPU hosts that CockroachDB prefers (see their recent benchmarks), that’s equivalent to about 5 seconds of runtime.
A fair retort is that in container land, this becomes death by 1000 cuts as ea
Read the rest
ch binary includes all the same stuff over and over again (for minikube, the .iso is nearly 1 GB [1] now, but not because of the 50MB binary itself).
Even so, a 1GB image takes almost as long to pull from a object store like GCS as a 100 MB image (at many Gbps, the constant factors dominate). If you’re trying to run something this large as a function on Cloud Run or Lambda or Knative, you’ll probably be sad (you’ve burned about 800 vcpu seconds, economically, of compute time) but that’s why there are layers.
tl;dr: your 50 or 100 MB binary doesn’t “cost” much, but a 1 GB container image without shared layers does.
[1] https://github.com/kubernetes/minikube/issues/5013