reshape

Although useful, I think migration linters like this one don't give enough peace of mind. I have maintained a zero-downtime schema migration tool for several years now that tries to cover all the different ways one can shoot oneself in the foot: https://github.com/fabianlindfors/reshape It ensures migrations don't lock the database but maybe more importantly, it allows zero-downtime rollouts for your application as well by supporting both the old and new schema during the deployment, and automatically data between them. It also handles backfills and more that usually require multiple, separate deployments when using standard SQL commands.

Certainly complicated! This is one of those places where I don't think Postgres compatibility is achievable or even desirable, schema changes in a distributed databases are a whole different beast (and very complicated in large Postgres deployments as well). It will probably end up being something similar to online schema changes in Cockroach where the changes that can be made are limited for safety and run as a background job that can be tracked: https://www.cockroachlabs.com/docs/stable/online-schema-chan... I've got a bit of experience with schema changes from building Reshape: https://github.com/fabianlindfors/reshape. I'm hoping to transfer over some concepts from there into pgfdb eventually!

My project Reshape (https://github.com/fabianlindfors/reshape) might be interesting to you, it's a bit like blue/green DB deployments in that it uses a two-step process where it can support both the old and the new schema at the same time during the deployment phase.

For people interested in this subject, you might also be interested in my project Reshape: https://github.com/fabianlindfors/reshape. It takes database migrations a step further than most tools, guaranteeing zero-downtime migration for both schema and data changes.

These pitfalls are one reasons why I built Reshape [0], which aims to automate zero-downtime schema migrations. I can’t say it avoids all of them but we are working on a new product that would. If you are interested in this space (and Postgres specifically), I’d love to hear from you: fabian@reshapedb.com [0] https://github.com/fabianlindfors/reshape

The database aspect of this problem is particularly interesting to me. I’ve previously built Reshape [0], a zero-downtime migration tool for Postgres, and am now working on ReshapeDB [1], a full database designed from the ground up to tackle this problem. [0] https://github.com/fabianlindfors/reshape [1] https://reshapedb.com

Congrats to the lovely Xata team on another great launch! I'm the creator of Reshape [0], a similar tool which inspired pgroll, and seeing the concept taken even further is really exciting. [0] https://github.com/fabianlindfors/reshape

Cool stuff! Do you have any thoughts about how this compares to https://github.com/fabianlindfors/reshape?

best to use tools for this such as https://github.com/fabianlindfors/reshape or gh-ost for mysql

This is precisely what I did for my automated zero-downtime migration tool and it works pretty well: https://github.com/fabianlindfors/reshape. At least for Postgres, simple views like these have almost no overhead as queries are simply rewritten for the underlying table.

Anybody interested in this subject might also be interested in a tool for Postgres I’ve been working on, Reshape: https://github.com/fabianlindfors/reshape. It aims to fully automate away all the pain and manual steps zero-downtime migrations normally requires :)

Nice tool! Shadow tables are quite convenient as they support arbitrary ALTER TABLE statements but they also seem a bit wasteful to me. I'm working on a similar tool called Reshape (http://github.com/fabianlindfors/reshape) which instead uses temporary columns to reduce the time a migration takes and the extra storage necessary.