github

gh-ost

Go2016–2025

github / gh-ost

gh-ost

GitHub's Online Schema-migration Tool for MySQL

Data & Databases

Explore on GitHub Read Hacker News discussions

36Distinct people

42Mentions

1Explicit recommenders

13.6KGitHub stars

REPOSITORY AT A GLANCE

What to know before exploring

What it is

GitHub's Online Schema-migration Tool for MySQL

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

We use gh-ost at Harvest[1] and it's a dream in comparison to manually migrating on a replica and switching master/slave roles [2]. Also the linked post[3] in the readme hit us…

zmoazeni used a recommendation phrase. One indexed comment matches. This is a phrase match, not a verdict.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 2:17:07 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year42 dated mentions

Observed mentions: 2016-08-02 06:23:47 UTC to 2025-03-01 21:37:46 UTC.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges

UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
Period	Start (UTC)	End (exclusive, UTC)	Mentions	Submissions	Comments
2016	2016-01-01 00:00:00 UTC	2017-01-01 00:00:00 UTC	1	0	1
2017	2017-01-01 00:00:00 UTC	2018-01-01 00:00:00 UTC	2	0	2
2018	2018-01-01 00:00:00 UTC	2019-01-01 00:00:00 UTC	12	2	10
2019	2019-01-01 00:00:00 UTC	2020-01-01 00:00:00 UTC	1	1	0
2020	2020-01-01 00:00:00 UTC	2021-01-01 00:00:00 UTC	2	0	2
2021	2021-01-01 00:00:00 UTC	2022-01-01 00:00:00 UTC	7	0	7
2022	2022-01-01 00:00:00 UTC	2023-01-01 00:00:00 UTC	8	0	8
2023	2023-01-01 00:00:00 UTC	2024-01-01 00:00:00 UTC	6	0	6
2024	2024-01-01 00:00:00 UTC	2025-01-01 00:00:00 UTC	2	1	1
2025	2025-01-01 00:00:00 UTC	2026-01-01 00:00:00 UTC	1	0	1

From the project README

An excerpt from the repository, fetched Nov 13, 2026.

Read the project’s introduction

gh-ost GitHub's online schema migration for MySQL gh-ost is a triggerless online schema migration solution for MySQL. It is testable and provides pausability, dynamic control/reconfiguration, auditing, and many operational perks. gh-ost produces a light workload on the master throughout the migration, decoupled from the existing workload on the migrated table. It has been designed based on years of experience with existing solutions, and changes the paradigm of table migrations. How? All existing online-schema-change tools operate in similar manner: they create a ghost table in the likeness of your original table, migrate that table while empty, slowly and incrementally copy data from your original table to the ghost table, meanwhile propagating ongoing changes (any INSERT, DELETE, UPDATE applied to your table) to the ghost table. Finally, at the right time, they replace your original table with the ghost table. gh-ost uses the same pattern. However it differs from all existing tools by not using triggers. We have recognized the triggers to be the source of many limitations and risks. Instead, gh-ost uses the binary log stream to capture table changes, and asynchronously applies them onto the ghost table. gh-ost takes upon itself some tasks that other tools leave for the database to perform. As result, gh-ost has greater control over the migration process; can truly suspend it; can truly decouple the migration's write load from the master's workload. In addition, it offers many operational perks that make it safer, trustworthy and fun to use. Highlights Build your trust in gh-ost by testing it on replicas. gh-ost will issue same flow as it would have on the master, to migrate a table on a replica, without actually replacing the original table, leaving the replica with two tables you can then compare and satisfy yourself that the tool operates correctly. This is how we continuously test gh-ost in production. True pause: when gh-ost throttles, it truly ceases writes on master: no row copies and no ongoing events processing. By throttling, you return your master to its original workload Dynamic control: you can interactively reconfigure gh-ost, even as migration still runs. You may forcibly initiate throttling. Auditing: you may query gh-ost for status. gh-ost listens on unix socket or TCP. Control over cut-over phase: gh-ost can be instructed to postpone what is probably the most critical step: the swap of tables, until such time that you're comfortably available. No need to worry about ETA being outside office hours. External hooks can couple gh-ost with your particular environment. Please refer to the docs for more information. No, really, read the docs. Usage The cheatsheet has it all. You may be interested in invoking gh-ost in various modes: a noop migration (merely testing that the migration is valid and good to go) a real migration, utilizing a replica (the migration runs on the master; gh-ost figures out identities of servers involved. Required mode if your master uses Statement Based Replication) a real migration, run directly on the master (but gh-ost prefers the former) a real migration on a replica (master untouched) a test migration on a replica, the way for you to build trust with gh-ost's operation. Our tips: Testing above all, try out --test-on-replica first few times. Better yet, make it continuous. We have multiple replicas where we iterate our entire fleet of production tables, migrating them one by one, checksumming the results, verifying migration is good. For each master migration, first issue a noop Then issue the real thing via --execute. More tips: Use --exact-rowcount for accurate progress indication Use --postpone-cut-over-flag-file to gain control over cut-over timing Get familiar with the interactive commands Also see: requirements and limitations common questions what if? the fine print Community questions Using gh-ost on AWS RDS Using gh-ost on Azure Database for MySQL What's in a name? Originally this was named gh-osc: GitHub Online Schema Change, in the likes of Facebook online schema change and pt-online-schema-change. But then a rare genetic mutation happened, and the c transformed into t. And that sent us down the path of trying to figure out a new acronym. gh-ost (pronounce: Ghost), stands for GitHub's Online Schema Transmogrifier/Translator/Transformer/Transfigurator License gh-ost is licensed under the MIT license gh-ost uses 3rd party libraries, each with their own license. These are found here. Community gh-ost is released at a stable state, but with mileage to go. We are open to pull requests. Please first discuss your intentions via Issues. We develop gh-ost at GitHub and for the community. We may have different priorities than others. From time to time we may suggest a contribution that is not on our immediate roadmap but which may appeal to others. Please see Coding gh-ost for a guide to getting started developing with gh-ost. Download/binaries/source gh-ost is now GA and stable. gh-ost is available in binary format for Linux and Mac OS/X Download latest release here gh-ost is a Go project; it is built with Go 1.15 and above. To build on your own, use either: script/build - this is the same build script used by CI hence the authoritative; artifact is ./bin/gh-ost binary. build.sh for building the release artifacts (tar.gz archives plus .rpm/.deb packages and a SHA256SUMS manifest) in /tmp/gh-ost-release Generally speaking, master branch is stable, but only releases are to be used in production. Authors gh-ost is designed, authored, reviewed and tested by the database infrastructure team at GitHub: @jonahberquist @ggunson @tomkrouper @shlomi-noach @jessbreckenridge @gtowey @timvaillancourt

Read the full README on GitHub

What Hacker News says

7 sources

shlomi-noach·Dec 6, 2021

Online schema change solutions have been around for over the past decade and are commonly used to ALTER TABLE with no downtime (or with minimal interruption) on the largest deployments of MySQL today. The two most common solutions are pt-online-schema-change and gh-ost, and if you are running MySQL today and still running direct ALTER TABLE suffering outage, then you're in for a pleasant change. On top of that, most MySQL ALTER TABLE operations with InnoDB tables support non-blocking, lockless operation as well. My main concern with these is that they're still replicated sequentially leading to replication lags. MySQL is also slowly adding "Instant DDL", currently still limited to just a few types of changes. Disclosure: I authored gh-ost (at GitHub), oak-online-alter-table (the original schema change tool) and am a maintainer for Vitess

Read the rest

and working on online schema changes in Vitess. Links: - https://www.percona.com/doc/percona-toolkit/3.0/pt-online-sc... - https://github.com/github/gh-ost - Past HN discussion: https://news.ycombinator.com/item?id=16982986 - https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-op... - https://vitess.io/docs/user-guides/schema-changes/ Edited for formatting.

In Leaving MySQL

shlomi-noach·Nov 17, 2021

Again, thank you for the questions. I am estimating that your database space isn't MySQL, which is just fine of course. Reason I'm asking/guessing, is that in the MySQL space, online schema change toold have been around for over a decade and are the go-to solution for schema changes. A small minority of the industry, based on my understanding as a member of the community, uses other techniques such as rolling migrations on replicas etc., but the vast majority uses one of the common schema change tools: - pt-online-schema-change - facebook's OSC - gh-ost I authored the original schema change tool, oak-online-alter-table https://shlomi-noach.github.io/openarkkit/oak-online-alter-t..., which is no longer supported, but thankfully I did invest some time in documenting how it works. Similarly, I co-designed and was the main author for gh-ost,

Read the rest

https://github.com/github/gh-ost, as part of the database infrastructure team at GitHub. We developed gh-ost because the existing schema change tools could not cope with our particular workloads. Read this engineering blog: https://github.blog/2016-08-01-gh-ost-github-s-online-migrat... to get better sense of what gh-ost is and how it works. I in particular suggest reading these: - https://github.com/github/gh-ost/blob/master/doc/cheatsheet.... - https://github.com/github/gh-ost/blob/master/doc/cut-over.md - https://github.com/github/gh-ost/blob/master/doc/subsecond-l... - https://github.com/github/gh-ost/blob/master/doc/throttle.md - https://github.com/github/gh-ost/blob/master/doc/why-trigger... At PlanetScale I also integrated VReplication into the Online DDL flow. This comment is far too short to explain how VReplication works, but thankfully we again have some docs: - https://vitess.io/docs/user-guides/schema-changes/ddl-strate... (and really see entire page, there's comparison between the different tools) - https://vitess.io/docs/design-docs/vreplication/ - or see this self tracking issue: https://github.com/vitessio/vitess/issues/8056#issue-8771509... Not to leave you with only a bunch of reading material, I'll answer some questions here: > Can you elaborate? How? Do they run on another servers? Or are they waiting on a queue change waiting to be applied? If they run on different servers, what they run there, since AFAIK the migration is only DDL, there's no data? The way all schema change tools mentioned above work is by creating a shadow aka ghost table on the same primary server where your original table is located. By carefully both copying data from original table as well as tracking ongoing changes to the table (whether by utilizing triggers or by tailing the binary logs), and using different techniques to mitigate conflicts between the two, the tools populate the shadow table with up-to-date data from your original table. This can take a long time, and requires an extra amount of space to accommodate the shadow table (both time and space are also required by "natural" ALTER TABLE implementations in DBs I'm aware of). With non-trigger solutions, such as gh-ost and VReplication, the tooling have almost ocmplete control over the pace. Given load on the primary server or given increasing replication lag, they can choose to throttle or completely halt execution, to resume later on when load has subsided. We have used this technique specifically at GitHub to run the largest migrations on our busiest tables at any time of the week, including at peak traffic, and this has show to pose little to no impact to production. Again, these techniques are universally used today by almost all large scale MySQL players, including Facebook, Shopify, Slack, etc. > who will throttle, the migration? But what is the migration? Let's use my example: a column type change requires a table rewrite. So the table rewrite will throttle, i.e. slow down? But where is this table rewrite running, on the main server (apparently not) or on a shadow server (apparently either since migrations have no data)? Actually you mention "when your production traffic gets too high". What is "high", can you quantify? The tool (or Vitess if you will, or PlanetScale in our discussion) will throttle based on continuously collecting metrics. The single most important metric is replication lag, and we found that it predicts load more than any other matric, by far. We throttle at 1sec replication lag. A secondary metric is the number of concurrent executing threads on the primary; this is mroe improtant for pt-online-schema-change, but for gh-ost and VReplication, given their nature of single-thread writes, we found that the metric is not very important to throttle on. It is also trickier since the threshold to throttle at depends on your time of day, particular expected workload etc. > We run customers that do dozens to thousands of transactions per second. Is this high enough? The tooling are known to work well with these transaction rates. VReplication and gh-ost will add one more transaction at a time (well, two really, but 2nd one is book-keeping and so low volume that we can neglect it); the transactions are intentionally kept small so as to not overload the transaction log or the MVCC mechanism; rule of thumb is to only copy 100 rows at a time, so exepect possibly millions of sequential such small transaction on a billion row table. > Will their migrations ever run, or will wait for very long periods of time, maybe forever? Some times, if the load is so very high, migrations will throttle more. At other times, they will push as fast as they can while still keeping to low replication lag threshold. In my experience a gh-ost or vreplication migration is normally good to run even on the busiest times. If a database system is such that it _always_ has substantial replication lag, such that a migration cannot complete in a timely manner, then I'd say the database system is beyond its own capacity anyway, and should be optimized/sharded/whatever. > How is this possible? Where the migration is running, then? A shadow table, shadow server... none? So I already mentioned the ghost table. And then, SELECTs are non blocking on the original table. > What's cut-over? Cut-over is what we call the final step of the migration: flipping the tables. Specifically, moving away your original table, and renaming the ghost table in its place. This requires a metadata lock, and is the single most critical part of the schema migration, for any tooling involved. This is where something as to give. Tooling such as gh-ost and pt-online-schema-change acquire a metadata lock such that queries are blocked momentarily, until cut-over is complete. With very high load the app will feel it. With extremely high load the database may not be able to (or may not be configured to) accommodate so many blocked queries, and app will see rejections. For low volume load apps may not even notice. I hope this helps. Obviously this comment cannot accommodate so much more, but hopefully the documentation links I provided are of help.

In PlanetScale is now generally available

aseipp·Nov 17, 2021

Every branch in PlanetScale has its own separate MySQL connection credentials first off (and conceptually just looks like a separate DB), so no. You'll be using your main production database as usual, and you'll have separate credentials for a branch where you can make schema changes like ALTER TABLE, and you "connect" to that branch with... whatever. Deploy a canary version of your app, use your SQL IDE, whatever. The main branch always prohibits changes like ALTER TABLE or CREATE DATABASE. Merging a branch where the CREATE/ALTER happened is the only way to bring changes to the production database that serves your users queries. When you merge a branch in PlanetScale, you can still write to the database while the migration is happening, online. This is done through "Online Schema Change", but the TL;DR is that a system in the background

Read the rest

creates the new database, reads the mysql binlogs from your original database, including all the writes, and propagates those writes to the new database with the changes applied. It does this continuously, in real time. Once the new database is caught up, the system switches over transparently to make the changes atomically appear live.[1] The requirement is that the running application server has to be able to handle both the old and new schema at the same time. This is easy if you do something like "add a column that isn't yet used and will be soon", but harder for "dropping a column that is in use". No database alone can solve this however, you have to coordinate your application layer correctly so that it can handle the switch over when you apply the changes. But this means you can have very high uptime with e.g. blue/green deployments and staged rollouts, and you'll never have to take your DB offline for it. It's also trite but "hundreds of writes a second" is nothing. Vitess can handle 100x that write load (probably much more) and deal with schema changes to petabyte-scale databases completely online. PlanetScale currently doesn't let you control the Vitess keyspace sharding configuration (yet) which is crucial to this but I suspect it will happen eventually. But they'll also let you run Vitess in your own K8S environment while they provide a control plane, so you wouldn't pay list price, anyway. I do not have any affiliation with PlanetScale but it's a pretty cool product I could see myself using for my low-volume needs, just because of the pay-as-you-go model where I don't pay for e.g. RDS allocation. And I say that as a really big PostgreSQL fan. [1] https://github.com/github/gh-ost

In PlanetScale is now generally available

evanelias·Mar 12, 2021

GitHub uses MySQL, not Postgres. They built the best-in-class online schema change tool gh-ost [1], and have a custom declarative schema change execution system built around Skeema [2], which contains a wealth of linters [3]. Even so, it's always possible for an engineer to submit a schema change which is detrimental to performance. For example, dropping an important index, or changing it such that some necessary column is no longer present. Linters simply cannot catch some classes of these problems, as they're application/workload-specific. Usually they must be caught in code review, but people make mistakes and could approve a bad change. Disclosure: I'm the author of Skeema, but have not worked for or with GitHub in any capacity. [1] https://github.com/github/gh-ost [2] https://github.blog/2020-02-14-automating-mysql-schema-migra... [3

Read the rest

] https://www.skeema.io/docs/options/#lint

In GitHub was down

srcreigh·Feb 19, 2021

SQLite did get support for renaming columns recently [1]. Anyways, the process you describe is also used in MySQL for doing online schema migrations. [2] "proper" database migrations cause downtime [1]: https://stackoverflow.com/questions/805363/how-do-i-rename-a... [2]: https://github.com/github/gh-ost

In Many small queries are efficient in SQLite

pizza234·Feb 12, 2021

I confirm the sibling post that adding a column is instaneous on MySQL 8 (although there are limitation), and point to the reference document: https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-op.... Outside the context of db comparisons, and in relation to the specific case, if you don't have triggers/foreign keys on a given MySQL table, Gh-ost¹ solves the DDL locking issues. ¹=https://github.com/github/gh-ost

In AWS Babelfish: The Elephant in the PostgreSQL Room?

t3rabytes·Jan 25, 2021

Re db migrations: they've built their own DB management tooling (https://github.com/openark/orchestrator) and online migration tooling (https://github.com/github/gh-ost)

In Improving how we deploy GitHub