pgtestdb

Peter actually makes this exact point in the project's README. See this section here: https://github.com/peterldowns/pgtestdb#how-do-i-make-it-go-... I'd just say that a nice thing about it on disk (even if you disable fsync) is that in case of a failing test, you can examine the post-run state which is occasionally extremely valuable.

Really interesting article, I didn't know that the template cloning strategy was configurable. Huge fan of template cloning in general; I've used Neon to do it for "live" integration environments, and I have a golang project https://github.com/peterldowns/pgtestdb that uses templates to give you ~unit-test-speed integration tests that each get their own fully-schema-migrated Postgres database. Back in the day (2013?) I worked at a startup where the resident Linux guru had set up "instant" staging environment databases with btrfs. Really cool to see the same idea show up over and over with slightly different implementations. Speed and ease of cloning/testing is a real advantage for Postgres and Sqlite, I wish it were possible to do similar things with Clickhouse, Mysql, etc.

That's not always true — database-backed tests can be extremely fast, see https://github.com/peterldowns/pgtestdb.

Here's what I've found — https://github.com/peterldowns/pgtestdb#how-do-i-make-it-go-... If you come up with any better options please let me know so I can update this readme!

This is better than nothing but if you have to run migrations before your database is usable, you want an approach like the one I use in pgtestdb — use one server for all your tests, but give them each different databases by cloning from one that's already set up with all the schemas correctly migrated. https://github.com/peterldowns/pgtestdb

Don't know what language or database you use, but check this out: https://github.com/peterldowns/pgtestdb If you happen to use Postgres, the approach is ultimately portable: it uses Pg database templates (also, regarding perf, the author recommends using a ramdisk and turning off fsync on your test DBs; you'll see this in the project readme). But you’ll have to write the code yourself.

OK, that goal makes sense, thanks for explaining. For what it's worth I'm pretty sure you can do this with postgres, tmpfs, and template databases — see my project, pgtestdb [0]. I haven't done a formal perf measurement, but each test would get a fresh and migrated database in about 20ms on my local machine. The setup described runs postgres in a container for convenience, but you could probably also just run a postgres binary and store the data on a memory-backed filesystem. [0] https://github.com/peterldowns/pgtestdb

Selfishly, I'd love a documentation review if you (or anyone else) has the time to take a try out some golang projects I've been working on. The code is frankly not very polished and not worth reviewing, but I'm curious to hear feedback on the documentation / README and whether or not you clearly understand what these libraries are for and how to use them. * https://github.com/peterldowns/pgtestdb * https://github.com/peterldowns/testy * https://github.com/peterldowns/pgmigrate

This seems great but is actually quite slow. This will create a new container, with a new postgres server, and a new database in that server, for each test. You'll then need to run migrations in that database. This ends up being a huge pain in the ass. A better approach is to create a single postgres server one-time before running all of your tests. Then, create a template database on that server, and run your migrations on that template. Now, for each unit test, you can connect to the same server and create a new database from that template. This is not a pain in the ass and it is very fast: you run your migrations one time, and pay a ~20ms cost for each test to get its own database. I've implemented this for golang here — considering also implementing this for Django and for Typescript if there is enough interest. https://github.com/pet

Interesting idea to implement a proxy that does something on each new connection. I’ll have to think about that but very cool concept that lets this work for tests written in any language. Will each testcase need to run migrations after connecting for the first time? Also, I’m used to using clients which lazily pool connections, do they still work with this model? Op, I’d be curious what you think of my project pgtestdb, which takes a different approach: use a single server, but create new databases each time. Maybe there is some middle ground where a proxy creates a new cloned db after each connection? https://github.com/peterldowns/pgtestdb

Agreed with most of this but I’m skeptical of the rsc.io/script dsl approach. I’ll try it, though, because Russ is often right. shameless advert: do you wish testify was implemented with generics and go-cmp, and had a more understandable surface area? Check out my small library, testy https://github.com/peterldowns/testy shameless advert: do you want to write tests against your postgres database, but each new test adds seconds to your test suite? Check out pgtestdb, the marginal cost of each test is measured in tens of milliseconds, and each test gets a unique and isolated postgres instance — with all your migrations applied. https://github.com/peterldowns/pgtestdb

Hi HN, I'd like to get your feedback on pgtestdb, a new library for helping you write more useful database-backed tests. Many applications rely heavily on their database — and why shouldn't they! But writing tests that use the database is always a bit awkward, or in the worst case, very slow. Sometimes teams even mock their database out for an in-memory one, which works but only as long as you're not relying on specific features of your database. Postgres is a fantastic database, and at my last job we relied heavily on its views, functions, data types, etc. A lot of our application logic was tied up with it. But writing tests against it was slow. Naively, each test would have to: - Create a new database - Run our migrations on it - Then run the test - Then tear down the database pgtestdb makes this all a lot faster with two major strategi

Hi Weston, congratulations on launching. You've done a great job of explaining what the project is and how it works. I'm not in the ecosystem you've built this for so I can't comment too much on the project itself, but nice work communicating it. EDIT: after reading through the linked docs a bit more, I have to say that they are quite confusing. I'm not sure what the difference is between Ratifier, Kulvert, and SQL Simulator. I'd recommend finding a potential user and watching them go through the setup/install process and see where they have questions. If anyone is interested in something similar for postgres, I recently released https://github.com/peterldowns/pgtestdb. It uses template databases and advisory locks to give each test its own unique database with a near-zero marginal cost for each additional test. Combined with a ram/tmpfs-