JEjeffdavis·May 15, 2020
I've been working on postgres-extension.rs[1]. The idea is that, instead of writing a PostgreSQL extension in C (which is the only option for interesting extensions), you can also write one in pure rust.
I've eagerly awaited many features to make this work reasonably well, and I've been very pleased how much rust has helped my use case over the last few years.
Although lots of languages have a C FFI, that's really not enough to extend a complex codebase. Postgres has it's own setjmp/longjmp-based error handling, it's own system of allocators, it needs a way to find the right functions in an extension and call them the right way, its own way of dealing with signals, etc. There are zillions of internal structs, and the extension needs to be able to read/modify them without copying/translation. Oh, and also, postgres doesn't like threads at
Read the rest
all, so the extension better not make any (at least not ones that call back into postgres APIs).
The only language even close to getting all of this right is rust:
* it has no runtime that causes problems with threading, scheduling, signal handling, or garbage collection
* typically GC'd languages can't operate very well on unmodified C structs; rust doesn't have a GC so it can
* rust goes out of its way to support C-compatible structs without any copying/translation, and can even treat some plain C representations as more interesting types in a binary-compatible way (like a nullable pointer in C could be treated as an Option<&MyStruct> in rust)
* the tokio library allows nice concurrency without creating threads if you use the CurrentThread runtime
* it supports changing the global allocator to be the postgres allocator
* rust has good procedural macro support, which is important because a lot of postgres APIs heavily use macros, so making the rust version ergonomic requires similar macro magic
Areas rust could be more helpful, but which I'm trying to address on my own to the extent that I can:
* Support for setjmp/longjmp. I know this is not easy, but important for interacting with C code that already uses it. I realize it would be unsafe, and that it can't be wrapped up in a safe way directly, and it would be delicate to create safe APIs that use it at all. But I still want it. I made a crate[2] that adds support, but it has a few issues that can't be resolved without compiler support.
* Better support for cdylib shared libraries that might call back into the host program that loads the library. Right now, you have to pass platform-specific flags to get it to link without complaining about undefined symbols (because they won't be resolve until the library is loaded into the host program). Also, it's difficult to test the shared library, because you have to guess at the location of the built library to be able to tell the host program where to find it before you can begin your test. I made a crate[3] to help with these things also, but it would be nice to have better support.
Oh, and one more thing on my wishlist not directly related to this project is that it would be nice to have support for datastructure-specific allocators (like a mini-allocator just for a hash table). Then, you'd be able to monitor and control memory usage by inspecting that allocator; and when you destroy or reset the hash table, you can know that the memory is freed/cleared as well (without worrying about fragmentation). Maybe these mini-allocators could also be used in other contexts, too, but it would probably be easiest to get the lifetimes to work out if it was tied to a data structure.
[1] https://github.com/jeff-davis/postgres-extension.rs
[2] https://github.com/jeff-davis/setjmp.rs
[3] https://github.com/jeff-davis/cdylib-plugin.rs