sgr

player” (teams and orgs). On Splitgraph, every dataset – and every version of every dataset – has an address. Think of it like tagged Docker images. The address either points to an immutable “data image” (in which case we can optionally download objects required to resolve a query on-the-fly, although loading up-front is possible too) or to a live data source (in which case we proxy directly to it via FDW translation). This simple idea of _addressable data products_ goes a long way – for example, it means that computing a diff is now as simple as joining across two tables (one with the previous version, one with the new). Please excuse the Frankenstein marketing site – we’re in the midst of redesign / rework of info architecture while we build out our SaaS product. Feel free to reach out if you’ve got questions. And if you have a business case, we have spots available in our private pilot. My email is in my profile – mention HN :) [0] https://www.splitgraph.com/connect [1] examples: https://github.com/splitgraph/splitgraph/tree/master/example...

e.g. due to pathological JOIN queries), you can ingest any data source into a Splitgraph image, a versioned snapshot of data similar to a Docker image. [3] Our focus right now is on the internal enterprise use case. We use the Socrata plugin as a demo, but our customers are more interested in plugins for data sources they use internally, e.g. Postgres, Snowflake, Google Search Console, CSV in S3, etc. We support all those too; you can mount them all locally, and you can also mount them on Splitgraph.com (or an internal deployment), but still thin docs / no web UI yet. [0] https://www.splitgraph.com/blog/foreign-data-wrappers [1] https://github.com/splitgraph/splitgraph/blob/master/splitgr... [2] https://www.splitgraph.com/workspace/ddn?layout=hsplit&query... [3] https://www.splitgraph.com/docs/concepts/images

torage. You can "check out" an entire image to materialize it locally, at which point it will be like any other Postgres schema. Or you can use "layered querying" which will return a result set while only materializing the fragments necessary to answer the query. Regarding ClickHouse, you could watch this presentation [0] my co-founder Artjoms gave at a recent ClickHouse meet-up on the topic of your question. We also have specific documentation for using the ClickHouse ODBC client with the DDN [1], as well as an example reference implementation. [2] [0] https://www.youtube.com/watch?v=44CDs7hJTho [1] https://www.splitgraph.com/connect [2] https://github.com/splitgraph/splitgraph/tree/master/example...

th declarative recipes called Splitfiles, all using our open source tool. [3] Then you can push the images to Splitgraph and query them alongside all the other data on the platform. The public platform is mostly community data right now. Soon you'll be able to connect arbitrary upstreams to it (e.g. connect directly to a Mongo/Postgres/MySQL database by providing its read-only credentials in the web UI). As an example, we have a repository that forwards queries to the OxCOVID19 database, which is a Postgres database maintained by a lab at Oxford. [4] We're also building an enterprise program for hosting private deployments of Splitgraph. We have a pilot program where customers can use Splitgraph as a data cataloging and governance solution for their internal company data. We've also raised a round of funding but haven't announced it yet. If anyone is interested in joining the pilot program, get in touch. You can also read more details about how we dogfood Splitgraph in our own analytics stack. [5] [0] https://www.splitgraph.com/ [1] https://www.splitgraph.com/blog/data-delivery-network-launch [2] https://www.splitgraph.com/connect [3] https://www.github.com/splitgraph/splitgraph [4] https://www.splitgraph.com/splitgraph/oxcovid19 [5] https://www.splitgraph.com/blog/splitgraph-matomo-elasticsea...

vate deployments. In an ideal world, the private deployments will be able to subsidize the costs of some of the open data on the platform. Certainly we'd like to be able to support projects like this one. [0] https://github.com/splitgraph/splitgraph/tree/master/example...

taGrip, pgcli, pgAdmin, DBeaver...), tools (Metabase, dbt...) and extensions, including those that define custom types (e.g. PostGIS for geospatial data)[0]. We also offer a way of querying Splitgraph images without checking them out[1]. This lets it download required table regions on the fly (IIRC with Dolt, you have to fully clone the whole dataset and all of its history before querying it) and in a lot of cases can be faster than PostgreSQL itself[2]. This is also completely transparent to and compatible with existing clients. Splitgraph is decentralized and can treat any instance as a remote and push data there, with authorization provided by PostgreSQL, so you can use methods like LDAP/RADIUS/Kerberos to control access. It looks like you can spin up a Dolt remote with [3] but it's not very well documented so I can't comment on it. Finally, we offer a lot of features beyond data versioning (reproducible dataset builds with provenance tracking, mounting other databases, automatically generated REST API for all datasets on Splitgraph Cloud etc). When I tried it out a couple of months ago, Dolt's MySQL server didn't work with mysql_fdw. But their MySQL compatibility is progressing at an impressive pace and if it works now, you'll be able to query Dolt datasets directly from Splitgraph (by running sgr mount mysql[4]) and use them in Splitfiles. In the meantime, you can import data from Dolt into Splitgraph by using their dolt-to-pg adapter. [-1] https://www.splitgraph.com/docs/concepts/splitfiles [0] https://www.splitgraph.com/product/splitgraph/integrations [1] https://www.splitgraph.com/docs/large-datasets/layered-query... [2] https://github.com/splitgraph/splitgraph/blob/master/example... [3] https://github.com/liquidata-inc/dolt/tree/master/go/utils/r... [4] https://www.splitgraph.com/docs/ingesting-data/foreign-data-...