citus

The Citus database is 100% open source. Learn what's new in the Citus 14 release blog and the Citus Updates page. What is Citus? Citus is a PostgreSQL extension that transforms Postgres into a distributed database—so you can achieve high performance at any scale. With Citus, you extend your PostgreSQL database with new superpowers: Distributed tables are sharded across a cluster of PostgreSQL nodes to combine their CPU, memory, storage and I/O capacity. Reference tables are replicated to all nodes for joins and foreign keys from distributed tables and to maximize read performance. Distributed query engine routes and parallelizes SELECT, DML, and other operations on distributed tables across the cluster. Columnar storage compresses data, speeds up scans, and supports fast projections, both on regular and distributed tables. Query from any node enables you to utilize the full capacity of your cluster for distributed queries You can use these Citus superpowers to make your Postgres database scale-out ready on a single Citus node. Or you can build a large cluster capable of handling high transaction throughputs, especially in multi-tenant apps, run fast analytical queries, and process large amounts of time series or IoT data for real-time analytics. When your data size and volume grow, you can easily add more worker nodes to the cluster and rebalance the shards. Our SIGMOD '21 paper Citus: Distributed PostgreSQL for Data-Intensive Applications gives a more detailed look into what Citus is, how it works, and why it works that way. Since Citus is an extension to Postgres, you can use Citus with the latest Postgres versions. And Citus works seamlessly with the PostgreSQL tools and extensions you are already familiar with. Why Citus? Getting Started Using Citus Schema-based sharding Setting up with High Availability Documentation Architecture When to Use Citus Need Help? Contributing Stay Connected Why Citus? Developers choose Citus for two reasons: Your application is outgrowing a single PostgreSQL node If the size and volume of your data increases over time, you may start seeing any number of performance and scalability problems on a single PostgreSQL node. For example: High CPU utilization and I/O wait times slow down your queries, SQL queries return out of memory errors, autovacuum cannot keep up and increases table bloat, etc. With Citus you can distribute and optionally compress your tables to always have enough memory, CPU, and I/O capacity to achieve high performance at scale. The distributed query engine can efficiently route transactions across the cluster, while parallelizing analytical queries and batch operations across all cores. Moreover, you can still use the PostgreSQL features and tools you know and love. PostgreSQL can do things other systems can’t There are many data processing systems that are built to scale out, but few have as many powerful capabilities as PostgreSQL, including: Advanced joins and subqueries, user-defined functions, update/delete/upsert, constraints and foreign keys, powerful extensions (e.g. PostGIS, HyperLogLog), many types of indexes, time-partitioning, and sophisticated JSON support. Citus makes PostgreSQL’s most powerful capabilities work at any scale, allowing you to handle complex data-intensive workloads on a single database system. Getting Started The quickest way to get started with Citus is to use the Elastic Clusters in Azure Database for PostgreSQL Flexible Server managed service in the cloud—or set up Citus locally. Citus Managed Service on Azure You can get a fully-managed Citus cluster in minutes through Elastic Clusters in Azure Database for PostgreSQL Flexible Server. Azure will manage your backups, high availability through auto-failover, software updates, monitoring, and more for all of your servers. To get started with Citus on Azure, use the Elastic Cluster quickstart. Running Citus using Docker The smallest possible Citus cluster is a single PostgreSQL node with the Citus extension, which means you can try out Citus by running a single Docker container. # run PostgreSQL with Citus on port 5500 docker run -d --name citus -p 5500:5432 -e POSTGRES_PASSWORD=mypassword citusdata/citus # connect using psql within the Docker container docker exec -it citus psql -U postgres # or, connect using local psql psql -U postgres -d postgres -h localhost -p 5500 Install Citus locally If you already have a local PostgreSQL installation, the easiest way to install Citus is to use our packaging repo Install packages on Ubuntu / Debian: curl https://install.citusdata.com/community/deb.sh | sudo bash sudo apt-get -y install postgresql-18-citus-14.1 Install packages on Red Hat: curl https://install.citusdata.com/community/rpm.sh | sudo bash sudo yum install -y citus141_18 To add Citus to your local PostgreSQL database, add the following to postgresql.conf: shared_preload_libraries = 'citus' After restarting PostgreSQL, connect using psql and run: CREATE EXTENSION citus; You’re now ready to get started and use Citus tables on a single node. Install Citus on multiple nodes If you want to set up a multi-node cluster, you can also set up additional PostgreSQL nodes with the Citus extensions and add them to form a Citus cluster: -- before adding the first worker node, tell future worker nodes how to reach the coordinator SELECT citus_set_coordinator_host('10.0.0.1', 5432); -- add worker nodes SELECT citus_add_node('10.0.0.2', 5432); SELECT citus_add_node('10.0.0.3', 5432); -- rebalance the shards over the new worker nodes SELECT rebalance_table_shards(); For more details, see our documentation on how to set up a multi-node Citus cluster on various operating systems. Using Citus Once you have your Citus cluster, you can start creating distributed tables, reference tables and use columnar storage. Creating Distributed Tables The create_distributed_table UDF will transparently shard your table locally or across the worker nodes: CRE