sirixdb

sirix

Java2013–2026

sirixdb / sirix

sirix

SirixDB is an an embeddable, bitemporal, append-only database system and event store, storing immutable lightweight snapshots. It keeps the full history of each resource. Every commit stores a space-efficient snapshot through structural sharing. It is log-structured and never overwrites data. SirixDB uses a novel page-level versioning approach.

Explore on GitHub Read Hacker News discussions

9Distinct people

56Mentions

0Explicit recommenders

1.2KGitHub stars

REPOSITORY AT A GLANCE

What to know before exploring

What it is

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 9:01:12 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year56 dated mentions

Observed mentions: 2013-06-18 17:54:29 UTC to 2026-07-15 15:46:50 UTC.

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges

UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
Period	Start (UTC)	End (exclusive, UTC)	Mentions	Submissions	Comments
2013	2013-01-01 00:00:00 UTC	2014-01-01 00:00:00 UTC	1	1	0
2014	2014-01-01 00:00:00 UTC	2015-01-01 00:00:00 UTC	1	1	0
2015	2015-01-01 00:00:00 UTC	2016-01-01 00:00:00 UTC	0	0	0
2016	2016-01-01 00:00:00 UTC	2017-01-01 00:00:00 UTC	0	0	0
2017	2017-01-01 00:00:00 UTC	2018-01-01 00:00:00 UTC	0	0	0
2018	2018-01-01 00:00:00 UTC	2019-01-01 00:00:00 UTC	3	1	2
2019	2019-01-01 00:00:00 UTC	2020-01-01 00:00:00 UTC	12	3	9
2020	2020-01-01 00:00:00 UTC	2021-01-01 00:00:00 UTC	6	0	6
2021	2021-01-01 00:00:00 UTC	2022-01-01 00:00:00 UTC	2	0	2
2022	2022-01-01 00:00:00 UTC	2023-01-01 00:00:00 UTC	15	3	12
2023	2023-01-01 00:00:00 UTC	2024-01-01 00:00:00 UTC	12	4	8
2024	2024-01-01 00:00:00 UTC	2025-01-01 00:00:00 UTC	1	0	1
2025	2025-01-01 00:00:00 UTC	2026-01-01 00:00:00 UTC	1	0	1
2026 (partial)	2026-01-01 00:00:00 UTC	2027-01-01 00:00:00 UTC	2	2	0

What Hacker News says

12 sources

lichtenberger·Dec 9, 2019

Pretty nice tool :-) But what if the data store would do all of that already? - retaining the history of your data: - free deduplication through copy on write and sharing unchanged structures - further store space savings through page-level versioning (not all records in a page have to be written: just the changed/inserted/deleted ones plus depending on the algorithm a few more to fast track it's reconstruction) - read and write performance is balanced using a sliding snapshot algorithm: This avoids having to read long chains of incremental page changes and it avoids write peaks from intermittant full page snapshots. Instead they get trailed over a few (usually three or four revisions). - the sorage manager stores timestamps for all revision in a RevisionRootPage, which itself is indexed - Optional commit author and m

Read the rest

essage - Reverting to and committing a past "good" revision is just adding a few bytes to index a new RevisionRootPage which references the past revision data (simple pointer) - You can diff taking rolling hashes into account - Currently I'm working on a diffing JSON format and a web front-end to display these in special space filling approaches - Sophisticated time travel queries - typed, user-defined and versioned secondary index structures (based on AVL trees) - a path summary kept up to date always - a lot of metadata stored and kept up to date for each record/node It's already implemented in https://sirix.io / https://github.com/sirixdb/sirix I'm working on getting it stable, but help from you, the Open Source community is always needed :-)

In Show HN: Version Control for Databases

lichtenberger·Dec 9, 2019

I'm working on a temporal document store in my spare time. If done right and you have an index structure which let's say indexes each revision of a table (in my case binary XML or JSON as of now) as in Git for instance you simply append a new RevisionRootPage. This page is indexed of course and points to the former revision-data. Should be super fast, so you don't have to replay changes made in the meantime. Even if I'd write to a distributed log for instance and would read from it the changes since revision n. I think that way you can also version the whole database easily. https://sirix.io / https://github.com/sirixdb/sirix

In Show HN: Version Control for Databases

lichtenberger·Nov 12, 2019

I'm working on storing data in a log-structure persistently without the overhead of a transaction log in my spare time (Open Source[1]). Instead, an UberPage which guarantees consistency is atomically swapped during a transaction commit (inspired by ZFS). It should be used for data, which doesn't fit into main memory. Revisions are always appended, whereas almost only changed data plus some metadata is written to a file during commits. You can easily retain the full version history in a log structure, but you need fast random access (option of parallel "real" access would be best) on a flash drive -- PCIe SSDs for instance. Basically in order to balance read and write performance only a fraction of each database page (with changed records) needs to be written sequentially in batches to the end of a file. Each revision is indexed under a R

Read the rest

evisionRootPage and these are indexed under the UberPage with keyed tries. This opens up a lot of opportunities for analysing data and its history. [1] https://github.com/sirixdb/sirix

In When your data doesn’t fit in memory: the basic techniques

lichtenberger·Sep 25, 2019

Great work on Dolt and seeing temporal data stores emerging :-) Regarding XML and JSON SirixDB[1] already provides full blown time-travel queries using a fork of Brackit[2], that is basically XQuery to process and query both the XML as well JSON documents. That said SirixDB in principal could also store relational data or graph data. The storage engine has been built from scratch to offer the best possible versioning capabilities. However I've never implemented branching/merging as I didn't come up with good use cases. It seems it's then always more of a versioning system like Git, but more fine granular. I always struggled to implement this as SirixDB currently only allows a single read-write transaction on a resource. Thus, if it would support branching and merging users would have to manually handling conflicts when merging (or automat

Read the rest

ically -- using a merge-strategy which is often case not good). There's however plently of optimization potential, as SirixDB optionally stores a lot of metadata for each node (number of descendants, a rolling hash, Dewey-IDs, number of children... as well as user-defined, typed secondary index-structures). I'll have to look how to build AST rewrite rules and implement a lot of optimizations into my Brackit binding in the future, so it's just the starting point (but everything should at least work already) :-) [1] https://sirix.io and https://github.com/sirixdb/sirix [2] http://wwwlgis.informatik.uni-kl.de/cms/fileadmin/publicatio...

In Launch HN: Quilt (YC W16) – A versioned data portal for S3

lichtenberger·Sep 24, 2019

So you basically store S3 Buckets in Elastic Search and you're using Git for versioning a hierarchy of buckets, right? It's interesting that versioning now finally seems to be getting some traction in mainstream database systems (even though they are not really optimal in these systems my opinion) and for instance also in your data store. You position this as a Dropbox or Google Drive replacement, right? :-) I'm asking all these questions, because I'm engineering a temporal, versioned Open Source storage system myself (since I studied at the University of Konstanz until 2012), possibly on a much more database oriented level -- currently for storing both XML and JSON data in a binary format. A resource in this storage system basically stores a huge tree of database pages whereas an UberPage is the main entry point (reminiscent of ZFSs Uber

Read the rest

Page, from which SirixDB borrows some ideas and puts these to the sub-file level), consisting of various more or less hash-array based subtrees as in ZFS. Thus, levels of indirect pages are added if more data needs to be stored. I've added some optimizations from in-memory hash-array based tries. Each revision is indexed. SirixDB stores per revision and per page deltas based on a copy-on-write log-structure. I've thought about storing each database page fragment in a S3 storage backend as another storage option and using Apache BookKeeper directly or Apache Pulsar for distributing an in-memory intent log (it doesn't need to be persisted before committing to the data files, as the UberPage just needs to be swapped atomically for consistency). For the interested reader: https://sirix.io and https://github.com/sirixdb/sirix

In Launch HN: Quilt (YC W16) – A versioned data portal for S3

lichtenberger·Sep 23, 2019

I've already written with one of the core committers to BookKeeper and yes, I think I'll use it. That said, I'm always looking for users and contributors to https://sirix.io/ or https://github.com/sirixdb/sirix. That would be super awesome, but I'm sure you have your own ideas already for using BookKeeper. I'm currently not sure if I first want to build a frontend (I'm a backend engineer, but would like to learn some TypeScript along with using Vue.js and D3js), to interact with SirixDB and to build interactive visualizations to compare revisions of JSON- or XML-resources in SirixDB (stored in a binary format of course highly optimized for space-efficient snapshots). I think as I'm lacking users it might be more useful, but for sure I'm at least as eager to put forth the idea of scalable SirixDB databases :-) So, I'd like to use BookKeepe

Read the rest

r most probably (single writer, read your own write consistency, using synchronous -- for a quorum -- and asynchronous replication for the rest, exactly once semantics...). The thing I don't like is that we also need ZooKeeper, but yeah. BTW: Why do you think BookKeeper is better than Kafka for this purpose? :-)

In Building a Relational Database Using Kafka

burtonator·May 5, 2019Submission

SirixDB – An Evolutionary, Temporal NoSQL Storage System

88 points · Open discussion

lichtenberger·Apr 26, 2019

Hi all, if anyone is interested, the implementation is available as Open Source software: https://github.com/sirixdb/sirix I recently wrote quiet some documentation at https://sirix.io regarding use cases, the APIs, put up a community forum and wrote a complete native JSON storage (besides XML) :-) The system greatly advanced since Marc's Ph.D. thesis (now, as mentioned above it stores also JSON besides XML in a tailored binary format, I've added sophisticated query capabilities through Brackit (https://brackit.org) and temporal enhancements in Brackit (https://github.com/sirixdb/brackit), added versioned index-structures, diffing capabilities to import differences between XML-documents or to diff revisions once stored in Sirix (and added a currently outdated GUI with novel interactive visualizations of the encountered differences found

Read the rest

between any two revisions). Next steps will be adding rewriting rules for index accesses in the query compiler, and with the help of you, the Open Source community I want to publish version 1.0.0 soon. Afterwards, I'd love to look into partitioning/replication, but it's also up to you :-) Any kind of comment, help, questions regarding the APIs or anything related are more than welcome :-) I'll contact Marc, soon that I'm still super eager to put forth his ideas (and some of mine and of the other students involved, notably Sebastian Graf, who also rewrote a lot of stuff for his Ph.D.) More than happy to answer any questions :-) kind regards Johannes

In Evolutionary Tree-Structured Storage (2014) [pdf]

lichtenberger·Apr 24, 2019

Hi, Sirix.io is all about efficient versioning of your data. That is on the one hand it reduces the storage cost of storing a new revision during each transaction-commit while balancing read- and write-performance through a novel sliding snapshot algorithm and dynamic page compression. On the other hand Sirix supports easy query capabilities for instance to open a specific revision by a timestamp or revisions by a given timespan, to navigate to future or past versions of nodes in the tree-structure and so on. It basically never overwrites data and is heavily inspired by ZFS and Git and borrows some ideas and puts these to test on the sub-file level. In stark contrast to other approaches Sirix combines copy-on-write semantics with node-level versioning. It all started around 2006 as a university / Ph.D. project of Marc Kramis and I already

Read the rest

spend my Bachelor's Thesis, Master's Thesis as well as several HiWi-Jobs on the project and I'm still more eager than ever to put forth the idea of a versioned, analytics plattform to perform analytical tasks based on current as well as the history of the data. I've been working a lot on Sirix.io the last couple of months and spent most evenings after my day to day software engineering Job. I had already posted around christmas, but now I've added a native JSON storage layer besides XML, that is the storage itself, XQuery extensions for JSON for joint XML- and JSON-querying, a RESTful-API also for JSON, added a website, a forum and first documentation... :-) and last night I release version 0.9.0. So, any feedback, comments, suggestions, just everything is more than welcome. I'll happily answer any questions you might have. kind regards Johannes https://sirix.io https://github.com/sirixdb/sirix

In ShowHN: Query the past, do audits, compare JSON/XML;lessons learned from ZFS/Git

based2·Apr 20, 2019Submission

Sirix: Effective and efficient storing and querying of your temporal data

1 points · Open discussion

lichtenberger·Apr 20, 2019

Hi, I had already posted yesterday, but maybe it's of some value for some of you, I still hope at least :-) Sirix.io is all about efficient versioning of your data. That is on the one hand it reduces the storage cost of storing a new revision during each transaction-commit while balancing read- and write-performance through a novel sliding snapshot algorithm and dynamic page compression. On the other hand Sirix supports easy query capabilities for instance to open a specific revision by a timestamp or revisions by a given timespan, to navigate to future or past versions of nodes in the tree-structure and so on. It basically never overwrites data and is heavily inspired by ZFS and borrows some ideas and puts these to test on the sub-file level. It all started around 2006 as a university / Ph.D. project of Marc Kramis and I already spend my

Read the rest

Bachelor's Thesis, Master's Thesis as well as several HiWi-Jobs on the project and I'm still more eager than ever to put forth the idea of a versioned, analytics plattform to perform analytical tasks based on current as well as the history of the data. I've been working a lot on Sirix.io the last couple of months and spent most evenings after my day to day software engineering Job. I had already posted around christmas, but now I've added a native JSON storage layer besides XML, that is the storage itself, XQuery extensions for JSON for joint XML- and JSON-querying, a RESTful-API also for JSON, added a website, a forum and first documentation... :-) and last night I release version 0.9.0. So, any feedback, comments, suggestions, just everything is more than welcome. I'll happily (try to? ;)) answer any questions you might have. Happy easter everyone :-) kind regards Johannes https://sirix.io https://github.com/sirixdb/sirix

In Show HN: Sirix.io – versions XML/JSON efficiently and allows time-travel queries

lichtenberger·Apr 19, 2019Submission

Show HN: SirixDB (OS) – versioning through efficient snapshotting (XML/JSON)

Hi, I've been working a lot on Sirix.io the last couple of months and spent most evenings after my day to day software engineering Job. I had already posted around christmas, but now I've added a native JSON storage layer besides XML, that is the storage itself, XQuery extensions for JSON for joint XML- and JSON-querying, a RESTful-API also for JSON, added a website, a forum and first documentation... :-) and last night I release version 0.9.0. Sirix.io is all about efficient versioning of your data, that is on the one hand it reduces the storage cost of storing a new revision during each transaction-commit while balancing read- and write-performance through a novel sliding snapshot algorithm and dynamic page compression and on the other hand Sirix supports easy query capabilities for instance to open a specific revision by a timestamp or

Read the rest

revisions by a given timespan, to navigate to future or past versions of nodes in the tree-structure. It basically never overwrites data. It all started around 2006 as a university / Ph.D. project of Marc Kramis and I already spend my Bachelor's Thesis, Master's Thesis as well as several HiWi-Jobs on the project and I'm still more eager than ever to put forth the idea of a versioned, analytics plattform to perform analytical tasks based on current as well as the history of the data. So, any feedback, comments, suggestions, just everything is more than welcome. Have fun :-) kind regards Johannes https://sirix.io https://github.com/sirixdb/sirix And some further reading http://pubsys.mmsp-kn.de/pubsys/publishedFiles/Kramis2014.pd... https://kops.uni-konstanz.de/bitstream/handle/123456789/2725... http://kops.uni-konstanz.de/bitstream/handle/123456789/21041... http://wwwlgis.informatik.uni-kl.de/cms/fileadmin/publicatio...

3 points · Open discussion