obi1kenobi

trustfall

Rust2022–2026

obi1kenobi / trustfall

trustfall

A query engine for any combination of data sources. Query your files and APIs as if they were databases!

Web Development Data & Databases

Explore on GitHub Read Hacker News discussions

8Distinct people

24Mentions

0Explicit recommenders

2.9KGitHub stars

REPOSITORY AT A GLANCE

What to know before exploring

What it is

A query engine for any combination of data sources. Query your files and APIs as if they were databases!

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

obi1kenobi/cargo-semver-checks · linked in 2 of the same discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 9:01:12 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year24 dated mentions

Observed mentions: 2022-04-29 22:05:24 UTC to 2026-05-28 21:52:18 UTC.

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges

UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
Period	Start (UTC)	End (exclusive, UTC)	Mentions	Submissions	Comments
2022	2022-01-01 00:00:00 UTC	2023-01-01 00:00:00 UTC	13	2	11
2023	2023-01-01 00:00:00 UTC	2024-01-01 00:00:00 UTC	8	1	7
2024	2024-01-01 00:00:00 UTC	2025-01-01 00:00:00 UTC	1	0	1
2025	2025-01-01 00:00:00 UTC	2026-01-01 00:00:00 UTC	0	0	0
2026 (partial)	2026-01-01 00:00:00 UTC	2027-01-01 00:00:00 UTC	2	0	2

What Hacker News says

8 sources

obi1kenobi·Sep 7, 2023

If you'd like to dig deeper, here are some links: - cargo-semver-checks: https://github.com/obi1kenobi/cargo-semver-checks - Trustfall query engine, which powers cargo-semver-checks: https://github.com/obi1kenobi/trustfall - Trustfall playground, where you can query Rust library APIs in your browser -- for example, "which structs in `itertools` are importable by more than one path": https://play.predr.ag/rustdoc#?f=2&q=*3-Structs-importable-f... - 10min conference talk on Trustfall: https://www.hytradboi.com/2022/how-to-query-almost-everythin... I'm also giving a talk at P99 CONF in a few months about how Trustfall's new optimizations API made cargo-semver-checks over 2300x faster: https://twitter.com/PredragGruevski/status/16893002495908003...

In Semver violations are common, better tooling is the answer

obi1kenobi·Jul 9, 2023

No worries! Trustfall is even more general than semver actually, one can use it to query any data source -- even HN itself: https://play.predr.ag/hackernews Here's a link to that project if you'd like to check it out: https://github.com/obi1kenobi/trustfall I gave a 10min conference talk on it last year titled "How to query (almost) everything" https://www.hytradboi.com/2022/how-to-query-almost-everythin...

In Cargo-semver-checks: Scan your Rust crate for semver violations

obi1kenobi·May 16, 2023

I've had my eye on Taxi for a while, and it's neat! I agree that the problems it aims to solve are real and painful in real life. In my experience, I'm not sure people care about schemas or schema languages — they are just implementation details best left under the hood. This is why in my own work, I started on the query end of the spectrum instead. This is why I built Trustfall, a query engine able to query any data source: DBs, APIs, files like PDF, CSV, or JSON, or any combination of those. GitHub: https://github.com/obi1kenobi/trustfall Playground showing the query "which GitHub/Twitter users comment on HackerNews stories about OpenAI?": https://play.predr.ag/hackernews#?f=1&q=IyBDcm9zcyBBUEkgcXVl...

In Why we created Taxi, and why we felt the need for another schema language

obi1kenobi·Mar 3, 2023

It's a very exciting time to be working in this space! Going beyond structured databases and file formats like JSON/CSV, there are also systems that can query APIs, source code, ML models, etc. My own Trustfall query engine is one of them: https://github.com/obi1kenobi/trustfall For example, you can query the HackerNews APIs from your browser: "Which Twitter/GitHub users comment on stories about OpenAI?" https://play.predr.ag/hackernews#?f=1&q=IyBDcm9zcyBBUEkgcXVl... One of its real-world use cases is at the core a Rust semver linter: https://predr.ag/blog/speeding-up-rust-semver-checking-by-ov...

In DuckDB: Querying JSON files as if they were tables

obi1kenobi·Feb 9, 2023

I bumped into him and showed it to him in person one time :) Realistically, the docs are still very sparse -- I wasn't planning on posting this for another few months, but someone else noticed my project and beat me to it! New data providers need a schema and an Adapter implementation. For Rust, this is the Adapter trait that data providers need to implement: https://docs.rs/trustfall_core/latest/trustfall_core/interpr... This is the equivalent abstract class in Python: https://github.com/obi1kenobi/trustfall/blob/pytrustfall-v0.... For Python, the package docs have some (underwhelming) additional examples of building a schema and passing an adapter to use to run a query: https://pypi.org/project/trustfall/ The process in Rust is equivalent, here's the code powering the demo in the talk: https://github.com/obi1kenobi/trustfall/blob/main/d

Read the rest

emo-hytra... I realize this is all extremely suboptimal, but like I said, I wasn't ready Please stay tuned! I'm planning to spruce all of this up in the coming weeks!

In Trustfall: How to Query (Almost) Everything

obi1kenobi·Feb 9, 2023

As I wrote in reply to your other comment, the join algorithms are the adapter's choice. I wouldn't necessarily recommend using it as a SQL replacement, though. There's no measurable MoM growth yet. This is the first major look the broader community has had at the system outside of a relatively small conference where I gave a talk, and demos I've given to friends and colleagues. I wasn't planning on submitting to HN for another few months -- but someone else beat me to it today :) There are production-grade adapters for HackerNews APIs and for Rust's rustdoc JSON format, used by the Playground (https://play.predr.ag/hackernews) and the Rust semver-checking linter cargo-semver-checks, respectively. There are also a few more demo adapters in the Trustfall repo itself: https://github.com/obi1kenobi/trustfall I expect the number of adapters t

Read the rest

o grow significantly in the coming months, so please stay tuned and let me know if you have datasets you'd like to try it with! The "largest dataset" is a bit of a trick question: how big is the dataset of "all HackerNews data available via its Firebase and Algolia APIs"? Because that's what the Playground queries. The Rust semver linter has been used by dozens if not hundreds of crates, and the JSON payloads in question there are in the 100-400MB range. The example in this blog post runs 40 quite complex Trustfall queries (they express semver rules!) over 400MB across two JSON files in 8 seconds: https://predr.ag/blog/speeding-up-rust-semver-checking-by-ov... You can also see more real-world use cases in this talk I gave last year: https://www.hytradboi.com/2022/how-to-query-almost-everythin... Probably around 200-300 people or so have written queries in the language, with experience levels ranging from seasoned engineers to Excel analysts and SQL analysts with no programming experience outside of that domain. The language itself is actually a refinement from an earlier open-source project I developed and open-sourced through my previous job. That open-source project was used to query everything from a multi-TB SQL cluster to APIs to ML models, and with Trustfall I've taken the opportunity to revisit and update design decisions that the previous project got to regret in retrospect.

In Trustfall: How to Query (Almost) Everything

obi1kenobi·Feb 9, 2023

Thanks for the detailed feedback, I appreciate it! I plan on writing more about all the topics you mentioned, and add docs for writing adapters and queries. I was planning on doing a "Show HN" when more of that was in place, but I'm not the OP here -- someone else submitted it :) Obviously all of the below needs to be more discoverable, and is something I'll work on! Here are the slides from my talk from the HYTRADBOI conference last year, containing some more real-world query use cases: https://docs.google.com/presentation/d/1foUdlEDOQ1WcTadhAxsA... This is the current stable adapter API: it's a trait that adapters need to implement https://docs.rs/trustfall_core/latest/trustfall_core/interpr... It's also available in the Python bindings: https://github.com/obi1kenobi/trustfall/blob/pytrustfall-v0.... Tl;Dr of the optimizations API is th

Read the rest

at every call into the adapter gets a reference to a `QueryInfo` struct. It's totally safe to ignore if the adapter isn't interested in optimizing, and otherwise allows the adapter to ask questions like "from this location in the query, will future operations include traversing edge X and then filtering on property Y?" If so, it's possible to use the API to resolve the expected values for that Y property and use them to speed up the lookup, as in the case with the hashtables in cargo-semver-checks. The optimizations API is still pretty raw and I didn't want to dig into it too much in that post (which is mostly about cargo-semver-checks' architecture, not just Trustfall internals) in case it changed dramatically. If you really want to see the current state of the API, here's a link to the draft PR: https://github.com/obi1kenobi/trustfall/pull/131/files#diff-...

In Trustfall: How to Query (Almost) Everything

sbt567·Feb 8, 2023Submission

Trustfall: How to Query (Almost) Everything

288 points · Open discussion