jist

Hi HN! Happy to share that jist can now search JSON 17x faster than jq for a provided input (string or a file). The major change is using the bleeding edge simdjson parser as the preferred way of parsing and querying the input. Since simdjson can only work on files up to 4.2GB, jist falls back to its earlier implementation of scanning through input in chunks of 1MB which means it can still query files of virtually infinite size 3x faster than jq ever could while using ~10MB of memory. The next major feature will be summarizing input JSON into a schema - and it will do this by scanning the whole file to get a complete picture. Stay tuned! Any feedback (or code review) is much appreciated!

Hi HN, jist is a JSON path search tool that feels like accessing a JavaScript object. It’s 3x faster than jq for the same use case, and over 2000x more memory efficient when searching a 3.3GB file for the last element (jist reads data in chunks so has constant space use). It doesn’t implement pattern matching yet. A feature I’m looking to add is generating a full JSON schema using the same streaming approach so stay tuned! Any feedback and feature requests are more than welcome