tree-sitter-javascript

I get the sense that treesitter is more for syntax highlighting than for real parsing, since I raised this issue[0] a while ago and I don't think anyone's really interested in it (not really interested enough myself in tree-sitter to see if it can be fundamentally solved; solving it involves making almost all production rules in the grammar parametric over two booleans). Admittedly I haven't tested the TypeScript treesitter grammar, but I'd be surprised if the issue is fixed there. I've put together a sample file[1] that demonstrates various cases of these context dependencies. If I remember correctly, Sublime's highlighter was the best at handling these cases out of various editors I tried, though it still failed at some of the ones at the bottom involving multi-line function expressions within object literal keys. GitHub/gist uses trees

No, I did not open an issue. When I checked the issues for tree-sitter-javascript, I saw the most recent issue was one reporting the same thing (segfault) from some weeks ago, with no attention. It still has had no attention, so I don't think reporting would have done much. Anyways, since you're here, I dug a bit more into this to make a more useful report. Starting with v0.20.2, the following file will cause a segfault when parsed using tree-sitter-javascript: https://github.com/tursodatabase/libsql/blob/main/libsql-sql... . It worked fine in v0.20.1, and it's still broken with the latest v0.20.4. Based on the diff here: https://github.com/tree-sitter/tree-sitter-javascript/compar..., I don't on the surface see a way to dig deeper into this than trying to read through a 170,000 line (!!!) diff to parser.c. And looking at that diff raises

Here is the tree-sitter grammar of Elisp: https://github.com/Wilfred/tree-sitter-elisp/blob/main/gramm... (approx. 200 lines) and here is the grammar of JavaScript: https://github.com/tree-sitter/tree-sitter-javascript/blob/m... (approx. 1200 lines) JavaScript evolved into a language of similar complexity as Perl 5 (the corresponding tree sitter syntax table counts almost 2000 lines, currently).

Here's the js grammar[1]. Despite the worrying talk of tokenization, it looks scannerless. Yay. One big grammar. JSX is blended in. And ecmascript versions aren't segregated. Which I didn't expect... but the motivation wasn't writing picky compilers. [1] https://github.com/tree-sitter/tree-sitter-javascript/blob/m...

FWIW, I'm developing a library based on the technique outlined in this paper (and others by Time Wagner). Like the original paper, it uses LR(1) (and GLR), not LL(k). The library itself is here: https://github.com/tree-sitter/tree-sitter and here are some existing grammars: * https://github.com/tree-sitter/tree-sitter-javascript * https://github.com/tree-sitter/tree-sitter-go * https://github.com/tree-sitter/tree-sitter-ruby

This is a parser generator that I'm working on: https://github.com/tree-sitter/tree-sitter It produces concrete syntax trees that can be queried by line or character index. The library is specifically focused on incremental parsing, for use in a text editor, but it also works fine for normal 'batch' parsing workloads. It has a simple C API that you should be able to use from any language. Currently, there are bindings to JavaScript and Haskell. Here are some example grammars: * C - https://github.com/tree-sitter/tree-sitter-c * JavaScript - https://github.com/tree-sitter/tree-sitter-javascript * Go - https://github.com/tree-sitter/tree-sitter-go * Ruby - https://github.com/tree-sitter/tree-sitter-ruby * Python - https://github.com/tree-sitter/tree-sitter-python