prism

I am most excited about the parser change, previously discussed here: https://news.ycombinator.com/item?id=36310130 - Rewriting the Ruby parser (2023-06-13, 176 comments) I remember being taught to use yacc in our compiler course because "writing it by hand is too hard". But looks like Ruby joins the growing list of languages that have hand-written parsers, apparently working with generated parsers turned out to be even harder in the long run. That said, replacing a ~16k line parse.y[1] with a 22k line prism.c[2] is a pretty bold move. [1] https://github.com/ruby/ruby/blob/master/parse.y [2] https://github.com/ruby/prism/blob/main/src/prism.c

Excitingly, Ruby 3.3+ shipped with a new alternative parser called Prism[0]! See also: [1][2]. [0]: https://github.com/ruby/prism [1]: https://railsatscale.com/2023-06-12-rewriting-the-ruby-parse... [2]: https://railsatscale.com/2024-04-16-prism-in-2024/

IRB type completion comes as a result of a chain of events which starts from the incredible work done by Kevin Newton (et al) to write a new canonical Ruby parser called Prism in C99 with no dependencies [1]. With Prism, you can then create tool suites like syntax_tree [2], which then leads Prettier formatters [3], a new Ruby LSP [4], which unlocks a new Ruby LSP VS Code extension [5], not to mention a laundry list of other gems like Rubocop and of course Ruby itself that will benefit from a faster and more maintainable Ruby parser. It's a beautiful illustration of the power of questioning conventions, going back to first principles to uncover better solutions to previously solved problems, whose new solutions create new capabilities which unlocks the ability to solve new problems. [1]: https://github.com/ruby/prism [2]: https://github.co

It’s disappointing there doesn’t seem to be an easily available blog post or announcement (maybe I just didn’t find it), but the design doc lists a few motivations: https://github.com/ruby/prism/blob/main/docs/design.md It looks like there was also a podcast interview last year that touches on the origins of the project: https://topenddevs.com/podcasts/ruby-rogues/episodes/the-new... Reading between the lines it looks to me like this is motivated by ripper (the old parser) not being a great fit for tooling around ruby like IDE LSP integrations and such. Ripper isn’t fault tolerant (if the script has a a syntax error you don’t get a partial tree, just an exception); being implemented in ruby enough itself that it kind of depends on ruby which isn’t always convenient for integration (IDEs like vscode make plugins in JS easy, prism comes wit