LibCST

TIL RustPython uses Ruff's AST which is fast. LibCST can modify the AST/CST to do python codemod, now in Rust too; rustpython-ruff_python_parser: https://crates.io/crates/rustpython-ruff_python_parser .. https://github.com/astral-sh/ruff LibCST: https://github.com/Instagram/LibCST

Could this ast.sprint ast.walk optimization make libCST or bandit faster? https://news.ycombinator.com/item?id=39111747 libCST: https://github.com/Instagram/LibCST bandit: https://github.com/PyCQA/bandit Links to codemod tools; "Baby Steps into Genetic Programming" https://news.ycombinator.com/item?id=43617655

Love it! …but of course I’d worry about a diff that added one offense while removing another, leaving the net sum the same. Perhaps the author handles this? You want to alert on the former and praise on the latter, not have them cancel out through a simple sum. Admittedly it’s a rare sounding edge case. The more trad technique for this would be to mark the offending line with # noqa or # ignore: foo. Another way is to have a .fooignore file but those are usually for paths or path globs to ignore. I like the author’s idea[1] of having the “ignore” mechanism next to the linter codebase itself, rather than mixed in with the production codebase. Adding the files and line numbers for known-offenders to that code could be a useful alternative to a simple sum? Perhaps more robustly, some kind of XPath like AST syntax to indicate which part

I was going to suggest libCST, it works really well and is much less of a hassle to set up than this. https://github.com/Instagram/LibCST

> I do wish tree-sitter had a mechanism to directly manipulate the AST. I was unable to simply rename/delete nodes and then write the AST back to disk. Instead I had to use Jedi or manually edit the source (and then deal with nasty off-set re-parsing logic). Or libCST: https://github.com/Instagram/LibCST docs: https://libcst.readthedocs.io/en/latest/ : > LibCST parses Python 3.0 -> 3.12 source code as a CST tree that keeps all formatting details (comments, whitespaces, parentheses, etc). It’s useful for building automated refactoring (codemod) applications and linters. libcst_transformer.py: https://gist.github.com/sangwoo-joh/26e9007ebc2de256b0b3deed... : > example code for renaming variables using libcst [w/ Visitors and Transformers] Refactoring because it doesn't pass formal verification: https://deal.readthedocs.io/basic/verification

How does libCST compare to e.g. pyCQA/redbaron? What about for EA Evolutionary Algorithms; does it preserve comments, or update docstrings and type annotations in mutating the code under test? Is it necessary to run `black` (and `precommit run --all-files`) to format the code after mutating it? Instagram/LibCST: https://github.com/Instagram/LibCST PyCQA/redbaron: https://github.com/PyCQA/redbaron E.g. PyCQA/bandit does static analysis for security issues in Python code: https://github.com/PyCQA/bandit https://news.ycombinator.com/item?id=38677294 https://news.ycombinator.com/item?id=24511280 ... https://analysis-tools.dev/tools?languages=python

Better late than never :) FB people have great experience with introducing typing into weakly typed languages (e.g. PHP vs Hack). So I actually started using pyre-check in place of mypy. The tool is really great and fast. I also like pysa quite a lot. Pyright only came into focus because it was best Python LSP implementation for NeoVim, and it is also the default for helix-editor which I play with. It's also nice that I could suggest pyright for some younger colleagues using VisualStudio Code (with strict mode enabled of course). P.S. I'm also using other interesting tools and libraries from FB (Instagram) like libcst [1] or fixit [2] [1] https://github.com/Instagram/LibCST/ [2] https://github.com/Instagram/Fixit

Never used it but it appears that Facebook/Instagram have a format preserving CST library for Python: https://github.com/Instagram/LibCST

Are you sure it wraps the LibCST library (https://github.com/Instagram/LibCST)? > Bowler builds on top of the Concrete Syntax Tree (CST) provided by lib2to3 in the standard library.

> One of the problems of patching code using ast package of Python is that it loses all the formatting and comments of the original source code. What you actually want here is a concrete syntax tree preserving all the comments and whitespace. There’s a sort of built-in one as part of lib2to3 used by the likes of black and yapf. There’s also a good third party options now in the form of https://github.com/Instagram/LibCST. There’s been talks about adding a CST module to PSL, but I don’t think the proposal went anywhere. https://bugs.python.org/issue33337