upb

that's doing register allocation and optimization. Writing this sort of JIT that uses a static register allocation and a fixed instruction sequence for each bytecode op doesn't seem that much more work than writing an interpreter in assembly language (especially if you use Mike Pall's excellent framework DynASM: http://luajit.org/dynasm.html). > Its certainly common to use a single indirect branch. However, it is suboptimal. I don't believe that a single indirect branch is a priori slower; while replicated dispatch makes better use of the branch predictor, it also results in less optimal i-cache usage (the LuaJIT code notes this trade-off, but observes that replicated dispatch is 20-30% faster in practice: http://repo.or.cz/w/luajit-2.0.git/blob/2ad9834df6fe47c7150d...). I'm not saying switch is better, what I'm saying is that you can't reliably replicate dispatch in C (at least according to Mike's message that I cited earlier), which makes switch() a totally reasonable choice for interpreters written in C. By the way, I'm not just making stuff up, I have written a JIT of my own: https://github.com/haberman/upb/blob/master/upb/pb/decoder_x...

ve this. My library performs no I/O (it's buffers in, buffers out), is completely resumable (so you can suspend the state machine at any point and resume it later), is completely composable (so you can create pipelines that feed the output of one parser to the input of another), loosely coupled, small (<5k loc in the core library), easy to bind to high-level languages, minimal enough that it can run inside the Linux kernel, and FAST (can parse protocol buffers at almost 1.5GB/second). And it's open-source. Don't be misled by the fact that it's advertised as a Protocol Buffer library; that's how it started, but I'm working on generalizing it to be capable of arbitrary parsing and serializing. Though Protocol Buffers schemas are used to specify the structure of any structured data that a parser wants to produce or a serializer wants to consume. https://github.com/haberman/upb/wiki

good C programs use these sparingly and clearly mark the macro with an UPPER_CASE_IDENTIFIER. C++ is much harder to "see through." Any function call could end up throwing an exception which does a non-local exit from the function. Any addition, assignment, pointer dereference, etc could actually be an overloaded operator. Any function call could be performing a user-defined implicit conversion on any of its parameters. For overloaded functions, the compiler decides what function is actually called by applying non-trivial pattern matching rules. The runtime will automatically invoke any static initialization code (like constructors of global statics) pre-main in an undefined order. etc. etc. I intend to get a thorough security review of my library once its design stabilizes. I asked Tavis Ormandy (a fellow Googler and security badass) which language is easier to audit for security and he said C for the reasons of transparency I listed above. People often say that you can just avoid using the parts of C++ that you don't like. That is true, but hard to enforce, especially on a multi-person project. If you're asking others to follow your set of rules, you have to write those rules down in meticulous detail, leading to something like the Google C++ style guide (http://google-styleguide.googlecode.com/svn/trunk/cppguide.x...), and at that point you're asking your contributors to study and internalize a large and complicated list of what is and isn't allowed just for the privilege of contributing to your project. I don't think this approach scales to lots of projects that are all choosing a different subset of C++. There are definitely things I miss by not having C++'s facilities. Constructors and an implicit "this" parameter are both really nice. Constructors are remarkable in that you can write a single function that gives you three different construction patterns for free (direct construction, heap construction with "new", construction in an arbitrary location with placement new). Automatic upcasts are nice and totally safe. Inheritance and public/private specifiers let you express directly lots of things you can only express with comments in C. But overall I think that using straight C is the right choice for a project like my library UPB.

e author asks "How would you do that in Python?" The answer is that it would be something like this: import ast class MyTransformer(ast.NodeTransformer): pass # Implement transformation logic here. node = MyTransformer().visit(ast.parse("x = 1")) print ast.dump(node) This works alright, but what I'm after is a more universal solution. With syntax trees there's a lot of support functionality you frequently want: a way to specify the schema of the tree, convenient serialization/deserialization, and ideally a solution that is not specific to any one programming language. My answer to this question might surprise some people, but after spending a lot of time thinking about this problem, I'm quite convinced of it. The answer is Protocol Buffers. It's true that Protocol Buffers were originally designed for network messaging, but they turn out to be an incredibly solid foundation on which to build general-purpose solutions for specifying and manipulating trees of strongly-typed data without being tied to any one programing language. Just look at a system like http://scottmcpeak.com/elkhound/sources/ast/index.html that was specifically designed to store AST's and look how similar it is to .proto files. (As an aside, programmers have spent the last 15 years or so attempting to use XML in this role of "generic language-independent tree structured serialization format," but it wasn't the right fit because most data is not markup. Protocol Buffers can deliver on everything people wanted XML to be). Why should manipulating syntax trees require us to write in syntax trees? The answer is that it shouldn't, but this is non-obvious because of how inconvenient parsers currently are to use. One of my life's goals is to help change that. If you find this intriguing, please feel free to follow: https://github.com/haberman/upb/wiki https://github.com/haberman/gazelle