NLnly·Oct 12, 2014
Much of the advantage of using a DFA generator like Ragel is lost because the HTTP header grammar is actually ambiguous in several places and can't be streamed. You can use it as a component, but it isn't entirely sufficient on its own.
The HTTP 1.1 RFC requires whitespace be stripped at the end of header values, yet also permits (although deprecates) header folding, giving rise to the following ambiguity (using _'s in place of leading spaces):
Foo: Hello\r\n
______\r\n
________\r\n
________ world!\r\n
Bar: smeg
This requires that the parser buffer all the whitespace between 'Hello' and 'world!' (and the RFC doesn't put a standard limit on header value length) just in case 'world!' never comes and the value of the Foo header has to be stripped back to just "Hello"
Here's a related observation from a commit[0] by the
Read the rest
Joyent guys, who wrote the streaming parser used by NodeJS: "For http-parser itself to confirm[sic] exactly would involve significant changes in order to synthesize replacement SP octets. Such changes are unlikely to be worth it to support what is an obscure and deprecated feature"
Another example is parsing the Request and Status lines:
GET <uri> HTTP/1.1
Technically <uri> can't contain spaces, but the RFC says you MAY accept them [RFC7230: 3.5. Message Parsing Robustness] ... which then gives rise to the possibility of <uri> containing the literal string " HTTP/1.1", and ultimately opens up bad user agents that send spaces to header injection.
Resolving these ambiguities require implementing your own buffering, and dropping down to Ragels 'state charts' feature to avoid your semantic actions being munged... which leaves you to design the top level state machine yourself.
[0] https://github.com/joyent/http-parser/commit/5d9c3821729b194...