Rezoom.SQL

uples! So they have functions like `pipe3` instead: pipe3 parser1 parser2 parser3 (fun x y z -> ...) Still, what a pain when you mix in some "ignored" parts in between the "captured" parts. Ugly and awkward to follow. pipe3 (whitespace >>. parser1) (char '(' >>. parser2 .>> char ')') (whitespace >>. parser3 .>> whitespace) (fun x y z -> ...) I wrote a library some years ago to alleviate my issues with that F# flavor of this problem. My solution is very similar to yours in that it also asks the user to think of it as like regex "capture groups"! https://rspeele.github.io/FParsec-Pipes/Intro.html My approach is more symbolic-operator flavored than yours though. In mine you always start a chain of parsers with `%%`, separate with `--` and end with `-|> function`. %% parser1 -- parser2 -- parser3 -|> fun () -> ... If you want to capture any part of the chain you use the unary prefix operator `+.` to mark it captured and it will get passed through to the consuming function. %% +.parser1 -- parser2 -- +.parser3 -|> fun p1 p3 -> ... I used it extensively in a SQL parser: https://github.com/fsprojects/Rezoom.SQL/blob/master/src/Rez...

the programs I use daily, how many have I ever even glanced at the source code? I could count them on my fingers. And I'm in the target audience: a programmer, enthusiastic about the trade, one who has contributed to open source... Conclusion: within a rounding error of NOBODY cares about the code. Of the weavers who were displaced by the automated looms, or the furniture-makers displaced by mass produced pressboard, some of the most skilled could still ply their trade for the rich who wanted to flex their bespoke handmade goods. Programmers writing code by hand will not even have that luxury market. It makes me depressed but I don't see any way around it. My hope is that I can use my knowledge, willingness to spend long hours fighting with a computer, and industry experience to at least make better software with agents than the average vibecoder. I sadly don't think I'll get anywhere trying to play John Henry vs. the Steam Drill, trying to beat it with my hand-typed code. Maybe somebody who's 10x the dev I am can do that, and Godspeed to them, but I thought I was pretty decent and I don't have a chance. Hell, it's not even possible to prove any code was written by hand, other than by having uploaded it pre-2024ish. For this reason I really feel bad for those coming up right now, who have the same enthusiasm and fascination for Real Programming that I did, but no way to prove themselves and no motivation to in a world that values that skill less all the time.

rder to trace in the rewritten, bulk forms of the functions, but they are efficient. Next the customer hits you with a request like "let's have a smart-frob function that works on all the selected foos. For foos that are red, it frobs them, if they are blue, it fizzles them". Now you have to bulk-load to select the redness or blueness of all your Foos, build two separate lists, red and blue, then call your bulk-frob and bulk-fizzle functions accordingly on the two lists. Again the machinery to turn the requirement into a batch-shaped thing is not a lot, but it does kind of obscure the original business requirement. At various times in the life of the project you will have a feature that starts as a "always done on one Foo" thing because it's triggered by a button on the detail screen. Then somebody will possibly come along and want to do it in bulk later and you have to rewrite the implementation. Unless you have very strict code review that everything MUST be written in batch-style taking a list of IDs up to the API layer. I wrote a library[1] many years ago to solve this problem and allow the straightforward, non-batch versions of the functions to be automatically batchable. The idea is kind of like what React did for frontend dev: React was not faster than mutating the page with jQuery soup, but it was much faster than replacing the entire DOM on every render, and it let you write your code as if that was what you were doing. That was a very simple mental model and much less buggy than jQuery soup. The idea of my library was basically borrowed from other functional languages with a resumption monad, meaning that instead of an opaque async task to go do a thing, you have a "plan" which could either be a. done or b. waiting on some errand that requires firing off a query. If you have a list of plans like from a loop, you could step all of them to the next errand they are waiting on, then fire those off in a batch. So plans could be composed linearly or "batch-style" depending on your preference[2]. What makes it very powerful is the combination with an F# type provider that could analyze your SQL and automatically determine a caching profile for each query. It knows what tables the query reads from, what tables it writes to, whether it uses any impure functions like random(), etc. So within one transaction, it wouldn't re-run the same pure query again, it would pull the results from a local cache -- except if another command issued in that transaction updates those tables, the cache is automatically invalidated. This solves the other code smell that starts to accumulate as you try to write efficient database code in a complex app -- keeping materialized objects loaded in memory and passing them around to other functions so they don't have to re-query for them. Anyway, it was a little too weird to catch on, and I was a little too burnt out to maintain it. [1]https://github.com/fsprojects/Rezoom.SQL [2]https://fsprojects.github.io/Rezoom.SQL/doc/Rezoom/README.ht...

the code myself with full understanding of what was going on. Now, it had been many years and the codebase did some pretty esoteric stuff. I was the only person in the world who ever knew what that code did, and I didn't know it like the back of my hand anymore. Particularly the stuff with runtime CIL generation for result set processing. I did a lot of reading my own code and a lot of thinking about it to get that understanding back, but it was a very helpful tool to be able to bounce my questions off an agent during the process and then investigate its answers for accuracy. And greater than 90% of the time its answers turned out to be correct. [1]https://github.com/fsprojects/Rezoom.SQL

programming language. "Expression" breaks down into cases: integer literal, string literal, variable name, binary operations like add(expr1,expr2), unary operations like negate(expr), function call(functionName, exprs), etc. Clearly all of these expression subtypes should belong to a base type `Expression`. But what methods do you put on `Expression`? If you're writing a compiler, you have to walk this syntax tree many times for very different purposes. First you might do a pass on it where you "de-sugar" syntax, then another pass where you type-check it and resolve names in the code, then another pass where you generate assembly code from it. Perhaps your compiler even supports different backends so you have a code-gen path for x86, another for ARM, etc. You'll likely want a pretty-printer so you can do automatic reformatting, maybe you want linting support, etc. If you look at all those concerns and say that each subtype of `Expression` must implement methods for each one, then you end up with untenable code organization. Every expression subtype now has a huge stack of methods to implement all in one file, dealing with stuff from totally different layers of the compiler. It's a mess. It's much cleaner to have the "shape" of the expression defined in one place without all that clutter, and then in each of those areas of the code you can write methods that consume expressions however they need, so each of those separate concerns lives in its own silo. ------------------------------------------------ Some real code (but it's F# not C#) to look at. AST for my SQL dialect: https://github.com/fsprojects/Rezoom.SQL/blob/master/src/Rez... Typechecker code: https://github.com/fsprojects/Rezoom.SQL/blob/master/src/Rez... Backend code that outputs MS TSQL from it: https://github.com/fsprojects/Rezoom.SQL/blob/master/src/Rez... ------------------------------------------------ If you're an old hand at OO you may be familiar with its actual answer to this problem, the "Visitor" pattern. See System.Linq.Expressions.ExpressionVisitor. However, once you've used a language with good union and pattern matching support, this feels like a clunky hack. Basically the mirror image of a language without real object orientation imitating it by passing around closures and structs-of-closures. ------------------------------------------------ It doesn't just have to be compiler stuff. A business app data model can use this too. Instead of having: public class DbUser { public EmailAddress Email { get; set; } public PasswordHash? Password { get; set; } // null if they use SSO public SamlEntityProviderId? SamlProvider { get; set; } // null if they use password auth } You could have: type UserAuth = | PasswordAuth of PasswordHash | SSOAuth of SamlIdentityProviderId The implementation details of those different auth methods, the UI for them, etc. don't have to be part of the data model. We do have to model what "shapes" of data are acceptable, but "doing stuff" based on those shapes is another layer's problem.

to the SQL one, but since they really are different under the covers, it's easy to produce hefty queries. For example a LINQ "group by" frequently cannot be translated to a SQL "group by". I tried to keep it simple and not deviate too far from straight SQL. But I still didn't want to force processing those damn row-based outputs onto the programmer, so I at least had to add notation for breaking out the columns of a top-level result set into an object hierarchy: https://rspeele.gitbooks.io/rezoom-sql/doc/Language/Navigati...

omplex SELECT syntax. Filtering results? That's a clause of SELECT. Ordering? Clause of SELECT. Filtering after aggregating? Oh, that's a different clause of SELECT. Once this philosophy has infected the brain of a SQL implementer, it spreads like wildfire. That's why you even see custom syntax pop up even in good'ol function calls sometimes, like in Postgres: overlay('abcdef' placing 'wt' from 3 for 2). SQL fans often talk about the beauty of relational algebra. Once you achieve relational enlightenment, SQL is supposed to be beautiful. But if we wrote math like SQL, you wouldn't say 2 * 3 + 4. There would be a grand COMPUTE statement with clauses for each operation you could wish to perform. So you'd write COMPUTE MULTIPLY 2 ADD 4 FROM 3. Of course, the COMPUTE statement is a pipeline, and multiplication comes after addition in the pipeline, so if you wanted to represent 2 * (3 + 4) you'll push that into a sub-compute, like COMPUTE MULTIPLY 2 FROM (COMPUTE MULTIPLY 1 ADD 4 FROM 3). SQL clauses could have been "functions" with well-defined input and output types, if the language designers had come up with a type system to match the relational algebra. WHERE - table<row type> -> predicate<row type> -> table<row type> JOIN - table<row x> -> table<row y> -> predicate<row x, row y> -> table<row x, row y> ORDER - table<row type> -> list<expression<row type>> -> table<row type> These could be pipelined, rather than nested, either with an OO-style method call syntax or a functional style pipe operator. You understand the idea. Again, as you've mentioned, the LINQ methods[1] are a great resource for people not familiar with this style. But, counterargument. Languages with minimal syntax and great power are often claimed to be unreadable. Sometimes it's nice to have special syntax, to help give a recognizable shape to what you're reading, instead of it being operator / function call soup. So what did SQL accomplish by making everything a special case of SELECT? Well, it's got this sensible flow to every statement. You see, the execution of SELECT logically flows as I've numbered the lines below. SELECT 8 DISTINCT 7 TOP n 5 column expressions 1 FROM tables 2 WHERE predicate 3 GROUP BY expressions 4 HAVING expressions 6 ORDER BY ... I sure am glad they cleared that up. If it had been a chain of individual operations I'd have been utterly baffled. Ignoring completely the syntactical design, the lack of basic operations is a pain too. Why can't I declare variables within queries? I often would like to do something like: select let x = compute_something(...) in x + q as column1 x + r as column2 from ... Instead, when I really need that, I end up wrapping the whole thing in an outer query and computing X in the inner query. Hooray for SELECT, the answer to all problems! Oh and yes, views and stored procedures and UDFs are no answer to the need for one-off local composability within queries. Then you have the sloppy design of the type system in virtually all SQL dialects. SQL Server doesn't even have a boolean type. There is no type you can declare for `@x` that will let you `set @x = (1 <> 2)`. And don't even get me started on the GROUP BY clause, the design of which contorts the whole rest of the language. If you GROUP BY some columns, you must not refer to any non-grouped columns in the later parts of your query (refer to my table above for which parts are "later"). Unless, that is, you are referring to them in aggregates. Then the HAVING clause was tacked on so that you'd have a way to do a filter -- the same thing as WHERE -- after the GROUP BY. Does it all make sense once you understand it? Yes, in that you can see how you'd end up with this system if you were adding things piece by piece and never went back to redesign from square one. Wow, I have a lot of ranting to do about SQL. I feel like I haven't even scratched the surface. And hell, I still pick SQL databases every time I start a project! The damn language is useful enough and the products work great. But it has all the design elegance of the US tax code. [0] https://github.com/rspeele/Rezoom.SQL [1] https://docs.microsoft.com/en-us/dotnet/api/system.linq.enum...