prql

Great questions! I'm not a database expert either but I can try answering these: 1) I think databases like to manage pages directly because the db can make more optimizations than the OS because the db has more context. For example, when aborting a transaction the db knows its dirty pages should be evicted (i'm not sure if mmap offers custom eviction). Also I believe if the db uses mmap, it loses control over when pages are flushed to disk. Flush control is necessary for guaranteeing transaction durability. 2) What you're describing here sounds similar to a LSM-tree database (e.g. RocksDB). They are used often for write-heavy workloads because writes are just appends, but they might not be great for read-heavy things. 3) This reminds me of PRQL[1] (which was trending on Hacker News last week) and Spark SQL. I'm not too familiar with this

Yes this is a good idea. dplyr has something similar with `mutate` & `transmute`. This could _mostly_ be enforced by PRQL. There's a case where we transpile to: select *, x+1 as x_plus_one ...where we don't know whether or not we're overwriting an existing column. But it's a minority of cases, and the contract could stand within PRQL. I opened an issue here: https://github.com/max-sixty/prql/issues/6

Yes, I agree with the downsides of the existing approach, and you're absolutely right re "Languages really should not have any implicit way of constructing identifiers". I'd still say it's fine if it implicitly names output columns, for convenience. I referenced your comment in an issue and listed some options: https://github.com/max-sixty/prql/issues/5

Thanks! I just fleshed out composing CTEs, which is a small step towards the broader goal of making composition easier: https://github.com/max-sixty/prql/commit/dc68fcaaceef26cc078... Let me know if you have a good case of the sort of composition you find difficult in SQL (either here or in an issue). Thank you!

I've made this change [1]. Thank you! [1] https://github.com/max-sixty/prql/commit/dde7fcfc13daaadbdce...

Thanks! > Did you consider using a keyword like `let` for column declarations Yeah, the current design for that is not nice. Good point re the keyword scanning. I actually listed `let` as an option in the notes section. Kusto uses `extend`; dplyr uses `mutate`; pandas uses `assign`. I opened an issue here: https://github.com/max-sixty/prql/issues/2