JAjampekka·Nov 15, 2023
In the notebook context the main gripe is that notebooks have the "invisible" memory state that means that one can't deduce from the notebook code what it actually does. Or more concretely the order of execution of the cells affects what the notebook does. This leads to sort of higher level side-effects. With usual side effects you get spaghetti, with notebooks you get moving spaghetti in five dimensional space.
Immutability and idempotencency are good, and related, ideals too, although I think these can get too "unergonomic" if taken too dogmatically (like in Haskell or Redux), they should be used with almost goto-level discretion.
Of course there's the clear (short term) usability benefit of maintaining the memory state in that stuff doesn't have to be recomputed. But we can have that benefit and be stateless with pure functions and mem
Read the rest
oization. I quite often whip up a buggy and brittle ad-hoc solution to do so. There was also the IncPy project [1] that did this more rigorously, but it hasn't been updated in 13 years.
In general I'm a bit baffled why pure function memoization is so rarely used or proposed. Despite the old adage, cache invalidation is not actually half of the three hard problems in CS. With pure functions it's trivial.
Another baffle is why snapshotting/change tracking (and compressing) file systems haven't caught on. Instead these tend to get implemented badly in any sufficiently complicated application.
[1] https://github.com/pajju/IncPy