tinker-cookbook

https://thinkingmachines.ai/tinker/ https://github.com/thinking-machines-lab/tinker-cookbook (As a TML person, I'm obviously biased, but I couldn't resist because of "tinkering"). TBF it's hard to imagine a real architecture change that wouldn't require a ton of compute, but you could certainly fine tune and play with different recipes, loss functions, etc. And Claude can carry you a lot of the way through doing this. One fun task is to invent a tool and then train a small model to use it. You could export that small model and run it locally for free forever to do your thing. I think this is what a lot of Software Engineering will look like later. There are a lot of other high level abstractions here to look at. Prime Intellect has one. The other thing to play with is self-hosting small models, but IMO most of the interesting stuff is act