hlb-CIFAR10

ral benchmarks, nearly tripling the speed of a longstanding CIFAR10 world record (~17.1s -> 6.3s, https://github.com/tysam-code/hlb-CIFAR10, recognized by Karpathy: https://x.com/karpathy/status/1620103412686942208), setting a modded-nanogpt speed record and helping with several others (5.03m -> 4.66m, https://github.com/KellerJordan/modded-nanogpt/blob/master/r... ), improving the relative loss of DiLoCo speedrunning on modded-nanogpt by over 40% (https://x.com/hi_tysam/status/1928561266533990587), and miscellaneous other misadventures throughout the years. I’m looking to be challenged in a strong technical environment with other people who I can learn from, and I’m open to work on most interesting problems that I think are neutral or bring something positive to the world. I’m very hard-problem oriented, and am quite skilled at distilling complex, slow, brittle training procedures down to simple, performant, and hackable ones. The kinds of problems that engage me the most are ones where I can always be growing and learning, and that’s the kind of environment that I’ll thrive in. Learning new subdomains (molecular, etc) is something that keeps me driven, I love the process of expanding out my toolkit. I also very much enjoy working on problems well within my domain of expertise. My focus is on data/architecture/training dynamics and debugging for training efficiency more than raw low-level kernel creation, though the two go hand-in-hand. I enjoy mentoring people quite a bit! I am looking to learn working with larger-scale distributed runs, scale is something that I’d like to work with a bit more. If you think this may align with what you’re doing – say hi! I’d love to chat.

status/1620103412686942208), setting a modded-nanogpt speed record (5.03m -> 4.66m, https://github.com/KellerJordan/modded-nanogpt/blob/master/r... ), improving the relative loss of DiLoCo speedrunning on modded-nanogpt by over 40% (https://x.com/hi_tysam/status/1928561266533990587), and miscellaneous other misadventures throughout the years. I’m looking to be challenged in a strong technical environment with other people who I can learn from, and I’m open to work on most interesting problems as long as they’re roughly net neutral or positive. I’m very hard-problem oriented, and am quite skilled at distilling complex, slow, brittle training procedures down to simple, performant, and hackable ones. The kinds of problems that engage me the most are ones where I can always be growing and learning, and that’s the kind of environment that I’ll thrive in. I’m certainly open to learning new subdomains (molecular, etc). I tend to focus more on data/architecture/training dynamics than raw low-level kernel stuff, though the two tend to go hand-in-hand. I have a tendency to enjoy mentoring people, casual mentorship is something that I greatly enjoy. I am looking to learn working with larger-scale distributed runs, scale is something that I’d like to work with a bit more. I’m keeping an eye open for a good opportunity that’s a good mutual fit for both sides, so I will approach most interview processes thoughtfully. If you think you have something that would be a good fit for us both – say hi!