Tricycle

I wanted to share Tricycle (https://github.com/bclarkson-code/Tricycle), a deep learning framework I built completely from scratch from Autograd to a GPT. I wanted a library that is fast and feature rich enough to train actual models while being simple enough that anyone with a bit of python experience can understand what is going on. The biggest milestone so far is training GPT-2 (124M) on 2.3B tokens in just under 3 days on my GPU (RTX 3090). So far, I've added the following to Tricycle: - An automatic differentiation engine - General matrix operations with einsum - Standard network layers (Dense, ReLU, GeLU etc) - Transformer blocks (MultiHeadSelfAttention and MLP blocks) - Optimisers (SGD, AdamW) - GPT-2 - etc The project is still under active development, I'm in the process of adding mixed precision and multi-gpu support with the goa

Location: London Remote: Happy to work fully in person although hybrid is preferred. Willing to relocate: For the right role Technologies: Machine Learning (Language modelling, Computer Vision etc), Data Science, Python, Kubernetes Resume: https://raw.githubusercontent.com/bclarkson-code/bclarkson-c... Linkedin: https://www.linkedin.com/in/benedict-clarkson-779b43154/ Email: bclarkson-code@proton.me I'm a Senior Machine Learning Engineer looking for my next challenge. I have a strong background in Deep Learning: I built a state of the art CV model for cancer detection and I'm the author of the Tricycle (https://github.com/bclarkson-code/Tricycle) deep learning framework. I built it from scratch (nothing but python, numpy and a bit of cuda) and recently used it to successfully train a GPT-2 small model from scratch. I'm looking for an AI E

Location: London Remote: Happy to work fully in person although hybrid is preferred. Willing to relocate: For the right role Technologies: Machine Learning (Computer Vision, NLP etc), Data Science, Python, Kubernetes Resume: https://raw.githubusercontent.com/bclarkson-code/bclarkson-c... Linkedin: https://www.linkedin.com/in/benedict-clarkson-779b43154/ Email: bclarkson-code@proton.me I'm a Senior Machine Learning Engineer looking for my next challenge. I have a strong background in Deep Learning: I built a state of the art CV model for cancer detection and I'm the author of the Tricycle (https://github.com/bclarkson-code/Tricycle) deep learning framework. I built it from scratch (nothing but python, numpy and a bit of cuda) and recently used it to successfully build and train a small language model. I'm looking for an AI Engineer role in

As a chronic premature optimizer my first reaction was, "Is this even possible in vanilla python???" Obviously it's possible, but can you train an LLM before the heat death of the universe? A perceptron, sure, of course. A deep learning model, plausible if it's not too deep. But a large language model? I.e. the kind of LLM necessary for "from vanilla python to functional coding assistant." But obviously the author already thought of that. The source repo has a great motto: "It don't go fast but it do be goin'" [1] I love the idea of the project and I'm curious to see what the endgame runtime will be. [1] https://github.com/bclarkson-code/Tricycle