TinyZero

This is really not true my friend. I would love to help you if I had some more time, but let me look for a tutorial. Because the FineWeb Dataset is already super good. You can train 7B or 32B Param models at home The >600B Param model isn't really using all the data effectively, but with a MacStudio Farm you can also train that one at home (if you have enough money to buy at least 100). Here's the easy way: https://github.com/FareedKhan-dev/train-deepseek-r1 More details: https://www.bentoml.com/blog/the-complete-guide-to-deepseek-... Here's how DeepSeek-R1-Zero was built, basically from 0 to Hero, including weights the FULL Training Data and everything you need to get it running locally or on servers.https://medium.com/@GenerationAI/how-deepseek-r1-zero-was-re... For $30 USD you can also train a small DeepSeek at home! https://github.com

Yes, the authors explicitly highlighted those two points in the abstract, in terms of them being the elicitation threshold for complex reasoning, namely, an extremely complete pre-trained foundation model, and a set of extremely high quality examples post-training. To your question on finetuning on the initial 10 million pool - intuitively, it would require tremendous amount of finetuning data to move the needle - you really won't be able to move the gradients much with just 817 examples, that initial pool is effectively enforcing pretty rigid regularization. There is now an increasing interest in showing that small data with inference time scaling is providing significant yield. Couple of recent examples: * TinyZero: https://github.com/Jiayi-Pan/TinyZero * s1 Simple Test Time Scaling: https://arxiv.org/abs/2501.19393

The weekly trending python project on github (https://github.com/trending/python?since=weekly) is *verl* (https://github.com/volcengine/verl) This framework is used by Bytedance to train RL reasoning models that achieves openAI O1 level performance on olympics math problems. Code is open sourced. Now many people are building on top of it to reproduce deepseek R1: - https://github.com/Jiayi-Pan/TinyZero (8.6k stars): reproducing ds-r1 zero on countdown tasks for just $30, with a model with 3 billion parameters - https://github.com/ZihanWang314/ragen (700 stars): ex-deepseek researcher use verl to build open source LLM + RL + agent model training framework Reinforcement learning is back as the hottest topic now. Why was it not popular in previous years?

They do mention it here > Note: This blog is inspired by Jiayi Pan [1] who initially explored the idea and proofed it with a small model. I might have written it as > Note: This blog is inspired by Jiayi Pan [1] who also reproduced the "Aha Moment" with their TinyZero [2] model. [1] https://x.com/jiayi_pirate/status/1882839370505621655 (1.1M views btw) [2] https://github.com/Jiayi-Pan/TinyZero A lot of people are busy reproing R1 right now. I think this is the spark.

https://github.com/Jiayi-Pan/TinyZero what about this one?

Guys I'm sorry but it appears the substack did NOT link to the original authors which is NOT acceptable! Credit GitHub: https://github.com/Jiayi-Pan/TinyZero Source on X: https://x.com/jiayi_pirate/status/1882839370505621655

@dang please link to either the GitHub https://github.com/Jiayi-Pan/TinyZero or the primary source twitter thread: https://x.com/jiayi_pirate/status/1882839370505621655

Reposting the comment from https://news.ycombinator.com/item?id=42843959: This is blogspam of https://github.com/Jiayi-Pan/TinyZero and https://nitter.lucabased.xyz/jiayi_pirate/status/18828393705.... This also doesn't mention that it's for one specific domain (playing Countdown). See also https://news.ycombinator.com/item?id=42819262.

Unless I missed it, it seems strange that the article wouldn’t link to the Github repo for the TinyZero model. https://github.com/Jiayi-Pan/TinyZero

This is blogspam of https://github.com/Jiayi-Pan/TinyZero and https://nitter.lucabased.xyz/jiayi_pirate/status/18828393705.... This also doesn't mention that it's for one specific domain (playing Countdown). See also https://news.ycombinator.com/item?id=42819262.

This is indeed a massive exaggeration, I'm pretty sure the $30 experiment is this one: https://threadreaderapp.com/thread/1882839370505621655.html (github: https://github.com/Jiayi-Pan/TinyZero). And while this is true that this experiment shows that you can reproduce the concept of direct reinforcement learning of an existing LLM, in a way that makes it develop reasoning in the same fashion Deepseek-R1 did, this is very far from a re-creation of R1!