DeepSeek-R1

Hi, this is Tianyi from DeepSeek AI's Harness team. I'm looking to get in touch with someone from GitHub, as I found it weird that https://github.com/trending page doesn't seem to include https://github.com/deepseek-ai/deepseek-harness at all. The project was rapidly being starred, forked, and plugin'ed by the GitHub community in the past few days. I wonder whether we were false positive'ed by some mechanism or something. Some colleague told me that they vaguely remembered that other popular DeepSeek repositories such as https://github.com/deepseek-ai/DeepSeek-R1 also didn't show up in the trending page when it was in fact trending in the community. (I didn't fact check that though.) Is there some way this can be resolved? Happy to help in some way that I can. Feel free to reach out by email via [my first name at deepseek dot com].

Please don't slander the most open AI company in the world. Even more open than some non-profit labs from universities. DeepSeek is famous for publishing everything. They might take a bit to publish source code but it's almost always there. And their papers are extremely pro-social to help the broader open AI community. This is why they struggle getting funded because investors hate openness. And in China they struggle against the political and hiring power of the big tech companies. Just this week they published a serious foundational library for LLMs https://github.com/deepseek-ai/TileKernels Others worth mentioning: https://github.com/deepseek-ai/DeepGEMM a competitive foundational library https://github.com/deepseek-ai/Engram https://github.com/deepseek-ai/DeepSeek-V3 https://github.com/deepseek-ai/DeepSeek-R1 https://github.com/deeps

It is hard to compare models with humans so not sure how to answer it for both. :) But, for models, this is an interesting finding because a lot of LRMs are LLMs with a _bunch_ of post-training done on top. We know this about DeepSeek R1 (one of the models evaluated in the Apple paper) for sure. They write extensively about how they took DeepSeek-V3-Base and made R1 with it. [1] If the post-training is resulting in lower performance on simpler tasks then it ought to inspire more research on how to make it so that it doesn't -- i.e., with more training (of any kind), we should be gaining more capabilities. This has been a problem with DNNs historically, btw. We had these issues when fine-tuning text/image classifiers as well. Some weight changes can be destructive. So, it has to be done with a _lot_ of care. And, I am sure folks are workin

I don't know if they'd changed the submission title or what, but it says quite explicitly "Deepseek R1 Distill 8B Q40" which is a far-cry from "Deepseek R1" which would be misrepresenting the result, indeed. However, if you refer to Distilled Model Evaluation[1] section of the official R1 repository, you will note that DeepSeek-R1-Distill-Llama-8B is not half-bad; it supposedly out-performs both 4o-0513 and Sonnet-1022 on a handful of benchmarks. Remember sampling from formal grammar is a thing! This is relevant, because llama.cpp has GBNF, and lazy grammar[2] setting now, which is making it double not-half-bad for a handful of use-cases, less of all deployments like this. That is to say, the grammar kicks in after </think>. Not to mention, it's always subject to further fine-tuning: multiple vendors are now offering "RFT" services, i.e.

We kind-of have that in DeepSeek-R1-zero [1], but it has problem. From the original authors: > With RL, DeepSeek-R1-Zero naturally emerged with numerous powerful and interesting reasoning behaviors. However, DeepSeek-R1-Zero encounters challenges such as endless repetition, poor readability, and language mixing. A lot of these we can probably solve, but as other have pointed out we want a model that humans can converse with, not an AI for the purpose of other AI. That said, it seems like a promising area of research: > DeepSeek-R1-Zero demonstrates capabilities such as self-verification, reflection, and generating long CoTs, marking a significant milestone for the research community. [1] https://github.com/deepseek-ai/DeepSeek-R1

Can you provide a link to the comment? R1's technical report (https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSee...) says the prompt used for training is "<think> reasoning process here </think> <answer> answer here </answer>. User: prompt. Assistant:" This prompt format strongly suggests that the text between <think> is made the "reasoning" and the text between <answer> is made the "answer" in the web app and API (https://api-docs.deepseek.com/guides/reasoning_model). I see no reason why deepseek should not do it this way, if not considering post-generation filtering. Plus, if you read table 3 of the R1 technical report, which contains an example of R1's chain of thought, its style (going back to re-evaluating the problem) resembles what I actually got in the COT in the web app.

Is there any good quick summary of what's special about DeepSeek? Yes, section 2.3 of the Deepseek R1 paper summarizes the training part you're asking about, in less than a page. https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSee...

DeepSeek published a bunch of benchmarks when they released the models: https://github.com/deepseek-ai/DeepSeek-R1?tab=readme-ov-fil... I'd like to see detailed benchmarks run by other unaffiliated organizations.

Yeah, they're so clear in fact that they call the distilled models "R1" in the url and everywhere on the page[1], instead of using the "DeepSeek-R1-Distill-" prefix, as DeepSeek themselves do[2]. [1]: https://ollama.com/library/deepseek-r1 [2]: https://github.com/deepseek-ai/DeepSeek-R1#deepseek-r1-disti...

If you’re referring to what I think you’re referring to, those distilled models are from deepseek and not ollama https://github.com/deepseek-ai/DeepSeek-R1

Cool project, but the icon looks like DeepSeek [^1] [1]: https://github.com/deepseek-ai/DeepSeek-R1

I see it in the "2. Model Summary" section (for [2]). In the next section, I see links to Hugging Face to download the DeepSeek-R1 Distill Models (for [3]). https://github.com/deepseek-ai/DeepSeek-R1?tab=readme-ov-fil... https://github.com/deepseek-ai/DeepSeek-R1?tab=readme-ov-fil...