gpt-3

early happening. https://arxiv.org/abs/2108.13349. If you trained a model on the Catalan tokens GPT-3 was trained on alone, you'd just get a GPT-2 level gibberish model at best. I don't doubt you, i just don't think it means what you think it means. As for papers, these are some interesting ones. How do languages influence each other? Studying cross-lingual data sharing during LLM fine-tuning - https://arxiv.org/pdf/2305.13286 Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer - https://arxiv.org/abs/2404.04042 Zero-shot cross-lingual transfer in instruction tuning of large language models - https://arxiv.org/abs/2402.14778 Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment - https://arxiv.org/abs/2305.05940 It's not like there is perfect transfer but the idea that there's none at all seemed so ridiculous to me (and why i asked the first question). Models would be utterly useless in multilingual settings if that were really the case. Like the 3rd paper, GPT's instruction tuning is done basically only in English - https://mobile.twitter.com/janleike/status/16252072516309606...