intrinsic-source-citation

It's just training the source association together with the training set: https://github.com/mukhal/intrinsic-source-citation The only 2 reasons big LLM providers refuse to do it is 1) to prevent a long slew of content creators filing class action suit. 2) to keep regulators in the dark of how feasible and actionable it would be, once regulators are aware they can perform the source-aware training themselves

Wait for the first large scale LLM using source-aware training: https://github.com/mukhal/intrinsic-source-citation This is not something that can be LoRa finetuned after the pretraining step. What we need is a human curated benchmark for different types of source-aware training, to allow competition, and an extra column in the most popular leaderboards, including it in the Average column, to incentivice AI companies to train in a source aware way, of course this will instantly invalidate the black-box-veil LLM companies love to hide behind so as not to credit original authors and content creators, they prefer regulators to believe such a thing can not be done. In meantime such regulators are not thinking creatively and are clearly just looking for ways to tax AI companies, and in turn hiding behind copyright complications as an excuse to

This paper is about making LLMs better at saying where they got their information from. Normally, LLMs learn a lot of stuff during their training but don’t remember where they learned it. The researchers are working on a way to fix that by teaching the models to cite their sources. This can make the models more transparent, easier to understand, and more reliable. To do this, they modify pretraining to teach the model to link bits of knowledge to specific pretraining documents. After pretraining, they teach the model how to cite these documents when generating answers. Code: https://github.com/mukhal/intrinsic-source-citation