llms-txt

Putting it in .well-known/ was immediately raised as an issue from the beginning; it’s issue #2 in fact: https://github.com/AnswerDotAI/llms-txt/issues/2 It’s been completely ignored ever since.

Resurfacing a proposal I put out on llms-txt: https://github.com/AnswerDotAI/llms-txt/issues/88 We should add optional `tips` addresses in llms.txt files. We're also working on enabling and solving this at Grove.city. Human <-> Agent <-> Human Tips don't account for all the edge cases, but they're a necessary and happy neutral medium. Moving fast. Would love to share more with the community. Wrote about it here: https://x.com/olshansky/status/2008282844624216293

How about following the precedent of all of these users of /.well-known/ https://en.wikipedia.org/wiki/Well-known_URI#List_of_well-kn... robots.txt was created three decades ago, when we didn’t know any better. Moving llms.txt to /.well-known/ is literally issue #2 for llms.txt https://github.com/AnswerDotAI/llms-txt/issues/2 Please stop polluting the web.

> You can't just put things there any time you want - the RFC requires that they go through a registration process. Actually, I can for two reasons. First is of course the RFC mentions that items can be registered after the fact, if it's found that a particular well-known suffix is being widely used. But the second is a bit more chaotic - website owners are under no obligation to consult a registry, much like port registrations; in many cases they won't even know it exists and may think of it as a place that should reflect their mental model. It can make things awkward and difficult though, that is true, but that comes with the free text nature of the well-known space. That's made evident in the Github issue linked, a large group of very smart people didn't know that there was a registry for it. https://github.com/AnswerDotAI/llms-txt/iss

> I paste this https://svelte.dev/llms-full.txt Intrigued, are there there other tools that provide this kind of /llms-full.txt document to load into your system promtpt/use for RAG? edit: I found this which seems to be similar https://github.com/AnswerDotAI/llms-txt and https://llmstxt.site/ to find similar files to give your own LLMs context on various tools

Cool. To be totally honest, I'd feel a lot better if we could just extend robots.txt especially given that we finally have a standard for it with IETF RFC 9309: https://datatracker.ietf.org/doc/html/rfc9309 Also might be worth looking at llms.txt (https://github.com/AnswerDotAI/llms-txt), though it's aimed to be more of an agent-friendly sitemap.xml alternative.

llms.txt has a section on "Existing standards" which completely forgets about well-known[0], there's an issue opened three months[1] ago but it seems it was ignored. [0] https://en.wikipedia.org/wiki/Well-known_URI [1] https://github.com/AnswerDotAI/llms-txt/issues/2