llmstxt

The link https://github.com/mandarwagh9/llmstxt is broken, and gives 404. I assume this is to help tool-enabled agents navigate websites? I'm curious how this balances against the otherwise anti-LLM position a lot of website owners / artists are taking.

Hey HN, I just built a small web app that helps you generate /llms.txt files for your website: https://llmstxt-cyuh.vercel.app GitHub: https://github.com/mandarwagh9/llmstxt What is /llms.txt? Think of it like robots.txt, but for large language models. It’s a new emerging standard that gives LLMs a clean, Markdown-based map of your most important content (docs, guides, APIs). Instead of crawling bloated HTML, they get straight to the point. The idea was popularized by Jeremy Howard and is gaining adoption across devtool companies like LangChain and Mintlify. Why I built this I wanted a dead-simple way to auto-generate /llms.txt from any docs site or URL. This tool scrapes your site, extracts key Markdown links, and gives you a copy-paste-ready llms.txt. No setup, no install, just paste your URL and download. Stack Frontend: Vercel + vani