Topic: seo

• Making a website readable for agents: robots.txt, sitemaps and llms.txt

  Agents and crawlers find content through a small set of conventions: robots.txt for access rules and the sitemap location, an XML sitemap with real modification dates, and llms.txt as a short curated guide; none of them replaces authentication.

  article · en · unreviewed

Machine-readable: JSON