docs: improve LLM discovery surfaces

.github/workflows/deploy-site.yml

@@ -171,6 +171,12 @@ jobs:
           if [ -f website/build/llms-full.txt ]; then
             cp website/build/llms-full.txt _site/llms-full.txt
           fi
+          if [ -f website/build/robots.txt ]; then
+            cp website/build/robots.txt _site/robots.txt
+          fi
+          if [ -f website/build/sitemap.xml ]; then
+            cp website/build/sitemap.xml _site/sitemap.xml
+          fi
 
       - name: Upload artifact
         uses: actions/upload-pages-artifact@56afc609e74202658d3ffba0e8f6dda462b719fa  # v3

README.md

@@ -117,6 +117,7 @@ hermes doctor       # Diagnose any issues
 ```
 
 📖 **[Full documentation →](https://hermes-agent.nousresearch.com/docs/)**
+🧠 **LLM entry points:** [`/llms.txt`](https://hermes-agent.nousresearch.com/llms.txt) · [`/llms-full.txt`](https://hermes-agent.nousresearch.com/llms-full.txt) · docs mirrors at [`/docs/llms.txt`](https://hermes-agent.nousresearch.com/docs/llms.txt)
 
 ---
 

llms.txt

@@ -0,0 +1,36 @@
+# Hermes Agent
+
+> The self-improving AI agent built by Nous Research. It ships with persistent memory, agent-created skills, multi-platform messaging, and a docs site optimized for humans *and* LLMs.
+
+Canonical docs: https://hermes-agent.nousresearch.com/docs/
+Live machine-readable docs:
+- https://hermes-agent.nousresearch.com/llms.txt
+- https://hermes-agent.nousresearch.com/llms-full.txt
+- https://hermes-agent.nousresearch.com/docs/llms.txt
+- https://hermes-agent.nousresearch.com/docs/llms-full.txt
+
+Repo: https://github.com/NousResearch/hermes-agent
+
+## Start here
+- [Installation](https://hermes-agent.nousresearch.com/docs/getting-started/installation)
+- [Quickstart](https://hermes-agent.nousresearch.com/docs/getting-started/quickstart)
+- [Learning Path](https://hermes-agent.nousresearch.com/docs/getting-started/learning-path)
+- [Configuration](https://hermes-agent.nousresearch.com/docs/user-guide/configuration)
+- [CLI](https://hermes-agent.nousresearch.com/docs/user-guide/cli)
+- [Messaging Gateway](https://hermes-agent.nousresearch.com/docs/user-guide/messaging)
+- [Tools & Toolsets](https://hermes-agent.nousresearch.com/docs/user-guide/features/tools)
+- [Memory](https://hermes-agent.nousresearch.com/docs/user-guide/features/memory)
+- [Skills System](https://hermes-agent.nousresearch.com/docs/user-guide/features/skills)
+- [MCP Integration](https://hermes-agent.nousresearch.com/docs/user-guide/features/mcp)
+- [Architecture](https://hermes-agent.nousresearch.com/docs/developer-guide/architecture)
+- [FAQ & Troubleshooting](https://hermes-agent.nousresearch.com/docs/reference/faq)
+
+## Why Hermes
+- Runs as a CLI, messaging bot, or service-backed agent
+- Uses a closed learning loop with persistent memory and reusable skills
+- Works across local, Docker, SSH, Daytona, Modal, and Singularity backends
+- Supports 20+ messaging platforms and agent tooling integrations
+
+## Agent-friendly docs
+- The live site mirrors this file at the root and under `/docs/`
+- The full markdown dump is available in `llms-full.txt` for one-shot ingestion

website/docs/index.mdx

@@ -138,4 +138,5 @@ Machine-readable entry points to this documentation:
 - **[`/llms.txt`](/llms.txt)** — curated index of every doc page with short descriptions. ~17 KB, safe to load into an LLM context.
 - **[`/llms-full.txt`](/llms-full.txt)** — every doc page concatenated into a single markdown file for one-shot ingestion. ~1.8 MB.
 
-Both files also resolve at `/docs/llms.txt` and `/docs/llms-full.txt`. Generated fresh on every deploy.
+Both files also resolve at `/docs/llms.txt` and `/docs/llms-full.txt`. They are mirrored at the site root on deploy so generic LLM crawlers and the docs path both work.
+

website/scripts/generate-llms-txt.py

@@ -9,13 +9,16 @@
                                     comments separating files.
 
 Both publish at:
+  https://hermes-agent.nousresearch.com/llms.txt
+  https://hermes-agent.nousresearch.com/llms-full.txt
   https://hermes-agent.nousresearch.com/docs/llms.txt
   https://hermes-agent.nousresearch.com/docs/llms-full.txt
 
 The `/docs/` prefix is not a mistake — Docusaurus serves `website/static/`
-at the `docs/` base path. Clients and IDE plugins that probe the classic
-`/llms.txt` root will miss these. Document the canonical URLs in the docs
-index and in the repo README.
+at the `docs/` base path. The deploy workflow mirrors the generated files
+to the site root as well, so both the classic `/llms.txt` probe and the
+`/docs/llms.txt` docs path work. Document both URLs in the docs index and
+in the repo README.
 
 Called from `website/scripts/prebuild.mjs` on every `npm run start` /
 `npm run build` so the output stays in sync with the docs tree.
@@ -250,6 +253,7 @@ def emit_llms_full() -> str:
         ),
         "Canonical site: https://hermes-agent.nousresearch.com/docs\n",
         "Short index: https://hermes-agent.nousresearch.com/docs/llms.txt\n",
+        "Root alias: https://hermes-agent.nousresearch.com/llms.txt\n",
         "\n---\n\n",
     ]
 

website/static/robots.txt

@@ -0,0 +1,7 @@
+User-agent: *
+Allow: /
+
+# Hermes Agent docs are optimized for both humans and agents.
+# The llms.txt entry points are mirrored at the site root and under /docs/.
+Sitemap: https://hermes-agent.nousresearch.com/sitemap.xml
+Sitemap: https://hermes-agent.nousresearch.com/docs/sitemap.xml