PLIP: Add llms.txt and llms-full.txt views to plone.restapi

[jensens]

PLIP (Plone Improvement Proposal)

Responsible Persons

Proposer: Jens W. Klein, @jensens

Seconder: Timo Stollenwerk, @tisto

Abstract

Add llms.txt and per-endpoint detail views to plone.restapi that auto-generate AI-friendly documentation of all registered REST API services at runtime. The source of truth are structured docstrings on existing service classes, metadata already present in the ZCA registry (permissions, context interfaces, HTTP methods), and the existing auto-generated request/response examples from the test suite.

No ZCML changes, no new dependencies, no hand-maintained docs to keep in sync.

llms.txt is a lightweight convention — not a formal standard, but one that the agent tooling our users already run looks for by default. This PLIP also defines the introspection substrate (a runtime-enumerable registry of REST services) that other AI-integration efforts — notably the Plone MCP Server PLIP (#4213) — can consume as a single source of truth.

The goal: any AI agent — Claude Code, Cursor, Copilot, Cline, Aider, or a custom pipeline — can fetch /llms.txt from a Plone site and immediately understand what the REST API offers, how to authenticate, and how Plone's content model works, then pull per-endpoint detail on demand rather than ingesting everything up front.

Motivation

AI agents need to understand what an API can do before they can use it. Right now there's no machine-friendly way to discover Plone's REST API capabilities at runtime.

plone.restapi has 119 registered service endpoints, JWT auth, and a comprehensive feature set. The infrastructure is all there — it just doesn't tell AI tools about itself.

Who actually fetches llms.txt (and who doesn't)

Honesty about the landscape, because it shapes the design:

• AI search crawlers (GPTBot, ClaudeBot, PerplexityBot, …) largely ignore /llms.txt and crawl HTML directly. This is not an SEO/answer-engine play, and the PLIP does not claim to be one.
• Coding and IDE agents — Claude Code, Cursor, Windsurf, GitHub Copilot, Cline, Aider — do routinely fetch /llms.txt when pointed at a target. This is a "business-to-agent" (B2A) play, and it is exactly the audience that benefits from a machine-readable description of a specific running site's API.

So the value is real, but narrow and concrete: it serves agents operating against a live Plone instance, not generic AI search visibility.

This matters beyond MCP. The MCP server PLIP (#4213) serves MCP-capable clients with a curated, task-oriented tool surface. An llms.txt serves the complementary case: any agent — including ones with no MCP client — that can fetch a URL and read prose. The two are not competitors; see Relationship to the MCP Server PLIP below.

What makes this different from our docs llms.txt?

We already ship llms.txt/llms-full.txt for the documentation site:

• https://6.docs.plone.org/llms.txt
• https://6.docs.plone.org/llms-full.txt

What the docs site provides:

• A table of contents of Plone's documentation: installation, admin, developer, contributing, …
• Static: generic and identical for every Plone site in the world.
• Helps an AI understand what Plone is and how to develop with it.

What I propose — https://mysite.com/llms.txt:

• Runtime-generated description of this specific site's REST API: actual endpoints available, permissions needed, installed add-ons with their own restapi endpoints, how to interact with this site's content.
• Includes site-specific context (what this site is about, provided by the site admin).
• Dynamic, site-specific: different for every Plone installation.
• Helps an AI operate this specific Plone site: CRUD content, manage workflows, query/search data.

The docs llms.txt is a textbook about Plone. The site llms.txt is the control-panel manual for a specific machine. The validated llms.txt use case today is static docs sites (which we already have); the genuinely novel contribution here is the runtime, per-instance view, justified below by Plone's traversal model rather than by adoption statistics.

Concrete comparison

The existing 6.docs.plone.org/llms.txt (Sphinx-generated, static) is a list of links to documentation pages — useful for learning Plone, but it tells an agent nothing about a specific site.

The proposed mysite.com/llms.txt (runtime-generated, site-specific):

# Plone REST API — kleinundpartner.at

> Website of Klein & Partner, an IT consultancy in Innsbruck, Austria.
> Content is in German and English. The site provides information about
> services, team members, and blog posts about Python/Plone development.
>
> RESTful hypermedia API. All endpoints accept and return application/json.
> Authenticate via JWT token (@login) or HTTP Basic Auth.
> Fetch any endpoint's detail document (linked per entry) for parameters
> and request/response examples — you do not need to read all of them.

## Content (any content object)

- [@content GET](@llms/content-get) (zope2.View): Retrieve a serialized content object
- [@content PATCH](@llms/content-patch) (cmf.ModifyPortalContent): Update a content object
- [@content DELETE](@llms/content-delete) (cmf.DeleteObjects): Delete a content object

## Content (folderish only)

- [@contents GET](@llms/contents-get) (zope2.View): List folder contents with batching
- [POST](@llms/content-post) (cmf.AddPortalContent): Create a new content object

## Workflow (any content object)

- [@workflow GET](@llms/workflow-get) (zope2.View): Get current state and transitions
- [@workflow POST](@llms/workflow-post) (cmf.ModifyPortalContent): Execute a transition

## EasyForm (any easyform object)

- [@easyform-data GET](@llms/easyform-data) (cmf.ModifyPortalContent): Get form submissions

The last section only appears if collective.easyform is installed. A different site with different add-ons gets different output. The site description at the top gives the agent immediate context.

Why llms.txt and Not OpenAPI

Plone uses traversal, not routing. Content lives in a tree, and the URL is the path through that tree. REST API endpoints are appended to the content path (/about/team/@workflow). Which endpoints are available depends on the type of object at that path — a Folder supports @contents and POST, a Document does not.

This is fundamentally at odds with OpenAPI's model, which assumes a fixed set of URL paths like /api/users/{id}. There's no way in the OpenAPI spec to express "this endpoint is available when the object at this path provides IFolderish."

An llms.txt can explain this in prose — and LLMs excel at reading prose and reasoning about it. An AI agent that reads "this works on folders, not documents" understands it immediately. So llms.txt isn't "OpenAPI but simpler" — it's a fundamentally better fit for Plone because it teaches the agent the concepts, not just a path list.

Relationship to the MCP Server PLIP (#4213)

The MCP server PLIP is active and has a working MVP (auth, content CRUD, block creation, workflow transitions, user management, search). This PLIP and that one are complementary, and should share machinery:

• MCP gives MCP-capable clients a curated, task-oriented tool surface with block abstraction and sensible boundaries — deliberately not a 1:1 dump of all 119 endpoints, because dumping raw endpoints into an LLM's context degrades performance.
• This PLIP gives any agent a discoverable, prose-first description of the raw API, plus the introspection substrate (see deliverable Ticket11189 #5) that enumerates registered services and extracts their metadata at runtime.

The integration commitment: the MCP server should be able to generate its tool definitions from the same introspection data this PLIP exposes, rather than maintaining a second, hand-written description of the same endpoints that can drift. This PLIP builds the substrate; #4213 sits on top of it. The two proposers (@jensens, @danalvrz) coordinate on the shared interface.

Avoiding context bloat (design principle)

Current agent-design consensus — echoed in the #4213 thread — is that dumping everything into an LLM's context up front hurts more than it helps. A single document inlining 119 endpoints plus a request/response pair for each would be tens of thousands of tokens, most irrelevant to any given task, and the inlined examples carry test-fixture URLs (localhost:55001/plone/…) that can mislead an agent operating a real site.

Therefore this PLIP deliberately replaces the original single llms-full.txt mega-dump with progressive disclosure:

1. llms.txt — thin index: site description, traversal explanation, one line per endpoint (name, method, permission, context, summary), each linking to its detail document.
2. Per-endpoint detail — fetched on demand via a detail view (e.g. /@llms/<endpoint-id>), returning the full description, parameters, and inlined request/response example for that endpoint only.

A llms-full.txt that concatenates all detail documents MAY still be offered for the rare "give me everything" case, but it is explicitly not the primary or recommended path, and nothing in the design requires an agent to fetch it to be productive.

Assumptions

• plone.restapi remains the primary REST interface.
• Service classes are registered via the plone:service ZCML directive (no change to this); add-ons use the same mechanism and are auto-included via z3c.autoinclude.
• plone.rest will accept a small, enumerable service registry populated during ZCML processing (deliverable Ticket11189 #5). This is a cross-package change — buy-in from plone.rest maintainers is a prerequisite.
• A bespoke docstring convention is acceptable only if aligned with an existing style. This PLIP adopts Google/napoleon-style docstrings (Parameters: etc.) so contributors learn no Plone-only dialect and existing parsers can be reused.
• The views are read-only and add no write surface, but a complete machine-readable map of every endpoint and its required permission is a mild reconnaissance aid — so access is gated by a dedicated permission (see Risks), not assumed universally public.

Proposal & Implementation

1. Site Description via Registry

A plone.registry record (plone.llms_site_description) where a site admin provides a short markdown description of the site, included as the blockquote summary at the top of llms.txt. This is the one piece that can't be auto-generated. If empty, the view falls back to a generic description ("Plone CMS REST API").

A companion boolean record (plone.llms_enabled, default depends on release policy — see Risks) lets an operator disable the views entirely, addressing the opt-out request.

2. Structured Docstring Convention (Google/napoleon style)

A convention on plone.restapi.services.Service subclasses, using a style existing tooling already understands:

class ActionsGet(Service):
    """Get available actions for the current context.

    Returns a list of available workflow transitions and object
    actions for the current content object.

    Category: Workflow
    Examples: actions

    Parameters:
        expand (str): Comma-separated list of elements to expand
    """

In the docstring (things the ZCA registry doesn't know): first-line summary, free-form description, Category: (grouping), Examples: (reference to tests/http-examples/{name}.req/.resp), Parameters:.

Not in the docstring (already in the ZCA registry): HTTP method, endpoint name, required permission, context interface (for_).

3. Graceful Degradation for Add-ons (and Undocumented Services)

The views use what's always available from the ZCA registry, so participation is incremental and nothing is required:

1. Nothing — endpoint shows up with name, method, permission, context, factory class. Already useful for discovery.
2. One-line docstring — adds a human-readable description.
3. Category: + description — proper grouping and full context.
4. Examples: + shipped .req/.resp files — full experience with inlined examples.

An endpoint with no docstring still yields its name, method, permission, and context — often enough to try a GET and inspect the response.

4. Utilizing Existing Auto-Generated Examples

plone.restapi's test_documentation.py already captures real request/response pairs via save_request_and_response_for_docs(), stored as .req/.resp files in tests/http-examples/ and shipped with the package. The per-endpoint detail view reads the relevant file(s) at runtime and inlines them for that endpoint only. The Examples: docstring field maps service class → example filename.

Add-ons can opt into the same experience by adopting the same documentation pattern; this is optional and will be documented for add-on authors.

5. Service Registry for Introspection

plone.rest registers services as ZCA adapters but doesn't maintain an enumerable list. Add a module-level registry (populated during ZCML processing) so the views — and the MCP server — can iterate all registered services without reaching into ZCA internals. This is the shared substrate referenced above.

6. llms.txt View

A browser view (not an @-prefixed JSON service) on the site root, returning text/markdown: site description, traversal explanation, and the categorized one-line-per-endpoint index with links to per-endpoint detail. Gated by the dedicated permission (see Risks).

7. Per-Endpoint Detail View (replaces the llms-full.txt mega-dump)

A view (e.g. /@llms/<endpoint-id>) returning text/markdown for a single endpoint: context, permission, description, parameters (from docstring + ZCA metadata), and the inlined request/response example where available. The llms.txt index links to these so an agent fetches only what it needs.

The traversal-model explanation (how Plone URLs work; which endpoints apply to which context types; "GET a path to see what's there") is served once, either inline at the top of llms.txt or as a dedicated /@llms/_traversal document linked from it.

Deliverables

• Google/napoleon-style docstring convention documented in the plone.restapi contributing guide.
• Documentation for add-on authors on opting into richer output.
• Registry records: plone.llms_site_description and plone.llms_enabled, with a control-panel field.
• Service registry in plone.rest for enumeration (coordinated with maintainers).
• llms.txt index view in plone.restapi.
• Per-endpoint detail view (/@llms/<id>) with inlined examples; optional concatenated llms-full.txt as a non-primary convenience.
• Dedicated permission protecting the views (see Risks).
• Docstrings backfilled on the ~20 most important existing services.
• Tests.

MVP

• Google/napoleon docstring convention defined and documented
• Registry records (plone.llms_site_description, plone.llms_enabled)
• Service registry in plone.rest for introspection
• Dedicated permission + rolemap entry
• llms.txt index view (site description, traversal explanation, categorized index with permissions/context, per-endpoint links)
• Per-endpoint detail view with inlined request/response examples
• Docstrings on the 20 most important services (content CRUD, search, workflow, users, types, auth)

Later

• Docstring coverage for all 119 existing services
• Prompt-recipe documents: common multi-step workflows ("create a page, add blocks, publish")
• Per-content-type schema documentation
• Volto block-type documentation with examples
• Auto-detect multilingual setup (plone.app.multilingual): include available languages and language-root structure
• ILLMSDocumentation adapter so add-ons can inject concept snippets
• Have the MCP server (Plone MCP (Model Context Protocol) Server #4213) generate its tool definitions from the shared introspection registry

Risks

Security / information disclosure — low but real. A complete, auto-generated map of every endpoint with its required permission is a reconnaissance aid for an attacker. Mitigations:

• The views are protected by a dedicated permission (e.g. plone.restapi: Access LLM API description), grantable to Anonymous by default but overrideable in a site's rolemap, per @davisagli's suggestion. Operators who don't want a public API map restrict the permission to authenticated roles.
• A registry flag (plone.llms_enabled) disables the views entirely (per @tisto).
• On-by-default is introduced only in a new minor release of Plone, never a patch.

Backwards compatibility — none affected. Two read-only views, a docstring convention, two registry records, one permission. No changes to existing behavior, no new runtime dependencies.

Context bloat — designed around. See Avoiding context bloat: progressive per-endpoint disclosure instead of a single mega-document; example URLs in fixtures are clearly fixture data and the traversal explanation tells the agent to treat them as illustrative.

Drift between MCP and llms.txt — mitigated by the shared substrate. Both consume deliverable #5; the MCP server is expected to generate its tool surface from it rather than maintain a parallel description.

Out of scope (own PLIPs):

• Scoped agent authentication. @davisagli's idea of OAuth-style scoped tokens for agents is now the settled industry pattern: the MCP spec (Nov 2025) mandates OAuth 2.1 + PKCE with scoped access tokens for remote servers. Whoever writes this PLIP should adopt that standard rather than design from scratch. It is independent of llms.txt discovery and deserves its own proposal.

Note on AI-Assisted Implementation

I plan to implement this PLIP with AI assistance (Claude Code), reviewing and committing all code myself in line with the Plone Foundation's emerging AI guidance. If a policy on AI-assisted contributions to core lands before implementation, I'll align with it.

Participants

• Jens W. Klein, @jensens — proposer, implementation lead
• Timo Stollenwerk, @tisto — seconder
• @danalvrz — MCP server PLIP (Plone MCP (Model Context Protocol) Server #4213); coordination on the shared introspection substrate
• plone.rest maintainers — review of the service-registry addition
• REST API / Framework teams — review and acceptance

[stevepiercy]

Where does Sphinx and its extensions fit into this? This PLIP omits mention of both.

How do you propose to autodoc docstrings?

[jensens]

You're thinking about sphinxcontrib-httpexample - probably - and it's actually related, but from the other side.

plone.restapi's test_documentation.py generates .req/.resp files. Sphinx extension consumes them at docs build time to render nice multi-format examples. The llms-full.txt view would consume the exact same files at request time to inline them as markdown for AI agents.

Same input, different output, different runtime. They're complementary.

The files are already shipped with the package (include_package_data=True + graft src/plone/restapi in MANIFEST.in), so the Zope/Plone view can read them via importlib.resources at runtime. No Sphinx involved, no build step needed.

Sphinx and autodoc aren't part of this - the views introspect service classes directly via the ZCA registry at runtime and read __doc__ from Python. No Markdown/MyST processing, no docs-build

[stevepiercy]

Thanks for addressing my previous questions. That makes sense.

When you bring up llms-full.txt, there's a Sphinx extension that already generates that in the main documentation, and Plone REST API gets included there as a git submodule. Are you talking about the same thing or something different?

[jensens]

The idea is not for documentation, but for the actual installed Plone site. But it makes sense to have that for the documentation as well, here its a different context and target group (Agents as Users of a specific Plone Site vs. Agents using Plone for writing for Core, Addons, using as Framework, writing customizations, - all our Documentation covers).

[jensens]

FYI: Updated main proposal addressing @stevepiercy valid questions (transparency: side channel chat included).

[tisto]

@jensens I am +1 on this, and I could imagine seconding this PLIP. I would like to do a bit more research on this topic before giving more feedback. I could imagine that some organisations would welcome this feature, while others might be reluctant to give an AI instructions on how to operate a public website. This question is not restricted to this PLIP but to our planned AI enhancements in general. We will position Plone as a system that enables people to use AI responsibly and openly, depending on their needs. For this PLIP, we would need an option to enable/disable this feature altogether and amend the files manually (the latter is already addressed, if I am not mistaken).

[davisagli]

Overall, this makes sense to me.

I agree with @tisto that there should be a way to opt out of publishing this (I'd suggest a custom permission protecting these views, which can be restricted in the rolemap.) And if we want to make it on by default, we should only do that in a new minor release of Plone.

I could also imagine that we might want to support OAuth for automated agents interacting with Plone. i.e. an agent can authenticate without needing the user's full credentials, but with limited token that authorizes it to operate on behalf of a specific user, possibly with a limited scope (think github personal access tokens)... as in it can only access some APIs, or only in a specific area of the site. In this case we could potentially publish llms.txt only for clients that are registered with the system.

[jensens]

I agree with both of you. And scoped tokens are a great idea, but way beyond the scope of this PLIP - probably worth an own PLIP.

[davisagli]

@jensens I agree, that would be best as a separate PLIP.

[tisto]

@jensens @davisagli who is going to write this additional PLIP? :)

Regarding the usage of Claude Code and AI in general. I use Claude for the plone-contributor-statistics and other things on a regular basis. The Plone Foundation has an AI guideline. My workflow is that I use Claude to generate code. I never check in code directly via Claude, but I keep a second terminal open where I manually commit and push (this avoids having Claude listed as a contributor on the commits). Claude is a tool and an assistant; I am still personally responsible for reviewing and committing the code. This is in sync with the Plone Foundation AI guideline.