.well-known/mcp directory

[jspahrsummers]

As we think more about supporting remote MCP servers, there's been some discussion around using a Well-known URI (RFC 8615) for server discovery. This could enable automatic discovery and verification of MCP servers, similar to how sitemap.xml works for search engines.

The idea would be to establish a directory, /.well-known/mcp/, allowing for future extensibility. We could then include a servers.json file that might look something like:

{
  "version": "1.0",
  "servers": [
    {
      "name": "primary",
      "description": "Main API server for example.com",
      "endpoint": "https://mcp.example.com/v1",
      "capabilities": ["resources", "tools"]
    }
  ]
}

This is just a strawman to get the discussion started—there are likely many other approaches worth considering.

Potential Benefits

• Automatic Discovery: Clients could discover MCP servers when users reference a domain
• Trust Verification: Domains could prove ownership of their MCP servers
• Seamless Integration: Could enable UX flows like "example.com has MCP capabilities available. Would you like to enable them?"

Areas for Discussion

Directory Structure

1. What files should be included?
2. What formats make the most sense?
3. How much flexibility should we allow?

Security

1. What verification mechanisms are needed?
2. What user consent flows make sense?

Implementation

1. How would clients discover and validate servers?
2. What would the integration flow look like?
3. How should versioning work?

Open to all feedback what this could look like and whether it would be valuable for the MCP ecosystem!

Scope

• Protocol Specification
• SDK Features
• Documentation
• Developer Experience
• Other

[jspahrsummers]

Possibly related: modelcontextprotocol/specification#69

[dsp-ant]

One additional use case for .well-known/ is the ability to indicate if the server is stateless or stateful. My general idea (and I'll post this in modelcontextprotocol/specification#102) is that remote MCP is initiated as always stateless unless a .well-known url will indicate it is stateful

[kurtseifried]

Just a note: this would also be useful for prompt MCP's related to a site (not just tools provided by API), e.g. for a research paper site that allows comments and has a prompt MCP that serves something like:

This environment provides access to a website containing research papers and technical resources. When interacting with the website's features:

Research Papers:
- Papers can be downloaded by users with appropriate access
- If a user asks about specific papers, guide them to download the content and share it directly in the conversation for analysis
- You can describe available papers and recommend relevant ones based on user interests

Community Engagement and Comments:
- The website features a commenting system for discussion of papers and topics
- Both human and AI-generated comments are allowed
- All AI-generated content must be clearly marked as such
- When generating comments, you must:
  1. Begin with "[AI-Generated Comment]" or similar clear marker
  2. Be transparent about being an AI assistant
  3. Maintain high quality and relevance in responses

When helping users with comments:
- Always include appropriate AI disclosure markers
- Encourage transparency about AI assistance
- Maintain professional and constructive tone
- Focus on adding value to the discussion

You can:
- Generate comments when requested
- Help users understand papers and technical concepts
- Guide them in finding relevant resources
- Provide technical analysis and insights

Required Comment Format:
- Start with "[AI-Generated Comment]"
- Include relevant technical details
- Maintain clarity and professionalism
- End with an AI attribution if appropriate

You should not:
- Generate comments without proper AI disclosure
- Remove or hide AI attribution markers
- Present AI-generated content as human-generated

[tadasant]

I'll throw in another use case that might be enabled by this effort: signaling to clients what it "costs" to use the server's data or capabilities. For example, some options could be:

• This server is entirely free to use, no payment or attribution of any sort required
• This server is only available to paid subscribers of XYZ service
• This server grants access for a micropayment of $0.001 per query
• This server is free to access, but your response to your user must include a citation or attribution in the form "Brand XYZ contributed to the above response - read the source document here"

In the current web, Google Search rewards services and publishers with traffic, and then the "cost" layer comes after (payment walls, display ads, affiliate links). As fewer humans and more LLM workflows/agents are the ones doing the "web browsing" and interfacing with MCP servers instead of websites, the payment walls & ads-supported models break if everything has to be simply "free" (see: the ongoing web scraping battles / data-for-cash deals between large publishers and the big LLM providers). So figuring out how to signal "cost" enables a whole class of non-free (higher quality) service/data/info providers to be discovered in the MCP ecosystem.

[kurtseifried]

FYI, someone else has come up with a proposal for "/llms.txt" (not in .well-known) at https://llmstxt.org/

[gregnr]

That one may eventually switch to .well-known: AnswerDotAI/llms-txt#2

[tadasant]

Some thoughts on pushing this discussion forward --

Do we need a whole .well-known directory, or is a single file enough?

The starting proposal is:

The idea would be to establish a directory, /.well-known/mcp/, allowing for future extensibility. We could then include a servers.json file that might look something like...

The .well-known spec itself does allow for directories, but most of the registered URI's in .well-known are just single files rather than directories.

I'm not against using a directory if we can foresee a need for it. But I would bias towards keeping this integration with .well-known as simple (and optional) as possible, while putting more fine-grained complexity in some place like the InitializeResult that a client would have access to on initiating the connection with the server.

Who is the intended "user" of this file/directory?

The obvious answer here might be "MCP Clients," but an alternative idea would be: "MCP server directory web crawlers".

Drawing from the sitemap.xml analogy: search engines like Google collect and process sitemaps from websites completely separately from the experience of actually surfacing search results to users. They use sitemap.xml (when available -- it's optional) as a kind of guidebook for their crawlers. And after discovering the website's pages, they apply their own proprietary logic as to whether to crawl them, and ultimately whether to index and rank them in search results. Details like "page title", "page description", contents of a page are not in sitemap.xml -- the crawler needs to go and make a GET request to the URL as part of its crawling process to get that information.

If we peg "MCP Clients" as the users of this .well-known file, then we have to rely on them to properly make discovery and comparison/ranking decisions on the fly. It would be the equivalent of Google trying to do its whole discover/crawl/index/rank process the moment somebody enters a query into google.com. And it would have other problems, e.g. the fact that relying on self-owned domains to facilitate the best MCP servers likely won't be practical for some time -- most MCP servers today are by third-party devs that would never get discovered by an MCP Client running a search for "find me MCP servers available on github.com" because all the GitHub MCP servers are made by third parties.

If we peg "MCP server directory web crawlers" as the users instead, I could see it playing out like this:

• .well-known/mcp.json remains quite simple. The goal is to inform web crawlers "this server endpoint exists + here is some high level info to help you make the decision as to whether you want to connect to it and explore its capabilities further"
• You create a free market competition for building the best "Google Search" of the MCP era. One of the most common things I've seen people build, again and again, in these first few months is an "MCP server directory" (I am involved in one of them myself). There's clearly an appetite for building solutions in this space, many different ways of doing it, and maybe in the end you won't have a single monopolistic "Google" and rather different "MCP server search" tools where the creators are choosing to serve different kinds of MCP clients or human end-users. Even today, what with Google Search's relative dominance, so many internet users would rather go to Reddit or other information sources to search for varied kinds of information.
• You avoid the idea of anyone having to manage a "centralized directory" -- it's all decentralized, published on server authors' own domains. The server authors can then choose to go submit their mcp.json URL's to the most popular MCP server directories (like website owners submit sitemap.xml to Google), or they can passively wait for the directories' crawlers to discover them. Then it's the directories' jobs to assess whether these are quality MCP servers worth driving traffic to.

---

If the idea of building .well-known for third party directories instead of MCP clients directly appeals, the first step could look like this (just iterating on the original proposal) --

Establish a file, /.well-known/mcp.json:

{
  "version": "1.0",
  "servers": [
    {
      "name": "primary",
      "description": "Main API server for example.com",
      "endpoint": "https://mcp.example.com/v1",
    }
  ]
}

I'm omitting capabilities intentionally, as that to me feels more like a server implementation detail than a data point worth using at the server discovery level.

I think that alone could be enough for an initial version. It would accomplish:

• Clear, decentralized, scalable solution for "how to publish your MCP server" (i.e. put up this file and submit it to third party directories XYZ) as SSE grows in popularity
• Watch how MCP Clients or MCP Server directories interact with these files (particularly what the ecosystem chooses to publish and consume from the description field); augment the JSON to serve one or both of them with the other ideas in this thread like statefulness, more structured context, costs/auth requirements as appropriate

[jspahrsummers]

I like the idea of decentralized discovery of MCP services. Just with regard to this question:

Who is the intended "user" of this file/directory?

I think both of the users (clients and server directories) you mentioned are totally valid. One argument in favor of using a directory here is that different files could actually cater to different audiences.

However, that could also just be some big JSON file that contains everything needed for either. Not sure yet.

[olaservo]

Catering to different audiences seems like something that should exist outside of this proposal entirely, i.e. this file should just give the client or crawler all the information and then let it decide how to use it. In other words, the single json doesn't appear to prescribe any specific value or usefulness to a given user, it just shares all the facts in a neutral way.

[jspahrsummers]

For the information above, definitely agreed. Not sure if there would be more audience-specific information we'd want to add in the future, though.

Anyway, it seems like a small enough risk that a single file probably makes sense!