How an MCP Client, an LLM, and an MCP Server Actually Talk to Each Other

MCP tool calling looks a lot like OpenAI-style function calling, with one twist: the tool lives behind its own JSON-RPC protocol, and as of the current spec, that protocol is stateless — no session, no handshake, every request self-contained.

Example: "What's the weather in Tokyo?"

Below is what the flow looks like under the latest MCP spec version 2026-07-28. The LLM doesn't know current conditions, but knows a get_weather tool exists via MCP.

sequenceDiagram


    actor User

    box rgb(232,244,248) Client Environment
        participant Client as MCP Client
(Agent)
    end
    box rgb(240,240,240) LLM Provider Environment
        participant LLM as LLM API
    end
    box rgb(255,243,224) MCP Server Environment
        participant MCPServer as MCP Server
(weather tool)
    end


    autonumber 1.1 0.1

    Note over Client,MCPServer: Phase 1: Tool discovery (tools/list) — no handshake required

    Client->>+MCPServer: tools/list (JSON-RPC request)
{ "jsonrpc": "2.0", "id": 1,
"method": "tools/list",
"params": { "_meta": {
"io.modelcontextprotocol/protocolVersion": "2026-07-28",
"io.modelcontextprotocol/clientInfo": {"name": "my-agent", "version": "1.0"} } } }
// HTTP headers: MCP-Protocol-Version, Mcp-Method: tools/list
    MCPServer-->>-Client: tools/list result
{ "jsonrpc": "2.0", "id": 1,
"result": { "resultType": "complete",
"tools": [{ "name": "get_weather",
"description": "Get current weather for a location",
"inputSchema": { "type": "object",
"properties": { "location": {"type": "string"} },
"required": ["location"] } }],
"ttlMs": 300000, "cacheScope": "public" } }

    Client->>Client: Cache tool list for ttlMs,
convert inputSchema →
LLM function-calling format

    autonumber 2.01 0.01
    Note over User,MCPServer: Phase 2: Runtime — user request

    User->>+Client: "What's the weather in Tokyo?"

    Client->>+LLM: POST /v1/chat/completions
{ "model": "gpt-4o",
"messages": [{ "role": "user",
"content": "What's the weather in Tokyo?" }],
"tools": [{ "type": "function",
"function": { "name": "get_weather",
"description": "Get current weather for a location",
"parameters": { "type": "object",
"properties": { "location": {"type": "string"} },
"required": ["location"] } } }] }

    LLM->>LLM: Decide tool use is needed
(extract location="Tokyo")

    LLM-->>-Client: Response (finish_reason: tool_calls)
{ "choices": [{ "message": { "role": "assistant",
"content": null,
"tool_calls": [{ "id": "call_abc123",
"type": "function",
"function": { "name": "get_weather",
"arguments": "{\"location\":\"Tokyo\"}" } }] },
"finish_reason": "tool_calls" }] }

    Client->>Client: Parse tool_calls,
map function name →
MCP tool + server

    Client->>+MCPServer: tools/call (JSON-RPC request)
{ "jsonrpc": "2.0", "id": 2,
"method": "tools/call",
"params": { "name": "get_weather",
"arguments": { "location": "Tokyo" },
"_meta": {
"io.modelcontextprotocol/protocolVersion": "2026-07-28",
"io.modelcontextprotocol/clientInfo": {"name": "my-agent", "version": "1.0"} } } }
// HTTP headers: Mcp-Method: tools/call, Mcp-Name: get_weather

    MCPServer->>MCPServer: Call external weather API
(e.g. OpenWeatherMap)

    MCPServer-->>-Client: tools/call result
{ "jsonrpc": "2.0", "id": 2,
"result": { "resultType": "complete",
"content": [{ "type": "text",
"text": "{\"temp\": 22, \"condition\": \"clear\", \"humidity\": 60}" }],
"isError": false } }

    Client->>Client: Extract result content,
convert MCP result →
tool-role message

    Client->>+LLM: POST /v1/chat/completions (continued)
{ "messages": [
{"role": "user", "content": "What's the weather in Tokyo?"},
{"role": "assistant", "tool_calls": [...]},
{ "role": "tool",
"tool_call_id": "call_abc123",
"content": "{\"temp\": 22, \"condition\": \"clear\"}" } ] }

    LLM->>LLM: Integrate tool result,
generate natural language answer

    LLM-->>-Client: Response (finish_reason: stop)
{ "choices": [{ "message": { "role": "assistant",
"content": "The weather in Tokyo is clear with a temperature of 22°C." },
"finish_reason": "stop" }] }

    Client->>Client: response.choices[0].message.content
    Client-->>-User: "The weather in Tokyo is clear
with a temperature of 22°C."

Important parts Step-by-step

Phase 1: Tool discovery

• 1.1–1.2 — Discover the tool. In previous MCP version an initialize and initializedhandshake was required but in latest it is not. The client only needs to send tools/list. Protocol version and client info ride along in a _meta field on the request itself (and an MCP-Protocol-Version HTTP header). The server replies with the tool's name, description, and input schema, plus a ttlMs/cacheScope telling the client how long it can cache this list.
• 1.3 — Translate the schema. The client converts MCP's inputSchema into the LLM provider's function-calling format (e.g., OpenAI's tools: [...]). This is what makes one MCP server work with any LLM vendor.

Phase 2: Runtime - User Request

• 2.01-2.02 — Ask the LLM. The user's question and the translated tool definitions go to the LLM.
• 2.03–2.04 — LLM requests a tool call. The LLM can't answer from its own knowledge, so it returns a tool call (get_weather, location: Tokyo) instead of a final answer — finish_reason: tool_calls.
• 2.05–2.08 — Client calls the MCP server. The client maps the function name to the right MCP tool, sends a self-contained tools/call request (arguments + _meta, with Mcp-Method/Mcp-Name headers for routing), the server hits the external weather API, and returns the result (resultType: complete) in a content array. Because nothing here depends on session state, any server instance behind a plain load balancer can handle this.
• 2.09–2.10 — Result goes back to the LLM. The client reshapes the MCP result into a role: "tool" message and resends the conversation.
• 2.11–2.12 — LLM writes the final answer. It integrates the result and returns finish_reason: stop.
• 2.13–2.14 — Back to the user. The client extracts the text and delivers it.

A more real implementation would also deal with:

• tools/list pagination,
• distinguish protocol errors from tool execution errors (isError: true),
• support resultType: "input_required" for mid-call prompts, and should confirm sensitive tool calls with the user first.

I hope this servers as a start point for understanding how MCP works internally.