DEEP DIVE SERIES · #07

Building Production MCP Servers with Streamable HTTP: Architecture, Scaling and Legacy SSE Migration

Luca Viscomi

· Published Sep 15, 2026 · Updated Sep 24, 2026 · 11 min read

The Model Context Protocol (MCP), open-sourced by Anthropic and adopted across the AI industry, has become the open standard for connecting AI agents to external tools, databases, enterprise SaaS, and execution runtimes. Most engineers begin their MCP journey using stdio transport—spawning a local subprocess like npx -y @modelcontextprotocol/server-sqlite and piping standard input and standard output directly into Claude Desktop or Cursor.

Local stdio is ideal for single-developer desktop workflows. However, when deploying enterprise multi-agent architectures—where autonomous subagents on cloud worker nodes must concurrently query centralized databases, authenticate through OAuth2 gateways, and stream long-running execution telemetry—stdio cannot cross machine boundaries.

Early remote MCP implementations relied on an asymmetrical HTTP with Server-Sent Events (HTTP+SSE) pattern. Under the modern Model Context Protocol transports specification, Streamable HTTP is the official standard remote transport, addressing the stateful bottlenecks of legacy SSE while establishing a streamlined foundation for cloud-native agent gateways.

Here is the complete engineering blueprint for building production-grade remote MCP servers using Streamable HTTP, managing enterprise infrastructure, and maintaining backward compatibility during legacy SSE migration.

1. The Remote MCP Transport Landscape

To select the right transport architecture, engineering teams must evaluate three distinct execution models:

2. Why Streamable HTTP Replaced Legacy HTTP+SSE

In the initial MCP transport drafts, remote connectivity was built around a split HTTP design:

1. The client opened a long-running downstream stream via GET /sse.
2. The server responded with an initial event advertising an upstream endpoint: /messages?session_id=<UUID>.
3. The client submitted JSON-RPC tool requests as separate HTTP POST calls to that upstream URL.
4. The server matched the session_id, pushed results back down the open SSE connection, and acknowledged the POST with an HTTP 202.

While functional, this split architecture introduced severe operational liabilities in production:

The Dual-Endpoint Synchronization Penalty

When multiple subagents operate across distributed Kubernetes pods or serverless instances, maintaining in-memory session mappings (SESSION_QUEUES[session_id]) breaks horizontal scaling. If an upstream POST /messages lands on Pod B while the downstream GET /sse is terminated on Pod A, the server returns an HTTP 404 unless backed by a distributed Pub/Sub backplane (e.g., Redis Streams or NATS).

Deadlock and Orphaned Connections

Mobile devices, intermittent Wi-Fi, and corporate proxy idle-timeouts routinely sever downstream SSE connections without emitting an explicit TCP FIN packet. The server’s in-memory session registry leaks memory, while the client’s subsequent upstream POST requests fail silently.

The Streamable HTTP Solution

Streamable HTTP unifies remote communication into standard HTTP request/response semantics enhanced with streaming capabilities:

• Unified Endpoint: All interactions target a single base endpoint (typically /mcp).
• Direct Request-Response Streaming: For standard unary calls (e.g., tools/list), the server returns an immediate application/json payload without requiring persistent event streams.
• Chunked Tool Streaming: When executing long-running tools, database queries, or progress notifications, the server responds with Transfer-Encoding: chunked and Content-Type: text/event-stream, delivering incremental JSON-RPC frames over the direct HTTP response body.
• Stateless Infrastructure: Edge proxies, Cloudflare workers, and standard ingress controllers can route requests without requiring sticky sessions or distributed session synchronization buses.

3. Production Architecture Blueprint

An enterprise-ready MCP gateway must deliver high-throughput Streamable HTTP execution while exposing a backward-compatible adapter for legacy clients:

[ LLM Client / Agent Cluster ]
           │
           │  Streamable HTTP: POST /mcp (JSON-RPC + Chunked Response)
           │  Legacy Fallback: GET /sse + POST /messages?session_id=...
           ▼
[ Cloudflare / Reverse Proxy ]
    ├── TLS Termination & DDoS Mitigation
    ├── Proxy Buffer Bypassing (`X-Accel-Buffering: no`)
    └── OAuth2 / API Token Validation
           │
           ▼
[ FastAPI / ASGI Gateway ]
    ├── Authentication & Rate Limiting Middleware
    ├── Transport Demuxer (Streamable HTTP vs Legacy SSE Adapter)
    ├── JSON-RPC 2.0 Dispatcher (`tools/list`, `tools/call`, `prompts/get`)
    └── Sandboxed Tool Execution Engine
           │
           ▼
[ Downstream Infrastructure ] (PostgreSQL, Vector Indices, Internal Microservices)

4. Production Python Implementation (Streamable HTTP with SSE Adapter)

Below is a production-hardened MCP server implemented with Python, FastAPI, and asyncio. It handles native Streamable HTTP requests while providing a fallback adapter for legacy SSE clients:

import asyncio
import json
import logging
import uuid
from typing import Dict, AsyncGenerator
from fastapi import FastAPI, Request, HTTPException, Depends, status
from fastapi.responses import StreamingResponse, JSONResponse
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel, Field

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("mcp-production-gateway")

app = FastAPI(
    title="Production MCP Gateway (Streamable HTTP & SSE)",
    version="2.0.0",
    docs_url="/docs"
)

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# Legacy SSE Session Storage (for backward compatibility)
LEGACY_SESSION_QUEUES: Dict[str, asyncio.Queue] = {}

class JSONRPCRequest(BaseModel):
    jsonrpc: str = "2.0"
    id: str | int | None = None
    method: str
    params: dict = Field(default_factory=dict)

# --- Authentication Middleware ---
async def verify_auth_token(request: Request) -> str:
    auth_header = request.headers.get("Authorization")
    if not auth_header or not auth_header.startswith("Bearer "):
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Missing or malformed Authorization header"
        )
    token = auth_header.split(" ", 1)[1]
    # Replace with JWT verification, OAuth2 introspection, or Vault lookup in production
    if token != "production-secret-token-2026":
        raise HTTPException(
            status_code=status.HTTP_403_FORBIDDEN,
            detail="Invalid or expired access token"
        )
    return token

# --- Deterministic Tool Dispatcher ---
async def dispatch_tool_call(name: str, args: dict) -> dict:
    """Executes validated tool logic with deterministic guardrails."""
    if name == "query_analytics":
        table = args.get("table", "telemetry")
        limit = min(int(args.get("limit", 10)), 100) # Enforce bound
        # Simulated database retrieval
        return {
            "table": table,
            "count": limit,
            "records": [{"id": i, "status": "active", "latency_ms": 12.4} for i in range(limit)]
        }
    raise ValueError(f"Unknown tool requested: {name}")

# ==============================================================================
# 1. MODERN STREAMABLE HTTP TRANSPORT (PRIMARY)
# ==============================================================================
@app.post("/mcp")
async def handle_streamable_mcp(
    request: Request,
    payload: JSONRPCRequest,
    token: str = Depends(verify_auth_token)
):
    """
    Primary Streamable HTTP endpoint.
    Handles JSON-RPC 2.0 requests with either direct JSON or chunked streaming responses.
    """
    logger.info(f"Streamable HTTP invocation: method={payload.method}, id={payload.id}")

    # Standard Discovery: tools/list
    if payload.method == "tools/list":
        return JSONResponse({
            "jsonrpc": "2.0",
            "id": payload.id,
            "result": {
                "tools": [
                    {
                        "name": "query_analytics",
                        "description": "Execute bounded analytics queries against production telemetry data.",
                        "inputSchema": {
                            "type": "object",
                            "properties": {
                                "table": {"type": "string", "enum": ["telemetry", "costs", "tokens"]},
                                "limit": {"type": "integer", "maximum": 100, "default": 10}
                            },
                            "required": ["table"]
                        }
                    }
                ]
            }
        })

    # Standard Execution: tools/call
    elif payload.method == "tools/call":
        tool_name = payload.params.get("name")
        tool_args = payload.params.get("arguments", {})

        # Check if client explicitly requests streaming response
        accept_header = request.headers.get("Accept", "")
        supports_streaming = "text/event-stream" in accept_header

        if supports_streaming:
            # Return progressive chunked stream
            async def response_stream() -> AsyncGenerator[str, None]:
                try:
                    # Initial progress notification
                    progress_notice = {
                        "jsonrpc": "2.0",
                        "method": "notifications/progress",
                        "params": {"progressToken": payload.id, "progress": 0.5, "total": 1.0}
                    }
                    yield f"event: message\ndata: {json.dumps(progress_notice)}\n\n"
                    await asyncio.sleep(0.05) # Yield to event loop

                    # Final result execution
                    result_data = await dispatch_tool_call(tool_name, tool_args)
                    final_response = {
                        "jsonrpc": "2.0",
                        "id": payload.id,
                        "result": {
                            "content": [{"type": "text", "text": json.dumps(result_data)}]
                        }
                    }
                    yield f"event: message\ndata: {json.dumps(final_response)}\n\n"
                except Exception as e:
                    error_response = {
                        "jsonrpc": "2.0",
                        "id": payload.id,
                        "error": {"code": -32603, "message": str(e)}
                    }
                    yield f"event: message\ndata: {json.dumps(error_response)}\n\n"

            return StreamingResponse(
                response_stream(),
                media_type="text/event-stream",
                headers={
                    "Cache-Control": "no-cache",
                    "Connection": "keep-alive",
                    "X-Accel-Buffering": "no"
                }
            )
        else:
            # Immediate synchronous JSON-RPC response
            try:
                result_data = await dispatch_tool_call(tool_name, tool_args)
                return JSONResponse({
                    "jsonrpc": "2.0",
                    "id": payload.id,
                    "result": {
                        "content": [{"type": "text", "text": json.dumps(result_data)}]
                    }
                })
            except Exception as e:
                return JSONResponse({
                    "jsonrpc": "2.0",
                    "id": payload.id,
                    "error": {"code": -32603, "message": str(e)}
                }, status_code=400)

    raise HTTPException(status_code=400, detail=f"Unsupported method: {payload.method}")

# ==============================================================================
# 2. LEGACY HTTP+SSE BACKWARD-COMPATIBILITY ADAPTER
# ==============================================================================
@app.get("/sse")
async def legacy_sse_connect(request: Request, token: str = Depends(verify_auth_token)):
    """Legacy endpoint: establishes downstream event stream for older clients."""
    session_id = str(uuid.uuid4())
    queue = asyncio.Queue()
    LEGACY_SESSION_QUEUES[session_id] = queue
    logger.info(f"Legacy client connected via SSE. Assigned session_id={session_id}")

    async def legacy_event_generator():
        try:
            # Announce upstream message destination
            endpoint_event = {"event": "endpoint", "data": f"/messages?session_id={session_id}"}
            yield f"event: {endpoint_event['event']}\ndata: {endpoint_event['data']}\n\n"

            while True:
                if await request.is_disconnected():
                    break
                try:
                    msg = await asyncio.wait_for(queue.get(), timeout=15.0)
                    yield f"event: message\ndata: {json.dumps(msg)}\n\n"
                    queue.task_done()
                except asyncio.TimeoutError:
                    yield ": ping\n\n"
        finally:
            LEGACY_SESSION_QUEUES.pop(session_id, None)
            logger.info(f"Cleaned up legacy SSE session: {session_id}")

    return StreamingResponse(
        legacy_event_generator(),
        media_type="text/event-stream",
        headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"}
    )

@app.post("/messages")
async def legacy_messages_receive(
    session_id: str,
    payload: JSONRPCRequest,
    token: str = Depends(verify_auth_token)
):
    """Legacy upstream handler: translates POST to the mapped SSE queue."""
    if session_id not in LEGACY_SESSION_QUEUES:
        raise HTTPException(status_code=404, detail="Legacy session expired or not found")

    target_queue = LEGACY_SESSION_QUEUES[session_id]
    
    if payload.method == "tools/list":
        await target_queue.put({
            "jsonrpc": "2.0",
            "id": payload.id,
            "result": {"tools": [{"name": "query_analytics", "description": "Legacy adapter analytics"}]}
        })
        return {"status": "accepted"}
    elif payload.method == "tools/call":
        result = await dispatch_tool_call(payload.params.get("name", ""), payload.params.get("arguments", {}))
        await target_queue.put({
            "jsonrpc": "2.0",
            "id": payload.id,
            "result": {"content": [{"type": "text", "text": json.dumps(result)}]}
        })
        return {"status": "accepted"}

    raise HTTPException(status_code=400, detail="Unsupported legacy method")

5. Reverse Proxies, Cloudflare, and Load-Balancing Realities

Deploying remote MCP servers into production environments (such as Render, AWS ECS, Fly.io, or GCP Cloud Run) introduces network edge complexities that do not exist during local development:

1. Reverse Proxy Response Buffering (X-Accel-Buffering)

By default, reverse proxies (Nginx, Traefik, Cloudflare) buffer incoming responses until their internal buffer threshold (typically 4 KB or 8 KB) is met before flushing bytes downstream to the client.

• The Symptom: When an agent initiates a tool execution, the request hangs until a timeout occurs, even though the backend finished execution in 20 milliseconds.
• The Fix: The server must explicitly emit the response header X-Accel-Buffering: no on all streaming responses. Furthermore, proxy-level Gzip and Brotli compression must be disabled for the text/event-stream MIME type.

2. HTTP/2 and Connection Multiplexing

Streamable HTTP performs significantly better when deployed over HTTP/2 or HTTP/3. Under HTTP/1.1, concurrent tool calls require multiple underlying TCP connections, hitting browser or client pool connection limits (often capped at 6 concurrent connections per host). With HTTP/2, multiple agent threads multiplex requests and streaming responses across a single persistent TCP/TLS handshake.

3. Graceful Timeouts and Dead-Man Reaping

Cloud load balancers (such as AWS ALB or Cloudflare CDN) enforce default connection idle timeouts (typically 60 to 100 seconds). For long-running agent tasks (e.g., executing a complex database migration or running a test suite):

• Emit periodic keep-alive comment frames (: ping\n\n or progress notifications) every 15 seconds.
• Configure client HTTP agents with aggressive read timeouts and exponential backoff retry policies.

6. Migration Playbook: Moving from HTTP+SSE to Streamable HTTP

For teams operating active legacy SSE gateways, migrating to Streamable HTTP should follow a structured four-phase transition:

Phase 1: Dual-Transport Gateway Deployment

Deploy the updated gateway exposing both the unified /mcp Streamable HTTP route and the legacy /sse + /messages endpoints simultaneously. Confirm that internal observability tracks connection volume by transport type.

Phase 2: Client Capability Negotiation

Update client configurations (such as Claude Desktop claude_desktop_config.json or custom LangGraph MCP client harnesses). Modern clients will automatically attempt Streamable HTTP when supplied with a direct URL endpoint:

{
  "mcpServers": {
    "enterprise-gateway": {
      "url": "https://mcp.agentjunky.com/mcp",
      "headers": {
        "Authorization": "Bearer production-secret-token-2026"
      }
    }
  }
}

Phase 3: Session State Decommissioning

As legacy clients migrate, eliminate in-memory session registries (LEGACY_SESSION_QUEUES). Transitioning to Streamable HTTP allows backend instances to operate completely statelessly, removing Redis Pub/Sub sync layers and simplifying horizontal autoscaling.

Phase 4: Deprecation and Decommissioning

Monitor access logs for requests hitting /sse and /messages. Once traffic drops to zero, remove the legacy endpoints and enforce strict HTTP/2 Streamable HTTP connections across the entire agent fleet.

7. Enterprise Security Fencing for Remote Gateways

When an agent operates over a remote network boundary, securing tool execution is critical:

1. Read-Only / Mutation Partitioning: Segregate dangerous mutating actions (execute_ddl, delete_records, provision_infra) into explicit schemas requiring human-in-the-loop authorization tokens.
2. Schema Sanitization: Strip internal server IPs, sensitive environment variables, and raw database connection strings before returning JSON-RPC error responses.
3. Payload Truncation Boundaries: Enforce strict response size limits (e.g., 64 KB). When tool outputs exceed this threshold, offload the full payload to an S3/GCS bucket and return a presigned URI pointer to prevent context saturation.

By adopting Streamable HTTP, your multi-agent architecture transitions from brittle desktop scripts into a durable, scalable, and secure enterprise AI systems engineering platform.