Skip to content
ProxAI

Architecture

Architecture

Section titled “Architecture”

ProxAI is a small local compatibility proxy. It accepts local OpenAI-compatible or Anthropic-style requests, normalizes protocol-specific request shapes, forwards them to a configured upstream provider, and translates upstream responses back to the client-facing protocol when needed.

Section titled “Related docs”
Request lifecycleSource-level path from inbound request through provider transport to outbound response.Module boundariesWhere parsing, routing, translation, provider behavior, HTTP carriers, errors, and observation belong.Config flowHow config.toml, examples, CLI overrides, validation, providers, and runtime state fit together.Error flowHow internal errors become compact client-facing HTTP and SSE errors.ConfigurationRuntime settings, routes, providers, capture, logging, and errors.Protocol overviewPhase axis, protocol axis, and supported runtime paths.Protocol conversionPair-oriented conversion rules and protocol alignment expectations.

Two-axis model

Section titled “Two-axis model”

The codebase is easiest to understand as two independent axes.

Phase axis

Section titled “Phase axis”

The phase axis describes where data is in the proxy pipeline:

  • inbound_request — the original client request received by ProxAI
  • provider_request — the request ProxAI prepares for the upstream provider
  • upstream_response — the response returned by the upstream provider
  • outbound_response — the response ProxAI returns to the client

Protocol axis

Section titled “Protocol axis”

The protocol axis describes the wire protocol used at a given phase:

  • openai_responses
  • openai_chat_completions
  • anthropic_messages

Each phase has its own protocol:

  • inbound_request.protocol is what the client sent.
  • provider_request.protocol is what ProxAI sends upstream, controlled by the selected provider.
  • upstream_response.protocol is what the provider returns.
  • outbound_response.protocol is what ProxAI returns to the client.

Provider names are user labels. They are not semantic protocol identifiers.

Top-level source layout

Section titled “Top-level source layout”
  • Directorysrc/
    • main.rs — entry point; delegates to cli::main
    • lib.rs — AppState, axum Router, proxy handler
    • Directorycli/ — CLI parsing and startup
      • …
    • config.rs — config.toml schema and loading
    • paths.rs — app directory resolution
    • request.rs — shared request carrier types
    • sse.rs — SSE helpers
    • formatting.rs — formatting helpers
    • Directoryerror/ — domain errors and rendering
      • …
    • Directoryhttp_support/ — HTTP carrier helpers
      • …
    • Directoryingress/ — inbound protocol parsing and normalization
      • …
    • Directoryprotocol/ — protocol wire types and protocol enums
      • …
    • Directoryrouting/ — route matching
      • …
    • Directoryprovider/ — provider request preparation and HTTP transport
      • …
    • Directoryupstream/ — upstream response reading
      • …
    • Directorytranslation/ — cross-protocol conversion
      • …
    • Directorypipeline/ — typed proxy pipeline
      • …
    • Directoryobserve/ — capture, logging, diagnostics
      • …
    • Directorymcp/ — MCP control surface
      • …

Request lifecycle

Section titled “Request lifecycle”

src/lib.rs registers these routes and sends all of them to the same proxy handler:

/v1/responses          /responses
/v1/chat/completions   /chat/completions
/v1/messages           /messages

The simplified inbound path is:

let prepared_provider = inbound_http
    .prepare_inbound()?                  // ingress: parse + normalize
    .route_to_provider(...)?             // routing: choose provider
    .prepare_provider_request()?;        // provider/request + translation


run_provider_flow(prepared_provider).await

run_provider_flow then handles the provider side:

let provider_http = prepared_provider
    .send_to_upstream().await?           // provider/transport + upstream
    .handle_upstream_response().await?;  // upstream: read body / stream


provider_http.translate_to_outbound().await?  // translation + http_support

Pipeline stages

Section titled “Pipeline stages”
  1. 1
    inbound_request
    Modules
    pipeline/inbound.rsingress/
    Responsibility

    Read body, detect protocol, normalize request shape, create InboundHttpFlow.

  2. 2
    Routing
    Modules
    pipeline/inbound.rsrouting/
    Responsibility

    Match by protocol and model, resolve default provider.

  3. 3
    provider_request
    Modules
    pipeline/provider_request.rsprovider/requesttranslation/request
    Responsibility

    Translate request payload, rewrite provider model, serialize body.

  4. 4
    Send upstream
    Modules
    pipeline/provider_request.rsprovider/transport
    Responsibility

    Build auth headers, construct upstream URL, send via reqwest.

  5. 5
    upstream_response
    Modules
    pipeline/upstream_response.rsupstream/
    Responsibility

    Read status, headers, and body or stream.

  6. 6
    outbound_response
    Modules
    pipeline/provider_response.rstranslation/responsetranslation/streaming
    Responsibility

    Translate response back to inbound protocol and rebuild HTTP response.

pipeline/ uses a typed ProxyFlow<S> state machine. Each phase consumes one flow state and returns the next one, keeping the phase order explicit.

Module responsibility map

Section titled “Module responsibility map”
protocol/

Protocol wire shapes and shared protocol enums. Models JSON only; no conversion and no HTTP carrier types.

ingress/

Inbound protocol detection, parsing, and normalization before routing and translation.

routing/

Provider selection from request protocol, model pattern, defaults, and route configuration.

translation/

Pure request, response, and streaming conversion across explicit protocol pairs.

provider/

Provider request rendering, model rewrite, auth headers, upstream URL construction, and transport.

upstream/

Reading upstream status, headers, complete bodies, or streaming byte carriers.

http_support/

Protocol-neutral HTTP helpers such as content-type checks, response reconstruction, and boxed streams.

observe/

Capture artifacts, structured logging, request hints, and privacy-preserving diagnostics.

error/

Domain-specific error types and client-facing response rendering.

Boundary rules

Section titled “Boundary rules”
  • protocol/ is low-level wire modeling: JSON shapes only, no conversion.
  • pipeline/ coordinates the full request lifecycle and owns phase order.
  • translation/ is pure at the HTTP carrier boundary: it accepts protocol values and payload/stream carriers, not HTTP Response/Body or provider-private structs.
  • provider/ owns provider request rendering and transport details such as auth headers, upstream URLs, and idle-read timeout.
  • observe/ cuts across the pipeline for diagnostics, but does not make routing or protocol decisions.
  • Semantic stream and HTTP errors should use domain errors instead of being hidden inside std::io::Error.

Dependency direction

Section titled “Dependency direction”
flowchart TD
  cli[cli/] --> lib[lib.rs AppState]
  config[config.rs] --> lib
  paths[paths.rs] --> cli
  lib --> pipeline[pipeline/]


  pipeline --> ingress[ingress/]
  pipeline --> routing[routing/]
  pipeline --> provider[provider/]
  pipeline --> upstream[upstream/]
  pipeline --> translation[translation/]
  pipeline --> observe[observe/]
  pipeline --> http_support[http_support/]
  pipeline --> error[error/]


  ingress --> protocol[protocol/]
  translation --> protocol
  provider --> protocol
  upstream --> http_support
  provider --> http_support
  translation --> http_support
  ingress --> sse[sse.rs]
  upstream --> sse
  translation --> sse
  observe --> error
  lib --> observe
  lib --> error

Key rules:

  • protocol/ is low-level wire modeling.
  • pipeline/ is the coordinator that knows about the full request lifecycle.
  • translation/ does not depend on HTTP Response/Body or provider-private types.
  • observe/ cuts across the pipeline but does not make protocol or routing decisions.

Translation selection

Section titled “Translation selection”

Translation is selected from two protocol values:

  • inbound request_protocol, detected by ingress/
  • provider protocol, configured on the selected provider

Rules:

  • Same protocol: pass through without protocol conversion.
  • Different protocols: dispatch to translation/<inbound_protocol>/to_<provider_protocol>/.
  • Unsupported pairs fail explicitly.

Data type conventions

Section titled “Data type conventions”
  • Prefer top-level enums keyed by protocol for protocol-specific request/response data.
  • Avoid parallel protocol / payload / projection / summary fields that can drift into impossible states.
  • Keep streams as ByteStream across HTTP carrier boundaries.
  • Keep provider names separate from protocol names.