Streaming
Set "stream": true to receive the response as Server-Sent Events in the OpenAI streaming format.
Request
json
{
"model": "gpt-4o",
"messages": [{ "role": "user", "content": "Write a haiku about routing." }],
"stream": true
}The response is delivered with Content-Type: text/event-stream as a sequence of data: chunks, terminated by data: [DONE].
Provider support
- OpenAI, DeepSeek, Grok — native token-by-token streaming.
- Anthropic, Google — return a single normalized JSON response (no token streaming yet).
Notes
- Streamed calls are still metered — the gateway reads the final usage chunk to record token counts and cost.
- Fallback routing happens only before the stream starts; there is no mid-stream retry.
- Output guardrails do not apply to streamed responses (they pass through untouched).