#Streaming

Streaming returns generated output incrementally instead of waiting for the complete response. GetRouter supports streaming request fields on Chat Completions and Responses endpoints when the selected model and channel support them.

#Chat Completions with cURL

Set stream to true and use curl -N to disable client-side buffering:

curl -N https://getrouter.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MODEL_ID",
    "stream": true,
    "messages": [
      {"role": "user", "content": "Write a four-line poem."}
    ]
  }'

Chat Completions streams Server-Sent Events. Text chunks appear under:

choices[0].delta.content

The stream normally finishes with a terminal event such as [DONE].

#Chat Completions with Python

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://getrouter.ai/v1",
)

stream = client.chat.completions.create(
    model="MODEL_ID",
    stream=True,
    messages=[{"role": "user", "content": "Write a four-line poem."}],
)

for chunk in stream:
    text = chunk.choices[0].delta.content
    if text:
        print(text, end="", flush=True)

#Responses API streaming

curl -N https://getrouter.ai/v1/responses \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MODEL_ID",
    "stream": true,
    "input": "List three benefits of streaming APIs."
  }'

Responses streaming uses Responses API event objects rather than Chat Completions choices[].delta chunks. Parse events according to their type and do not reuse a Chat Completions-only parser.

#stream_options

The Chat Completions schema accepts stream_options, but support depends on the selected model channel. If an upstream-compatible option is rejected, remove it and retry with only stream: true.

#Tool calls while streaming

Tool call arguments can arrive across multiple chunks. Accumulate tool call deltas by index or tool call ID before parsing and executing the complete arguments.

#Handle disconnects

A streaming connection may end before a terminal event because of a client timeout, proxy timeout, network interruption, or upstream error.

Your client should:

• Treat a missing terminal event as an incomplete response
• Preserve already-received text if appropriate
• Avoid blindly retrying non-idempotent tool actions
• Set timeouts long enough for the selected model
• Log the request ID and error without logging the API key

#Troubleshooting

#Output arrives all at once

A client, reverse proxy, or middleware may be buffering SSE data. Test directly with curl -N before debugging your UI.

#The connection closes immediately

Verify the model supports streaming on the endpoint you selected. Also check the error event or HTTP response body before treating it as a network failure.

#Usage is missing from chunks

Usage reporting during streaming depends on the protocol, request options, and model channel. Do not require usage to be present in every chunk.

#Next steps

• Chat Completions
• Responses API
• View pricing