Streaming

Set "stream": true to receive the answer as server-sent events (SSE). The response status is 200 with these headers:

content-type: text/event-stream; charset=utf-8
cache-control: no-store
x-accel-buffering: no

OpenAI format

Each event is a data: line holding a JSON chunk, and the stream ends with data: [DONE].

curl -sS -N https://api.ticonai.com/v1/chat/completions \
  -H "Authorization: Bearer $AIP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-id",
    "stream": true,
    "stream_options": { "include_usage": true },
    "max_tokens": 64,
    "messages": [{ "role": "user", "content": "Count to five." }]
  }'

data: {"id":"chatcmpl-req_…","object":"chat.completion.chunk","created":1767225600,"model":"…","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-req_…","object":"chat.completion.chunk","created":1767225600,"model":"…","choices":[{"index":0,"delta":{"content":"One"},"finish_reason":null}]}

data: {"id":"chatcmpl-req_…","object":"chat.completion.chunk","created":1767225600,"model":"…","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-req_…","object":"chat.completion.chunk","created":1767225600,"model":"…","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":9,"total_tokens":21,"prompt_tokens_details":{"cached_tokens":0}}}

data: [DONE]

The chunk with "choices": [] and usage appears only when you set stream_options.include_usage. Billing does not depend on it: the request is always billed from the provider's reported usage.

Anthropic format

The stream is a sequence of named events: message_start, content_block_start, one content_block_delta per piece of text, content_block_stop, message_delta (with the stop reason and the final usage), and message_stop.

curl -sS -N https://api.ticonai.com/v1/messages \
  -H "x-api-key: $AIP_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-id",
    "stream": true,
    "max_tokens": 64,
    "messages": [{ "role": "user", "content": "Count to five." }]
  }'

message_start carries zeroed usage; the authoritative token counts arrive in message_delta.

Keep-alive heartbeats

While the model is silent, the gateway sends an SSE comment line every 15 seconds so that proxies and load balancers keep the connection open:

: keep-alive

SSE parsers, including the official SDKs, ignore comment lines. If you parse the stream by hand, skip lines that start with a colon.

When something goes wrong

The gateway waits a few seconds (5 by default) for the first piece of output before it commits to a 200.

• If the provider fails before that, you get an ordinary JSON error with the normal status, nothing is streamed, and nothing is charged.
• If the provider is slow, the 200 is sent first with heartbeats. A failure before any output is then reported inside the stream and is still not charged.
• If the failure happens after output has started, the stream ends with an error event and the request is billed for what was produced (see below). On the OpenAI format the event is a data: line with an error object, and the stream ends without [DONE]. On the Anthropic format it is an event: error with the Anthropic error body.

A single stream can last up to 10 minutes. A client that stops reading is treated as gone.

Billing when you cancel

You can close the connection at any time, for example by aborting the HTTP request.

• Before the first output was produced: the hold is released and you are not charged.
• After output started: you are billed for the input and for the output already generated. Input is counted from your request, and output from the text the gateway had already received from the model, so a few tokens that were generated but not yet delivered can be included. Cached-token discounts are not applied to cancelled requests.

const res = await fetch("https://api.ticonai.com/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.AIP_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "your-model-id",
    stream: true,
    max_tokens: 200,
    messages: [{ role: "user", content: "Write a long story." }],
  }),
});
if (!res.ok || !res.body) throw new Error(`HTTP ${res.status}`);

const reader = res.body.getReader();
await reader.read(); // the first chunk
await reader.cancel(); // hang up: billed for input plus the output generated so far
console.log("cancelled after the first chunk");

stream requests ignore the Idempotency-Key header; see Idempotency.