Streaming

Set "stream": true on POST /v1/chat/completions or POST /v1/conversations/{conversationId}/messages and the answer arrives as server-sent events (SSE) while the model generates it. Chat completions stream the OpenAI chat.completion.chunk shape, so the official SDKs consume it unchanged. Conversation messages stream a smaller, eCourtDate-specific frame shape described below.

Response headers

A streaming response is 200 with these headers and no Content-Length:

Header	Value
Content-Type	text/event-stream; charset=utf-8
Cache-Control	no-cache
Connection	keep-alive
X-Accel-Buffering	no (tells intermediate proxies not to buffer)
X-Request-ID	Echoed or minted, as on every response
RateLimit-Limit, RateLimit-Remaining, RateLimit-Reset	The per-minute request window, as on every authenticated /v1 response (Rate limits)

Branch on Content-Type: application/json means the request was rejected before the stream started and the HTTP status is meaningful; text/event-stream means the status is 200 and any later failure can only arrive in-band.

Frame format

Every frame is the five characters data: , one compact JSON document on a single line, and a blank line (\n\n). The stream ends with the sentinel data: [DONE] followed by a blank line. Line endings are LF only. The API never sends event:, id:, or retry: lines, and never sends comment or keepalive lines.

data: {"id":"chatcmpl-8a317b1a95a74321b73b6567","object":"chat.completion.chunk","created":1787320991,"model":"court-assistant","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-8a317b1a95a74321b73b6567","object":"chat.completion.chunk","created":1787320991,"model":"court-assistant","choices":[{"index":0,"delta":{"content":"The Traffic Division is open Monday through Friday, 8:00 a.m. to 4:30 p.m. [Sou"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-8a317b1a95a74321b73b6567","object":"chat.completion.chunk","created":1787320991,"model":"court-assistant","choices":[{"index":0,"delta":{"content":"rce 1]"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-8a317b1a95a74321b73b6567","object":"chat.completion.chunk","created":1787320991,"model":"court-assistant","choices":[{"index":0,"delta":{"citations":[{"source_index":1,"document_id":"17136f01-972d-4956-868e-8159833072e4","source_filename":"traffic-division-faq.pdf","chunk_index":2,"text_preview":"Traffic Division hours: Monday through Friday, 8:00 a.m. to 4:30 p.m. Payments are accepted at the clerk's window until 4:00 p.m.","score":0.93}]},"logprobs":null,"finish_reason":"stop"}]}

data: {"id":"chatcmpl-8a317b1a95a74321b73b6567","object":"chat.completion.chunk","created":1787320991,"model":"court-assistant","choices":[],"usage":{"prompt_tokens":812,"completion_tokens":143,"total_tokens":955}}

data: [DONE]

(The last data frame appears only when stream_options.include_usage is true.)

Chat completion chunks

Every chunk has exactly the keys id, object, created, model, and choices; the usage chunk adds usage. id, created, and model are identical on every chunk of one stream, and model is the bot's canonical slug, as in the non-streaming response. choices[0] always has index: 0 and logprobs: null.

A stream is strictly ordered:

1. One role chunk. delta is {"role": "assistant", "content": ""} and finish_reason is null. It is sent before the model is contacted, so it arrives even when the model call later fails.
2. Zero or more content deltas. delta is {"content": "<fragment>"} with no role key. Concatenate the fragments in order.
3. Zero or more tool-call deltas, described below.
4. Exactly one finish chunk. The only chunk with a non-null finish_reason (stop, length, content_filter, or tool_calls; see finish_reason). Its delta is {} or carries citations.
5. An optional usage chunk, only when you asked for it.
6. data: [DONE].

Citations

Citations arrive once, on the finish chunk, as the extension field delta.citations: an array of citation objects. The key is present on the finish chunk of every text answer from a bot with a knowledge base and citations enabled, as an empty array when the answer cites nothing. It is absent (the finish delta is {}) on a tool-call turn and for bots that do not cite.

[Source N] markers can straddle content deltas ("[Sou" then "rce 1]" above), so never parse markers from partial text. Render the markers after the finish chunk, using delta.citations to resolve them. See Rendering citations.

Usage chunk

Send "stream_options": {"include_usage": true} with stream: true to get a final chunk after the finish chunk. It has "choices": [] and usage: {prompt_tokens, completion_tokens, total_tokens} with the same id, created, and model as the rest of the stream. No other chunk carries a usage key. Clients must tolerate the empty choices array.

stream_options is validated on every request but only acted on when stream is true (the ordinary JSON response already includes usage). include_usage must be a JSON boolean; anything else is rejected with 400 and param: "stream_options.include_usage", even when stream is false.

Token usage is metered against your daily quota when the model reports it, at the finish chunk, whether or not you read the stream to [DONE]. Nothing is metered if the stream fails before that point.

Tool-call deltas

When the model calls functions, the deltas carry tool_calls arrays instead of content. The first delta for a call carries its index, id, type, function.name, and an empty function.arguments; later deltas for the same index carry only function.arguments fragments:

data: {"id":"chatcmpl-8a317b1a95a74321b73b6567","object":"chat.completion.chunk","created":1787320991,"model":"clerk-tools","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-8a317b1a95a74321b73b6567","object":"chat.completion.chunk","created":1787320991,"model":"clerk-tools","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_01Hx7Qm2xT9vL4nR8sW1pB6c","type":"function","function":{"name":"lookup_hearing","arguments":""}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-8a317b1a95a74321b73b6567","object":"chat.completion.chunk","created":1787320991,"model":"clerk-tools","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"case_number\": "}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-8a317b1a95a74321b73b6567","object":"chat.completion.chunk","created":1787320991,"model":"clerk-tools","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"CR-2026-0412\"}"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-8a317b1a95a74321b73b6567","object":"chat.completion.chunk","created":1787320991,"model":"clerk-tools","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"tool_calls"}]}

data: [DONE]

• index is 0-based over the tool calls of this answer; a second call starts at index: 1. Accumulate function.arguments per index, then parse the concatenated string as JSON.
• Tool-call id values are opaque strings. Do not parse or validate their format; pass them back unchanged as tool_call_id.
• The finish chunk has delta: {} and finish_reason: "tool_calls" whenever any tool call was streamed. A tool-call turn never carries citations.

Continue the loop exactly as in the non-streaming case: see The tool loop.

Errors

Before the stream starts, a request is rejected with an ordinary JSON envelope and a real HTTP status: 400 validation, 401, 403, 404 model_not_found, 413, 429, 500, and 503 exactly as listed under chat completion errors. A retrieval failure on a bot with a knowledge base is also reported this way (for example 503 upstream_unavailable when the text embedding service fails), because retrieval runs before the stream opens.

After the stream starts, the status is already 200 and the headers are sent, so a failure is reported in-band: the chunks produced so far, then one error frame carrying the standard envelope, then data: [DONE]. No finish chunk and no usage chunk follow, even when include_usage was requested.

data: {"id":"chatcmpl-8a317b1a95a74321b73b6567","object":"chat.completion.chunk","created":1787320991,"model":"court-assistant","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}

data: {"error":{"message":"The model is currently rate limited. Please retry shortly.","type":"rate_limit_error","param":null,"code":"rate_limit_exceeded"}}

data: [DONE]

The error frame has no id, object, or choices: detect it by the presence of the error key rather than by position. Four frames can occur:

• code: rate_limit_exceeded with type: rate_limit_error (rate_limit_exceeded): the underlying language model is rate limited; wait about 10 seconds and resend.
• code: upstream_unavailable with type: server_error (upstream_unavailable): the model timed out or is unavailable; retry with backoff.
• code: null with type: server_error and the message Internal server error.: an unexpected failure; retry, then contact support with the X-Request-ID.
• code: null with type: invalid_request_error and the message The model provider rejected the request.: the underlying language model refused the request as sent (typically a tool definition, a JSON schema, or a parameter combination it cannot accept). Do not retry unchanged; change the request. This is the same condition that a non-streaming call reports as an HTTP 400.

The two null codes are told apart by type: server_error frames are retryable, invalid_request_error frames are not. No Retry-After header is possible in-band.

Nothing is metered or persisted for a stream that ends in an error frame, so resending the same request (once corrected, for an invalid_request_error frame) is safe.

The OpenAI SDKs surface the error frame as an exception (openai.APIError in the Python SDK, APIError in the Node SDK) carrying the frame's message, after having yielded the earlier chunks.

SDK examples

OpenAI Python SDK

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["API_KEY"],
    base_url="https://api.chatbots.ecourtdate.com/v1",
)

stream = client.chat.completions.create(
    model="court-assistant",
    messages=[{"role": "user", "content": "What are the Traffic Division hours?"}],
    stream=True,
    stream_options={"include_usage": True},
)

text = []
citations = []
for chunk in stream:
    if not chunk.choices:           # the usage chunk has an empty choices array
        print("tokens:", chunk.usage.total_tokens)
        continue
    delta = chunk.choices[0].delta
    if delta.content:
        text.append(delta.content)
        print(delta.content, end="", flush=True)
    extra = delta.model_extra or {}  # extension fields live here
    if extra.get("citations"):
        citations = extra["citations"]

print()
print("".join(text))
print(citations)

OpenAI Node SDK

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.API_KEY,
  baseURL: "https://api.chatbots.ecourtdate.com/v1",
});

const stream = await client.chat.completions.create({
  model: "court-assistant",
  messages: [{ role: "user", content: "What are the Traffic Division hours?" }],
  stream: true,
  stream_options: { include_usage: true },
});

let text = "";
let citations = [];
for await (const chunk of stream) {
  if (chunk.choices.length === 0) {
    console.log("tokens:", chunk.usage.total_tokens);
    continue;
  }
  const delta = chunk.choices[0].delta;
  if (delta.content) {
    text += delta.content;
    process.stdout.write(delta.content);
  }
  if (delta.citations) citations = delta.citations; // extension field
}
console.log("\n", citations);

curl

curl -N "https://api.chatbots.ecourtdate.com/v1/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "court-assistant",
    "messages": [{ "role": "user", "content": "What are the Traffic Division hours?" }],
    "stream": true,
    "stream_options": { "include_usage": true }
  }'

-N disables curl's output buffering so frames print as they arrive.

Conversation streams

POST /v1/conversations/{conversationId}/messages with "stream": true uses the same headers, framing, sentinel, and error rule, but its frames are not chat.completion.chunk objects. They are:

1. Zero or more delta frames: {"conversation_id": "<id>", "delta": {"content": "<fragment>"}}.
2. One done frame: {"conversation_id": "<id>", "message": {"role": "assistant", "content": "<full text>", "citations": [...] or null}, "done": true}.
3. data: [DONE].

data: {"conversation_id":"c856e096-99bb-45c1-a0df-044dbbe0aeb7","delta":{"content":"Jury duty at the county courthouse begins at 8:"}}

data: {"conversation_id":"c856e096-99bb-45c1-a0df-044dbbe0aeb7","delta":{"content":"30 a.m. Report to the jury assembly room on the second floor with your summons and a photo id. [Source 1]"}}

data: {"conversation_id":"c856e096-99bb-45c1-a0df-044dbbe0aeb7","message":{"role":"assistant","content":"Jury duty at the county courthouse begins at 8:30 a.m. Report to the jury assembly room on the second floor with your summons and a photo id. [Source 1]","citations":[{"source_index":1,"document_id":"2f6e1c0a-8d3b-4c57-9a41-5b2e8d7f0c13","source_filename":"juror-handbook.pdf","chunk_index":0,"text_preview":"Report to the jury assembly room on the second floor by 8:30 a.m. Bring your summons and a photo id.","score":0.91}]},"done":true}

data: [DONE]

Differences from a chat completion stream:

• There is no role frame, no id, object, created, model, or finish_reason, and no usage frame; stream_options is ignored.
• The done frame is the authoritative final text. Its message is the same object the non-streaming call returns, and citations is always present on it: an array (empty when nothing was cited) for a bot with a knowledge base and citations enabled, null otherwise.
• Because there is no leading frame, an error frame can be the very first frame. Key off the error key, not frame position.
• The turn (your message and the reply) is stored before the done frame is sent, so a GET issued as soon as you receive done: true already shows both messages. If the stream ends with an error frame, nothing was stored and nothing was metered: resend the message.

A minimal reader in Python, using the requests library:

import json
import os
import requests

url = f"https://api.chatbots.ecourtdate.com/v1/conversations/{CONVERSATION_ID}/messages"
with requests.post(
    url,
    headers={"Authorization": f"Bearer {os.environ['API_KEY']}"},
    json={"content": "What time does jury duty start?", "stream": True},
    stream=True,
) as response:
    response.raise_for_status()      # pre-stream errors have a real status
    for line in response.iter_lines(decode_unicode=True):
        if not line.startswith("data: "):
            continue
        payload = line[len("data: "):]
        if payload == "[DONE]":
            break
        frame = json.loads(payload)
        if "error" in frame:
            raise RuntimeError(frame["error"]["message"])
        if frame.get("done"):
            print("\nfinal:", frame["message"]["content"])
            print("citations:", frame["message"]["citations"])
        else:
            print(frame["delta"]["content"], end="", flush=True)

And in Node with fetch:

const response = await fetch(
  `https://api.chatbots.ecourtdate.com/v1/conversations/${conversationId}/messages`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ content: "What time does jury duty start?", stream: true }),
  },
);
if (!response.ok) throw new Error(`HTTP ${response.status}`); // pre-stream error

const reader = response.body.pipeThrough(new TextDecoderStream()).getReader();
let buffer = "";
while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  buffer += value;
  let end;
  while ((end = buffer.indexOf("\n\n")) !== -1) {
    const line = buffer.slice(0, end);
    buffer = buffer.slice(end + 2);
    if (!line.startsWith("data: ")) continue;
    const payload = line.slice(6);
    if (payload === "[DONE]") break;
    const frame = JSON.parse(payload);
    if (frame.error) throw new Error(frame.error.message);
    if (frame.done) console.log("\nfinal:", frame.message.content, frame.message.citations);
    else process.stdout.write(frame.delta.content);
  }
}

See Conversations for creating conversations and the non-streaming message shape.