Skip to content
WP EngineDocumentation

Stream a chat completion

Set "stream": true on a chat completion request to receive the response as Server-Sent Events (SSE) instead of a single JSON payload, so you can render tokens to the UI as they’re produced. SSE chunks don’t arrive aligned on event boundaries — a single read may contain several complete data: lines plus a partial line, or just a fragment of one — so the steps below read the stream incrementally and buffer partial lines rather than parsing each read in isolation.

1. Request the stream

Section titled “1. Request the stream”

Set stream: true in the request body. The response headers are the same as a non-streaming request; the body arrives as a sequence of data: { ... } events instead of a single JSON object.

const BASE_URL = "https://api.ai.wpengine.com";


const res = await fetch(`${BASE_URL}/v1/chat/completions`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.POWER_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "google/gemini-3.5-flash",
    stream: true,
    messages: [{ role: "user", content: prompt }],
  }),
});


if (!res.ok || !res.body) {
  throw new Error(`stream request failed: ${res.status}`);
}

2. Read the response incrementally

Section titled “2. Read the response incrementally”

Read the body in chunks instead of waiting for it to complete. Each read may return zero or more complete lines plus a trailing partial line — buffer that remainder and prepend it to the next read.

The streaming TextDecoder preserves multi-byte UTF-8 sequences across chunk boundaries, so decode with it rather than converting each chunk independently.

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = "";


while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });


  // continue to step 3 with `buffer`
}

3. Parse SSE lines and detect completion

Section titled “3. Parse SSE lines and detect completion”

Split the buffer on newlines, holding back the last (possibly incomplete) element for the next read. For each complete line: skip anything that doesn’t start with data: ; treat the sentinel data: [DONE] as the end of the stream; otherwise parse the JSON chat.completion.chunk and hand it to the caller.

const lines = buffer.split("\n");
buffer = lines.pop() ?? "";


for (const line of lines) {
  if (!line.startsWith("data: ")) continue;
  const data = line.slice(6).trim();
  if (data === "[DONE]") return;
  onDelta(JSON.parse(data));
}

4. Call it and handle each delta

Section titled “4. Call it and handle each delta”

Wrap steps 1-3 in a function that takes a prompt and a callback, then call it. This example writes each delta to stdout as it arrives.

async function chatStream(prompt, onDelta) {
  // steps 1-3 combined
}


await chatStream("Explain how a CDN works in about 100 words.", (chunk) => {
  process.stdout.write(chunk.choices?.[0]?.delta?.content ?? "");
});

5. Observe the output

Section titled “5. Observe the output”

Run the code from step 4 and watch stdout: tokens should print incrementally as the model produces them, not all at once at the end. If nothing appears until the whole response completes, the stream is being buffered somewhere in your stack — check for a proxy or middleware that buffers responses, and confirm you’re reading the body incrementally (res.body.getReader() or Guzzle’s stream option) rather than awaiting the full body first.

Last updated: