OpenAI

Spanlight can record every OpenAI call your application makes: the prompt, the answer, the tokens, the latency and the cost. There are three ways to do it. Ways to send data compares them in detail.

Way	Languages	What you change
Python SDK	Python	Wrap your OpenAI client with one line.
Gateway	Any	Point your OpenAI client at Spanlight with a new base URL and key.
OpenTelemetry	Any	Export spans from your own instrumentation.

Before you start

• A running Spanlight instance that your application can reach
• An OpenAI API key
• Python SDK: Python 3.10 or newer, and a project API key with the ingest:write scope (the default). Creating one needs the member, admin or owner role
• Gateway: an organization owner has stored an OpenAI credential, and an owner or admin has created a route and a gateway key
• JavaScript and TypeScript: Node.js 24 (the version this page was tested with)

Verified

Tested on 13 Oct 2026 with spanlight 0.1.0, openai (Python) 3.22.0 and openai (npm) 7.25.0.

Run with Python 3.13.9 and Node 24.11.1 against a local Spanlight server and a stand-in for the OpenAI API, not against OpenAI itself. The gateway run used an OpenAI-compatible credential, so its spans showed the provider openai_compatible and no cost. The provider label and cost for a real OpenAI credential are read from the gateway source and the price table. OpenTelemetry was not run.

Python SDK

The wrap_openai function instruments one OpenAI client in place. Every chat.completions.create call made with it becomes an llm span. Your code, the answer and the exceptions you get from OpenAI do not change.

Install the SDK with its OpenAI extra and the OpenAI package:

pip install "spanlight[openai]==0.1.0" "openai==3.22.0"

Give the SDK your Spanlight address and project API key, and OpenAI your OpenAI key:

export SPANLIGHT_HOST=http://localhost:8080
export SPANLIGHT_API_KEY=spl_live_...
export OPENAI_API_KEY=sk-...

SPANLIGHT_HOST is the address of the Spanlight web service. It defaults to http://localhost:8000, which is right only when you run the API directly for development.

A normal call

import spanlight
from openai import OpenAI

spanlight.init(environment="development")
client = spanlight.wrap_openai(OpenAI())

reply = client.chat.completions.create(
    model="gpt-4.1-mini",
    messages=[{"role": "user", "content": "Say hello."}],
)
print(reply.choices[0].message.content)

spanlight.flush()

The SDK queues spans and sends them from a background thread. spanlight.flush() waits until they are delivered. A long-running server does not need it. A short script does, or its last spans are lost.

A streamed call

import spanlight
from openai import OpenAI

spanlight.init(environment="development")
client = spanlight.wrap_openai(OpenAI())

stream = client.chat.completions.create(
    model="gpt-4.1-mini",
    messages=[{"role": "user", "content": "Say hello."}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="")
print()

spanlight.flush()

You iterate the stream as before. The wrapper returns a transparent proxy that records the chunks as you read them, and the span ends when the stream is exhausted or closed. It also records the time to first token.

Ask OpenAI for usage with stream_options={"include_usage": True}. Without it OpenAI sends no token counts for a stream, so the span has no tokens and no cost, and its attribute usage.unreported_reason is include_usage_not_set. A stream you close early or that fails is marked client_disconnected or stream_error the same way. A Responses stream that ends without its terminal event keeps the input counts it saw, with the output count and the cost unknown.

The wrapper also instruments client.responses.create. The samples on this page use Chat Completions only. To name the provider yourself when you point the OpenAI client at a compatible API, pass provider= to wrap_openai; see the reference.

Batch results

The Batch API writes its results to a file, so the wrapper cannot see them. Pass the output file to record_openai_batch_output once the batch has completed, and each request becomes an llm span priced at the batch rate:

batch = client.batches.retrieve("batch_abc123")
output = client.files.content(batch.output_file_id).text
spanlight.record_openai_batch_output(output, batch_id=batch.id)

Pass the error file the same way to record the requests that failed. The spans have no input and a duration of 0, because the result carries neither; Batch results describes them in full. The function needs SDK 0.2.0 or newer.

Gateway

The gateway is an HTTP proxy that speaks the OpenAI API. Your application keeps its OpenAI client and changes two things: the base URL, which becomes <host>/gw/v1, and the API key, which becomes a Spanlight gateway key (spl_gw_…). The gateway forwards the call to OpenAI with the key you stored and records it as a trace. No Spanlight package is installed in your application.

Do the one-time setup first: store the OpenAI key as a credential, create a route and create a gateway key. Gateway quickstart walks through it.

Give your application the address and the gateway key:

export SPANLIGHT_HOST=http://localhost:8080
export SPANLIGHT_GATEWAY_KEY=spl_gw_...

Python

pip install "openai==3.22.0"

import os

from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['SPANLIGHT_HOST']}/gw/v1",
    api_key=os.environ["SPANLIGHT_GATEWAY_KEY"],  # a gateway key, not an OpenAI key
)

reply = client.chat.completions.create(
    model="gpt-4.1-mini",
    messages=[{"role": "user", "content": "Say hello."}],
)
print(reply.choices[0].message.content)

JavaScript

npm install openai@7.25.0

Save this as gateway.mjs and run node gateway.mjs:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: `${process.env.SPANLIGHT_HOST}/gw/v1`,
  apiKey: process.env.SPANLIGHT_GATEWAY_KEY, // a gateway key, not an OpenAI key
});

const reply = await client.chat.completions.create({
  model: "gpt-4.1-mini",
  messages: [{ role: "user", content: "Say hello." }],
});
console.log(reply.choices[0].message.content);

The same code runs as TypeScript: Node 24 runs node gateway.ts directly, and it type-checks with strict on.

cURL

curl -sS "$SPANLIGHT_HOST/gw/v1/chat/completions" \
  -H "Authorization: Bearer $SPANLIGHT_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4.1-mini", "messages": [{"role": "user", "content": "Say hello."}]}'

Streaming needs no change: pass stream: true (or stream=True in Python) and read the stream as usual. When your request has no stream_options at all, the gateway adds stream_options.include_usage for you, so the span has token counts, and it keeps the extra usage chunk out of your stream. The span is written when the stream ends.

The gateway records the provider of the credential that answered. For an OpenAI credential that is openai, and the cost comes from the price table, which includes gpt-4.1-mini. For an OpenAI-compatible credential (a local model server, for example) the provider is openai_compatible, no price is known, and the cost is empty unless you set a price override for that model.

To join a call to a trace, session or user you already track, send x-spanlight-* headers; see Join a trace you already started.

OpenTelemetry

If your application already uses OpenTelemetry instrumentation for OpenAI, you can send its spans to Spanlight over OTLP/HTTP instead. Spanlight reads the gen_ai.* attributes. See OTLP ingestion for the endpoint and the exporter settings, and OTLP attribute mapping for which attribute fills which field.

Check it worked

Open Traces in the dashboard and pick the project that owns your key.

• With the Python SDK you see a trace named chat gpt-4.1-mini that holds one llm span of the same name.
• Through the gateway you see a trace with one llm span named chat_completions gpt-4.1-mini.

Open the span. It shows the messages you sent, the answer, the token counts and the cost.

What is recorded

Each call is one llm span. Where a column says “empty”, Spanlight stores nothing and the dashboard shows “—”.

Field	Python SDK	Gateway
Span name	chat <model>	chat_completions <model>
Trace name	The name of the root span, for example chat gpt-4.1-mini	Empty. The gateway does not name traces
Provider	openai, or the value you pass as provider=	The provider of the credential that answered: openai or openai_compatible
Model	The model OpenAI reports in its answer. If the answer has none, the model you asked for	The same
Input	The request parameters as JSON: model, messages, stream, stream_options and any others you pass	The request body as you sent it
Output	The assistant message	The assistant message
Input and output tokens	From OpenAI’s usage. A streamed call has them only with include_usage	From OpenAI’s usage. A streamed chat call without any stream_options has them, because the gateway adds include_usage. If you send stream_options without include_usage: true, it does not. On a response cache hit the tokens are recorded as 0, because nothing was billed
Cached tokens	When OpenAI reports them, otherwise empty	The same. On a response cache hit it is recorded as 0
Service tier	The service_tier at the top of the answer or the last stream chunk: default, auto and scale are stored as standard, flex as flex, priority and fast as priority. A batch result is always batch	The same
Web searches	For Responses, the answer’s web_search_call items whose status is completed. Chat Completions does not report a count	For Responses answers and streams, the same
Cost	From the price table and your price overrides, times the tier factor, plus $10.00 per 1,000 web searches. Empty for a model with no price, and for a stream without usage	The same
Finish reason	stop, length, tool_calls, content_filter or other	The same
Time to first token	For streamed calls	For streamed calls
Status	ok, or error with the exception type (error.type), plus http.status_code and provider.request_id when the exception carries them	ok, or error with a status message
Environment	The environment you pass to spanlight.init	The environment of the gateway key
User and session	Set with spanlight.update_trace	Set with the x-spanlight-user and x-spanlight-session headers

OpenAI bills flex and batch calls at half price, and Spanlight prices them that way. OpenAI publishes a separate priority rate for each model rather than one factor, so a priority call is priced at the standard rate and its cost explanation marks the tier as unmodelled: the figure is lower than the bill. Through the SDK, a Chat Completions call that passes web_search_options gets the attribute usage.fee_unknown set to openai_chat_web_search, because the answer lists sources, not billed searches, so the search fee is not priced.

The SDK and the gateway also store attributes. The SDK adds response.id, response.finish_reasons and finish_reason (the raw value from OpenAI). A streamed call adds request.stream, and a streamed call that ended without usage adds usage.unreported_reason. A call that reasons adds usage.reasoning_tokens when OpenAI reports a count. The gateway adds spanlight.gateway.* attributes (the key, route, target, attempts, overhead and upstream status) and spanlight.cache. Gateway quickstart lists them. A call with an unknown model or without usage has no cost; How cost is calculated explains when.

What is not recorded

• Your keys and headers. The wrapper never records api_key, extra_headers, extra_query, extra_body, timeout or http_client. The gateway does not forward your gateway key, cookies or any header outside a short allowlist to OpenAI.
• Prompts and completions, when the project turns capture off. Spanlight then keeps the model, tokens, cost and timing and drops the text. See Privacy. The server also redacts common secret patterns from payloads and truncates very large ones.
• Token counts of a streamed Python call without include_usage. The span carries the attribute usage.unreported_reason set to include_usage_not_set instead. SDK 0.1.2 and earlier named it usage.missing_reason.
• The web search fee of a Chat Completions call. OpenAI does not report how many searches it billed; the span carries usage.fee_unknown instead.
• Output of a call that fails before an answer arrives. The span has the error and the request, and no output. A stream that fails midway keeps the part of the answer that arrived, and is marked error.
• Calls the wrapper does not see. It instruments the client you wrap and the two methods chat.completions.create and responses.create. Embeddings, images, audio and other endpoints are not traced, and batch results only through record_openai_batch_output. A call made through with_raw_response or with_streaming_response is passed through untraced.
• A stream you abandon. The span of a stream ends when you read it to the end or close it. If you drop it without either, its span is never sent.
• Anything that bypasses the gateway. Calls that do not use the gateway base URL do not appear.

Troubleshooting

• No trace arrives. Work through No trace arrives. Two causes are common with the SDK. If SPANLIGHT_API_KEY is missing, the SDK logs Spanlight: no API key configured once, and your OpenAI call still works and returns its answer; see The SDK has no API key. If a script ends right after the call, the last spans are lost unless you call spanlight.flush(); see A short script exits before the spans are sent.
• The streamed call has no tokens and no cost. Add stream_options={"include_usage": True} to the call. Through the gateway you do not need it.
• The gateway answers 401 UNAUTHORIZED. The key is missing, unknown or revoked, or you sent an OpenAI key or a project API key instead of a gateway key. Use the spl_gw_… key. If you send both Authorization and x-api-key, they must carry the same key. Gateway error responses lists every code.
• The gateway cost is empty. The credential is OpenAI-compatible, or the model has no price, and you have not set a price override. Check the provider on the span, then see When a cost is unknown.
• The trace has no prompt or answer. The project has prompt and completion capture turned off. Privacy shows where.

Next steps

Python SDKTrace your own functions and set sessions and users.

Gateway quickstartCredentials, routes, retries and the trace headers.

How cost is calculatedThe price table and when a cost is unknown.

TroubleshootingFind out why no trace arrived.