Python SDK

The spanlight package traces LLM applications. It needs Python 3.10 or newer and httpx; openai and anthropic are optional extras.

Low overhead. Tracing never waits on the network: spans are queued in memory and sent in batches from a background thread.
Tracing never breaks your app. SDK failures are logged and swallowed. Exceptions from your code and from the providers are recorded and re-raised unchanged. (prompts.get is the one call that can block and raise; see Prompts and the errors in the reference.)
Typed. The package ships py.typed and passes mypy --strict.

Before you start

Python 3.10 or newer
A running Spanlight instance that your application can reach
A project API key from Settings, Project, API keys. Creating one needs the member, admin or owner role. The default scope ingest:write sends traces; prompts:read and scores:write are needed for prompts and scores

Install

pip install "spanlight[anthropic]"          # or spanlight[openai], or both

Plain pip install spanlight is enough if you only use @observe and span. To run the code on main instead of the last release:

pip install "spanlight[anthropic] @ git+https://github.com/dheerajkarwasra/spanlight#subdirectory=sdks/python"

Configure

Create a project in Spanlight, generate an API key under Settings → API keys, and give it to the SDK:

export SPANLIGHT_API_KEY=spl_live_...
export SPANLIGHT_HOST=http://localhost:8080   # your instance

import spanlight

spanlight.init(environment="production", release="2026.10.07")

Without an API key the SDK logs a warning and runs as a no-op.

Option	Environment variable	Default	Meaning
`api_key`	`SPANLIGHT_API_KEY`	none	Project API key.
`host`	`SPANLIGHT_HOST`	`http://localhost:8000`	Base URL of the Spanlight server. Through the Compose stack that is the web port, `http://localhost:8080`.
`environment`	`SPANLIGHT_ENVIRONMENT`	none	Attached to every trace, for example `production`.
`release`	`SPANLIGHT_RELEASE`	none	Your app version or git SHA.
`enabled`	`SPANLIGHT_ENABLED`	`true`	`false`, `0`, `no` or `off` turns the SDK into a no-op.
`flush_interval`		`2.0`	Seconds a span may wait before it is sent.
`batch_size`		`100`	Spans per request (at most 1000).
`max_queue`		`10000`	Buffered spans. When the buffer is full the oldest spans are dropped.
`gzip`		`True`	Gzip-compress request bodies. Turn it off only for a proxy that cannot forward a compressed body.
`timeout`		`10.0`	HTTP timeout per request, in seconds.
`max_retries`		`5`	Retries for network errors, `429` and `5xx`.
`shutdown_timeout`		`5.0`	Time budget for the flush at interpreter exit.

Trace your own code

import spanlight

@spanlight.observe(kind="retrieval")
def search_docs(query: str) -> list[str]:
    return ["Refunds take 5-7 business days."]

@spanlight.observe(name="support-request")
def handle(question: str, user_id: str) -> str:
    spanlight.update_trace(
        user_id=user_id, session_id="chat-123", tags=["beta"], metadata={"customer_id": "acme"}
    )
    docs = search_docs(question)
    with spanlight.span("format-answer", kind="chain") as span:
        answer = f"From our docs: {docs[0]}"
        span.set_output(answer)
    return answer

This produces one trace with three nested spans.

@observe(name=None, kind="chain", capture_input=True, capture_output=True) traces plain functions, async def, generators and async generators. kind is one of llm, tool, retrieval, chain, http or other. Both @observe and @observe(...) work.
span(name, kind="chain", input=None, attributes=None) is a context manager (also async with) that yields a Span with set_input, set_output, set_attribute, set_attributes, set_model, set_usage, mark_first_token and record_error. An exception that escapes the block marks the span as failed and is re-raised unchanged, even one whose __str__ raises. The setters never raise: a value they cannot use, such as a token count that is not a number, is ignored with a debug log.
update_trace(name=None, user_id=None, session_id=None, tags=None, metadata=None) sets trace-level fields from anywhere inside the trace. Tags accumulate. Metadata keys accumulate too, a later value replacing a key’s earlier one; the Attribution page groups cost by them. Up to 32 keys of a-z, 0-9, _, . and -, values of 1 to 256 characters; an invalid entry is dropped with a debug log. The other fields are made to fit the server’s limits instead of losing the trace: "" counts as unset, non-strings become text, ids and the name are cut to 256 characters, tags="beta" is one tag, and a trace keeps 20 tags of up to 64 characters (details).

Long-running servers need nothing else. Short scripts, CLIs and serverless handlers should call spanlight.flush() before they exit; an atexit hook also flushes, with a timeout.

Choose what an input records

@observe records every argument of the call as the span’s input, keyed by parameter name. For a chat function that receives the whole history, that is the full conversation on every turn. To record only the latest message, switch the automatic capture off with capture_input=False and set the input yourself with get_current_span().set_input(...):

import spanlight


@spanlight.observe(name="support-chat", capture_input=False)
def chat_turn(history: list[dict[str, str]], user_id: str) -> str:
    span = spanlight.get_current_span()
    if span is not None:
        span.set_input(history[-1]["content"])
    spanlight.update_trace(user_id=user_id)
    return "Refunds take 5-7 business days."


chat_turn(
    [
        {"role": "user", "content": "Hi"},
        {"role": "assistant", "content": "How can I help?"},
        {"role": "user", "content": "How long do refunds take?"},
    ],
    "user-42",
)

The span’s input is now the string How long do refunds take?. set_input replaces any earlier input of that span, and it is ignored once the span has ended. Calls to a wrapped provider client made inside the function still record their own messages on their own spans.

OpenAI and Anthropic

import spanlight
from anthropic import Anthropic
from openai import OpenAI

spanlight.init()
anthropic_client = spanlight.wrap_anthropic(Anthropic())
openai_client = spanlight.wrap_openai(OpenAI())

wrap_anthropic instruments messages.create (including stream=True) and messages.stream on Anthropic and AsyncAnthropic. In the next SDK release it also instruments messages.parse, the same methods of client.beta.messages (where fast mode, compaction and the tool runner live), and the clients that with_options(...) and copy(...) return. wrap_openai instruments chat.completions.create and responses.create on OpenAI, AsyncOpenAI and the Azure variants; an Azure client is recorded with the provider azure_openai, which stays unpriced until you add a price override for it. Streaming works without changes to your code and records time to first token. For OpenAI Chat Completions, ask for usage in the stream with stream_options={"include_usage": True}.

If you point the OpenAI client at a compatible API, name the provider yourself: wrap_openai(client, provider="openrouter").

For every LLM call the SDK records the provider, the model (from the response, falling back to the requested one), token usage, time to first token for streams, the request parameters and the assistant message, and the provider’s error message when the call fails. API keys, headers and extra_* arguments are never recorded.

Token counts follow one rule everywhere: input_tokens includes cached tokens, and cached_tokens says how many of them came from the provider’s prompt cache. For Anthropic, the wrapper also reports the tokens written to the cache as cache_write_tokens, and the part written with a 1-hour lifetime as cache_write_1h_tokens (since SDK 0.1.2), because Anthropic bills writes above the input rate. How cost is calculated explains the convention.

SDK 0.2.0, also records what changes the price of a call beyond its tokens:

The service tier the provider billed (usage.service_tier for Anthropic, the top-level service_tier for OpenAI), so a batch or flex call is priced at its own rate. See Service tiers.
Web searches the model ran: Anthropic’s server_tool_use.web_search_requests, and the completed web_search_call items of an OpenAI Responses answer. Each search adds the provider’s fee to the cost. See Fees (web search).
Streams cut before their usage. When a stream ends without the provider’s final usage event (you closed it early, it failed, or an OpenAI Chat Completions stream had no include_usage), the span keeps the counts that did arrive, leaves the output count and the cost unknown, and says why in usage.unreported_reason. See Streams cut before the usage arrives.
Options Spanlight does not price. A call that passes inference_geo (other than global) or speed="fast" gets the attribute usage.pricing_modifier_unknown, and is priced at the standard rate. A call that fails before any usage comes back was not priced, so its span drops the attribute.

Failed calls

When a provider call fails with an HTTP error, the span records what the provider said besides its message, so Provider health can tell which limit refused the call. From SDK 0.2.0, a failed call’s span carries:

Attribute	Value
`provider.error.type`	The provider’s error type, such as `rate_limit_error` or `insufficient_quota`.
`provider.error.code`	The provider’s error code: OpenAI’s `error.code`, or Anthropic’s `error.details.error_code`, such as `enforced_spend_limit_reached`.
`provider.retry_after_s`	The wait the provider asked for, in seconds: `retry-after-ms` divided by 1000, else a numeric `retry-after`. A date in `retry-after` is ignored.
`provider.ratelimit`	The response’s rate-limit headers, as an object of header name to value.
`http.status_code`, `provider.request_id`	The status and the provider’s request id, as before.

Connection errors and timeouts have no response, so they add none of these.

When your client talks to the Spanlight gateway and the gateway itself refused the call, for example for your gateway key’s rate limit, the answer is the gateway’s, not the provider’s. The SDK reads the gateway’s X-Spanlight-Code header, records the code as spanlight.gateway.error_code and sets none of the provider.* fields, so the refusal shows as the gateway key limit, not as a provider limit.

Batch results

A batch request is answered hours later, without its request, and billed at the batch rate. The SDK records each result as an llm span when you read it. These need SDK 0.2.0.

Anthropic. Results you read through a wrapped client are recorded as you iterate, sync or async. Pass the batch id, positionally or as message_batch_id:

import spanlight
from anthropic import Anthropic

spanlight.init()
client = spanlight.wrap_anthropic(Anthropic())

for result in client.messages.batches.results("msgbatch_..."):
    if result.result.type == "succeeded":
        print(result.custom_id, result.result.message.content)

spanlight.flush()

OpenAI. The Batch API writes its results to a file, so pass the file’s lines to record_openai_batch_output. It returns the number of spans it recorded and never raises:

import json

import spanlight

spanlight.init()

line = json.dumps(
    {
        "id": "batch_req_1",
        "custom_id": "request-1",
        "response": {
            "status_code": 200,
            "body": {
                "model": "gpt-4o-mini",
                "choices": [
                    {
                        "index": 0,
                        "message": {"role": "assistant", "content": "Hello!"},
                        "finish_reason": "stop",
                    }
                ],
                "usage": {"prompt_tokens": 12, "completion_tokens": 3, "total_tokens": 15},
            },
        },
        "error": None,
    }
)
recorded = spanlight.record_openai_batch_output([line], batch_id="batch_abc123")
print(recorded)  # 1
spanlight.flush()

With a real batch, read the output file with the OpenAI client and pass its text, or its lines: spanlight.record_openai_batch_output(client.files.content(batch.output_file_id).text, batch_id=batch.id). Pass the error file the same way to record the requests that failed.

What a batch result records:

One trace per batch, named batch <batch_id> and tagged spanlight:batch, with one span per request, named batch <model>.
The usage at the batch tier, whatever tier the provider reported; the provider’s value is kept in usage.service_tier_raw.
The output and the finish reason. There is no input, because the result does not carry the request (batch.request_available is false).
The attributes batch.id, batch.custom_id and batch.result_type.
A start and end time equal to the moment you read the result, so the duration is 0; timing.synthetic is true to say so.
For a request that failed (Anthropic errored, canceled or expired; OpenAI a status other than 200 or an error), an error span without usage, because it was not billed. An OpenAI Responses body with status: "failed" keeps the usage it reports.

The trace and span ids are derived from the batch id and each custom_id, so reading the same results twice updates the same spans instead of adding new ones.

Privacy

The server redacts common secret patterns from payloads on ingestion, and a project can switch payload capture off entirely, so inputs and outputs are dropped. To keep payloads out of the SDK for one function, use @observe(capture_input=False, capture_output=False).

The rate-limit headers of a failed call pass a fixed allowlist of 29 names before they leave your process: retry-after, retry-after-ms, OpenAI’s x-ratelimit-* headers and Anthropic’s anthropic-ratelimit-* and anthropic-priority-* headers. Each value must be printable ASCII of at most 64 characters. Every other response header is dropped, including authorization, x-api-key, cookie, set-cookie and Anthropic’s organization and workspace ids. The server applies the same allowlist again to whatever it receives.

Delivery guarantees

A batch goes out when it reaches batch_size spans or after flush_interval seconds. Network errors, 429 and 5xx are retried with jittered exponential backoff, and Retry-After is honoured. Other 4xx responses are not retried; they are logged to the spanlight logger, as are spans the server rejects. Ingestion is idempotent by design: a retried batch upserts the same rows, so it does not double-count. A span that cannot be encoded as JSON is dropped alone and counted as failed; the rest of its batch is sent.

From SDK 0.2.0, every batch of spans also reports what the SDK lost: a client object with the SDK’s name and version, a random exporter id (new in each process, and in a forked child) and two running totals, the spans and scores dropped because the queue was full and those lost after retries or rejected by the server. It holds no span content. The server turns the totals into telemetry completeness; an older server ignores the object.

See what the SDK is doing with:

import logging
logging.getLogger("spanlight").setLevel(logging.DEBUG)

Threads do not inherit the current span, so copy the context when you hand work to a thread pool:

import contextvars

context = contextvars.copy_context()
executor.submit(context.run, do_work, item)

spanlight.get_current_trace_id() returns the current trace id so you can put it in your logs.

Prompts and scores

spanlight.prompts.get("support/answer") fetches a prompt version by label from Spanlight, with a cache, a stale copy when the server is unreachable and an optional local fallback, and spanlight_prompt=compiled links the span to the version it used. spanlight.score(trace_id, "helpfulness", 0.8) attaches a score to a trace. Prompts need an API key with the prompts:read scope and scores one with scores:write. See Prompts and Scores.

Check it worked

Call handle("How long do refunds take?", "u_42") from Trace your own code, then call spanlight.flush(). Open Traces: the list has a trace named support-request. Open it. The header shows the User u_42, the Session chat-123 and the tag beta, and the waterfall shows support-request with search_docs and format-answer under it.

Troubleshooting

The log says “no API key configured” and nothing is sent. Without a key the SDK does nothing. Set SPANLIGHT_API_KEY or pass api_key=. See The SDK has no API key, or is switched off.
The log says “Dropping N spans after 6 attempts”. A batch failed on every attempt and was dropped. The causes are network errors, 429 and 5xx answers, which the SDK retries up to max_retries times. A wrong host is the most common one: the default is http://localhost:8000, and the Compose stack serves on 8080. See The SDK points at the wrong host and The whole request is refused.
The log says “Spanlight rejected a batch of N spans: HTTP 403”. The key has no ingest:write scope. The SDK does not retry a 403. Create a key with the scope. See The key lacks the ingest:write scope.
The last spans of a script, a serverless handler or a test are missing. The process ended before the background thread sent them. Call spanlight.flush() before it exits. See A short script exits before the spans are sent.
Spans created in a thread pool are not under the span that started the work. Threads do not inherit the current span. Run the work through contextvars.copy_context(), as shown in Delivery guarantees.

The Python SDK reference lists every function, class and option with its signature and defaults, and the limits of the wrappers. A complete example is a small three-step support bot.

Next steps

Python SDK referenceEvery function, class and option, with signatures and defaults.

PromptsVersions and labels that you can change without a deploy.

Scores and review queuesJudgements attached to traces, and queues to review them.

OTLP ingestionSend traces from OpenTelemetry instead of the SDK.