How ingestion works

Every span you send, by whatever route, goes through one pipeline. It checks the span, cleans it, prices it and stores it. This page describes each step and the limits you can run into.

The idea

Ingestion is synchronous and idempotent.

• Synchronous means that when the native or OTLP endpoint answers 200, the spans in accepted are already stored. There is no queue between the request and the database. The gateway is the one exception: it answers your model call first and stores the span right after.
• Idempotent means that sending the same spans twice leaves the same data as sending them once. That is what lets the SDK and OTLP exporters retry without double counting.

How it works

The native endpoint (POST /v1/traces) and the OTLP endpoint (POST /v1/otlp/traces) differ only in how the body is decoded. After decoding, both hand the same span shape to the same pipeline, and so does the gateway.

At the door

Before any span is read, the request must pass these checks.

• The key must be a project API key with the ingest:write scope. A key without it gets 403 KEY_SCOPE.
• Each key may send 50 requests a second, with bursts of up to 100.
• The body may be at most 5 MiB after decompression (gzip and deflate are accepted) and hold at most 1000 spans. A larger one gets 413 PAYLOAD_TOO_LARGE.
• The body must be valid JSON (or protobuf for OTLP). NaN, Infinity, numbers beyond a double and JSON nested more than 64 levels deep are refused with 400 INVALID_JSON.

One span at a time

Each span is judged on its own, so one bad span does not fail the batch. A refused span comes back in rejected with its index and a reason; the others are stored.

1. Validate. The shape is checked: ids are 32 and 16 hexadecimal characters and not all zeros, timestamps are ISO 8601 with a UTC offset or integer Unix nanoseconds, token counts are whole numbers, and text fields have length limits. Fields Spanlight does not know are ignored.
2. Check the times. A span is refused when it ends before it starts, starts more than 90 days ago, ends more than 10 minutes in the future, or names itself as its parent. The 10 minutes are the allowance for clocks that run ahead.
3. Check the usage. cached_tokens plus cache_write_tokens cannot be larger than input_tokens, because input tokens include the cache reads and the cache writes, and cache_write_1h_tokens cannot be larger than cache_write_tokens, nor be sent without it. How cost is calculated lists the exact reasons.
4. Redact. Secrets are replaced with [REDACTED] in the input, the output, the attributes, the status message, the span name and the trace name. See what is redacted.
5. Truncate. An input or output of more than 1 MB (1 MiB, after redaction) is replaced by a string holding the first 1 MB of its JSON text, and the span is marked truncated. Redaction runs first, so a secret cut in half by the limit cannot survive in part.
6. Limit the attributes. Attributes larger than 64 KB, or more than 256 of them, refuse the span. Attributes are never truncated.
7. Drop payloads if asked. When the project has payload capture off, the input and output are not stored at all. Token counts, cost and everything else still are.
8. Price. The span gets a cost when its model has a known price. See How cost is calculated.
9. Classify. A failed span gets an error class, and the span gets a request hash and a canonical finish reason. These are the fields the Doctor reads. The hash is worked out before payloads are dropped, so it exists even when you store no prompts.

If the same span id appears twice in one batch, the later one is kept and the earlier is reported as rejected.

One transaction

The accepted spans of a request are written together.

1. Spanlight takes a shared lock on the project row, so a project deleted at that moment makes the write wait and then fail cleanly instead of deadlocking.
2. It upserts the traces. Trace fields follow the rule from the data model: the last non-empty value wins, a missing value never erases a stored one, tags are combined and the start and end times widen.
3. It upserts the spans on their natural key, the project, trace id and span id.
4. It recomputes the totals of every touched trace from the spans that are now stored: span count, error count, tokens and cost.

Because the totals are recomputed from stored spans and never added to, a replayed batch cannot inflate them. Rows are written in sorted order so that two requests touching the same traces cannot block each other forever.

A span that the gateway recorded belongs to the gateway. If a native or OTLP span arrives with the same ids, the pipeline counts it as accepted but leaves the stored span alone, so a replay cannot rewrite the cost a gateway key is accountable for.

When a batch is replayed, the one thing that can change is the price. Spans are priced again with the prices in force at that moment, so a price change between two sends changes the cost and pricing_version of the replayed spans.

What is redacted

Redaction looks at the span and trace names and every string in the input, output, attributes and status message, including the names of keys in JSON objects. It replaces:

• Spanlight credentials of every kind.
• Anthropic and OpenAI-style API keys.
• Bearer tokens.
• AWS access key ids, and AWS secret keys that are labelled as such.
• Card numbers: groups of 13 to 19 digits that pass the Luhn check.

The patterns favour catching a secret over sparing a harmless string, but they are not broad enough to eat ordinary prose. A secret in a shape that none of them match is stored as sent. If a payload must never be stored, turn payload capture off for the project instead of relying on redaction.

What it means for you

• A 200 means stored. A span in rejected is not stored, so log or alert on it. The reason names the field.
• Retry freely. A network failure after you sent a batch is safe to retry whole.
• Send trace fields on at least one span of the trace. You do not need to repeat them on every span.
• Keep attributes small. Put large content in the input or output, where it is truncated instead of refused.
• Expect prompts over 1 MB to be cut when you read them back. The truncated flag tells you when.
• Do not use redaction as access control. It reduces accidents; it is not a guarantee.

Related pages

Ways to send dataThe SDK, OTLP and the gateway compared.

How cost is calculatedThe price table and what happens when a price is unknown.

OTLP ingestionSend OpenTelemetry GenAI traces and see how attributes map.

Python SDKDelivery guarantees, batching and privacy settings.