Experiments endpoints
GET /api/v1/projects/{project_id}/experiments
Section titled “GET /api/v1/projects/{project_id}/experiments”List experiments
Newest first, each with its run count and latest run (one query for the page).
| Parameter | In | Type | Required | Notes |
|---|---|---|---|---|
project_id |
path | string (uuid) | yes | |
limit |
query | integer | no | min 1, max 200 Default 50. |
cursor |
query | string or null | no | max length 512 |
| Status | Description | Body |
|---|---|---|
| 200 | Successful Response | application/json: Page_ExperimentOut_ |
| 422 | Validation Error | application/json: HTTPValidationError |
POST /api/v1/projects/{project_id}/experiments
Section titled “POST /api/v1/projects/{project_id}/experiments”Add an experiment
Check and store an experiment. 409 NAME_TAKEN; 404 for a dataset, evaluator or prompt
version the project lacks or a credential the organization lacks; 422 EXPERIMENT_MODEL_UNPRICED, or VALIDATION_ERROR for max_items over the server’s limit.
| Parameter | In | Type | Required | Notes |
|---|---|---|---|---|
project_id |
path | string (uuid) | yes |
Request body (application/json, required): ExperimentSpec
| Status | Description | Body |
|---|---|---|
| 201 | Successful Response | application/json: ExperimentOut |
| 422 | Validation Error | application/json: HTTPValidationError |
GET /api/v1/projects/{project_id}/experiments/{experiment_id}
Section titled “GET /api/v1/projects/{project_id}/experiments/{experiment_id}”Get an experiment
| Parameter | In | Type | Required | Notes |
|---|---|---|---|---|
project_id |
path | string (uuid) | yes | |
experiment_id |
path | string (uuid) | yes |
| Status | Description | Body |
|---|---|---|
| 200 | Successful Response | application/json: ExperimentOut |
| 422 | Validation Error | application/json: HTTPValidationError |
GET /api/v1/projects/{project_id}/experiments/{experiment_id}/runs
Section titled “GET /api/v1/projects/{project_id}/experiments/{experiment_id}/runs”List runs
Newest first, with the totals stored when each run ended (null while unfinished; read the run for live totals).
| Parameter | In | Type | Required | Notes |
|---|---|---|---|---|
project_id |
path | string (uuid) | yes | |
experiment_id |
path | string (uuid) | yes | |
limit |
query | integer | no | min 1, max 200 Default 50. |
cursor |
query | string or null | no | max length 512 |
| Status | Description | Body |
|---|---|---|
| 200 | Successful Response | application/json: Page_ExperimentRunOut_ |
| 422 | Validation Error | application/json: HTTPValidationError |
POST /api/v1/projects/{project_id}/experiments/{experiment_id}/runs
Section titled “POST /api/v1/projects/{project_id}/experiments/{experiment_id}/runs”Run an experiment
Queue a run; the worker executes it. Send an Idempotency-Key to make a retry safe (the
body is part of what a retry must repeat). target_override runs a prompt-version experiment
with another version of the same prompt, same credential and model (422 TARGET_OVERRIDE_INVALID otherwise). 409 RUN_IN_PROGRESS while another run of the
experiment is queued or running.
| Parameter | In | Type | Required | Notes |
|---|---|---|---|---|
project_id |
path | string (uuid) | yes | |
experiment_id |
path | string (uuid) | yes | |
Idempotency-Key |
header | string or null | no | Makes a retry of this request safe: it runs once, then is replayed. min length 1, max length 128, pattern ^[\x21-\x7e]+$ |
Request body (application/json): RunStart or null
| Status | Description | Body |
|---|---|---|
| 202 | Successful Response | application/json: ExperimentRunOut |
| 422 | Validation Error | application/json: HTTPValidationError |
GET /api/v1/projects/{project_id}/experiments/{experiment_id}/runs/{run_a}/compare/{run_b}
Section titled “GET /api/v1/projects/{project_id}/experiments/{experiment_id}/runs/{run_a}/compare/{run_b}”Compare two runs
Deltas are b - a. 404 when either run is not a run of this experiment; 409 RUN_IN_PROGRESS while either is queued or running.
| Parameter | In | Type | Required | Notes |
|---|---|---|---|---|
project_id |
path | string (uuid) | yes | |
experiment_id |
path | string (uuid) | yes | |
run_a |
path | string (uuid) | yes | |
run_b |
path | string (uuid) | yes |
| Status | Description | Body |
|---|---|---|
| 200 | Successful Response | application/json: ComparisonOut |
| 422 | Validation Error | application/json: HTTPValidationError |
GET /api/v1/projects/{project_id}/experiments/{experiment_id}/runs/{run_id}
Section titled “GET /api/v1/projects/{project_id}/experiments/{experiment_id}/runs/{run_id}”Get a run
The run with its totals: stored when it ended, computed from its items while it runs.
| Parameter | In | Type | Required | Notes |
|---|---|---|---|---|
project_id |
path | string (uuid) | yes | |
experiment_id |
path | string (uuid) | yes | |
run_id |
path | string (uuid) | yes |
| Status | Description | Body |
|---|---|---|
| 200 | Successful Response | application/json: ExperimentRunOut |
| 422 | Validation Error | application/json: HTTPValidationError |
GET /api/v1/projects/{project_id}/experiments/{experiment_id}/runs/{run_id}/items
Section titled “GET /api/v1/projects/{project_id}/experiments/{experiment_id}/runs/{run_id}/items”List a run’s items
In dataset item order.
| Parameter | In | Type | Required | Notes |
|---|---|---|---|---|
project_id |
path | string (uuid) | yes | |
experiment_id |
path | string (uuid) | yes | |
run_id |
path | string (uuid) | yes | |
limit |
query | integer | no | min 1, max 200 Default 50. |
cursor |
query | string or null | no | max length 512 |
| Status | Description | Body |
|---|---|---|
| 200 | Successful Response | application/json: Page_ExperimentItemOut_ |
| 422 | Validation Error | application/json: HTTPValidationError |