Experiments#

class ExperimentsClient(*, sdk_config: SDKConfiguration, generated_client: ApiClient)[source]#

Bases: object

Client for managing experiments including creation, execution, and result tracking.

This class is primarily intended for internal use within the SDK. Users are highly encouraged to access resource-specific functionality via arize.ArizeClient.

The experiments client is a thin wrapper around the generated REST API client, using the shared generated API client owned by arize.config.SDKConfiguration.

Parameters:

sdk_config (SDKConfiguration) – Resolved SDK configuration.
generated_client (ApiClient) – Shared generated API client instance.

list(*, dataset: str | None = None, space: str | None = None, limit: int = 100, cursor: str | None = None) → ExperimentsList200Response[source]#

List experiments the user has access to.

To filter experiments by the dataset they were run on, provide dataset.

Parameters:

dataset (str | None) – Optional dataset name or ID to filter experiments.
space (str | None) – Optional space name or ID used to resolve dataset by name.
limit (int) – Maximum number of experiments to return. The server enforces an upper bound.
cursor (str | None) – Opaque pagination cursor returned from a previous response.

Returns:

A response object with the experiments and pagination information.

Raises:

ApiException – If the REST API returns an error response (e.g. 401/403/429).

Return type:

ExperimentsList200Response

create(*, name: str, dataset: str, space: str | None = None, experiment_runs: builtins.list[dict[str, object]] | pd.DataFrame, task_fields: ExperimentTaskFieldNames, evaluator_columns: dict[str, EvaluationResultFieldNames] | None = None, force_http: bool = False) → Experiment[source]#

Create an experiment with one or more experiment runs.

Experiments are composed of runs. Each run must include:

example_id: ID of an existing example in the dataset/version
output: Model/task output for the matching example

You may include any additional user-defined fields per run (e.g. model, latency_ms, temperature, prompt, tool_calls, etc.) that can be used for analysis or filtering.

This method transforms the input runs into the server’s expected experiment format using task_fields and optional evaluator_columns.

Transport selection:

If the payload is below the configured REST payload threshold (or force_http=True), this method uploads via REST.
Otherwise, it attempts a more efficient upload path via gRPC + Flight.

Parameters:

name (str) – Experiment name. Must be unique within the target dataset.
dataset (str) – Dataset name or ID to attach the experiment to.
space (str | None) – Optional space name or ID used to resolve dataset by name.
experiment_runs (builtins.list[dict[str, object]] | pd.DataFrame) – Experiment runs either as: - a list of JSON-like dicts, or - a pandas.DataFrame.
task_fields (ExperimentTaskFieldNames) – Mapping that identifies the columns/fields containing the task results (e.g. example_id, output fields).
evaluator_columns (dict[str, EvaluationResultFieldNames] | None) – Optional mapping describing evaluator result columns.
force_http (bool) – If True, force REST upload even if the payload exceeds the configured REST payload threshold.

Returns:

The created experiment object.

Raises:

TypeError – If experiment_runs is not a list of dicts or a DataFrame.
RuntimeError – If the Flight upload path is selected and the Flight request fails.
ApiException – If the REST API returns an error response (e.g. 400/401/403/409/429).

Return type:

Experiment

get(*, experiment: str, dataset: str | None = None, space: str | None = None) → Experiment[source]#

Get an experiment by name or ID.

The response does not include the experiment’s runs. Use list_runs() to retrieve runs for an experiment.

Parameters:

experiment (str) – Experiment name or ID to retrieve.
dataset (str | None) – Optional dataset name or ID used to resolve experiment by name.
space (str | None) – Optional space name or ID used to resolve dataset by name.

Returns:

The experiment object.

Raises:

ApiException – If the REST API returns an error response (e.g. 401/403/404/429).

Return type:

Experiment

delete(*, experiment: str, dataset: str | None = None, space: str | None = None) → None[source]#

Delete an experiment by name or ID.

This operation is irreversible.

Parameters:

experiment (str) – Experiment name or ID to delete.
dataset (str | None) – Optional dataset name or ID used to resolve experiment by name.
space (str | None) – Optional space name or ID used to resolve dataset by name.

Returns:

This method returns None on success (common empty 204 response).

Raises:

ApiException – If the REST API returns an error response (e.g. 401/403/404/429).

Return type:

None

list_runs(*, experiment: str, dataset: str | None = None, space: str | None = None, limit: int = 100, all: bool = False) → ExperimentsRunsList200Response[source]#

List runs for an experiment.

Runs are returned in insertion order.

Pagination notes:

The response includes pagination for forward compatibility.
Cursor pagination may not be fully implemented by the server yet.
If all=True, this method retrieves all runs via the Flight path and returns them in a single response with has_more=False.

Parameters:

experiment (str) – Experiment name or ID to list runs for.
dataset (str | None) – Optional dataset name or ID used to resolve experiment by name.
space (str | None) – Optional space name or ID used to resolve dataset by name.
limit (int) – Maximum number of runs to return when all=False. The server enforces an upper bound.
all (bool) – If True, fetch all runs (ignores limit) via Flight and return a single response.

Returns:

A response object containing experiment_runs and pagination metadata.

Raises:

RuntimeError – If the Flight request fails or returns no response when all=True.
ApiException – If the REST API returns an error response when all=False (e.g. 401/403/404/429).

Return type:

ExperimentsRunsList200Response

annotate_runs(*, experiment: str, dataset: str | None = None, space: str | None = None, annotations: builtins.list[models.AnnotateRecordInput]) → AnnotationBatchResult[source]#

Write human annotations to a batch of runs in an experiment.

Annotations are upserted by annotation config name for each run. Submitting the same annotation config name for the same run overwrites the previous value. Retrying on network failure will not create duplicates.

Up to 500 runs may be annotated per request.

Parameters:

experiment (str) – Experiment ID or name.
dataset (str | None) – Optional dataset ID or name used to resolve experiment by name.
space (str | None) – Optional space ID or name used to resolve dataset by name.
annotations (builtins.list[models.AnnotateRecordInput]) – A list of AnnotateRecordInput items. Each item must include a record_id (the experiment run ID) and values (a list of AnnotationInput items with name, and optionally score, label, or text).

Returns:

An AnnotationBatchResult containing per-record results.

Raises:

ApiException – If the REST API returns an error response (e.g. 400/401/404/429).

Return type:

AnnotationBatchResult

run(*, name: str, dataset: str, space: str | None = None, task: ExperimentTask, evaluators: Evaluators | None = None, dry_run: bool = False, dry_run_count: int = 10, concurrency: int = 3, set_global_tracer_provider: bool = False, exit_on_error: bool = False, timeout: int = 120, force_http: bool = False) → tuple[Experiment | None, pd.DataFrame][source]#

Run an experiment on a dataset and optionally upload results.

This method executes a task against dataset examples, optionally evaluates outputs, and (when dry_run=False) uploads results to Arize.

High-level flow:

Resolve the dataset and space_id.
Download dataset examples (or load from cache if enabled).
Run the task and evaluators with configurable concurrency.
If not a dry run, upload experiment runs and return the created
experiment plus the results dataframe.

Notes

If dry_run=True, no data is uploaded and the returned experiment is None.
When enable_caching=True, dataset examples may be cached and reused.
When force_http=True, all gRPC/Flight calls are bypassed and pure REST is used instead. Given that the Flight protocol allows to handle more rows, the number of rows to run an experiment on is limited to 500 when using force_http=True.

Parameters:

name (str) – Experiment name.
dataset (str) – Dataset name or ID to run the experiment against.
space (str | None) – Optional space name or ID used to resolve dataset by name.
task (ExperimentTask) – The task to execute for each dataset example.
evaluators (Evaluators | None) – Optional evaluators used to score outputs.
dry_run (bool) – If True, do not upload results to Arize.
dry_run_count (int) – Number of dataset rows to use when dry_run=True.
concurrency (int) – Number of concurrent tasks to run.
set_global_tracer_provider (bool) – If True, sets the global OpenTelemetry tracer provider for the experiment run.
exit_on_error (bool) – If True, stop on the first error encountered during execution.
timeout (int) – The timeout in seconds for each task execution. Defaults to 120.
force_http (bool) – If True, bypass gRPC/Flight and use REST only. This is not recommended for large datasets since it limits the number of rows to 500 and may be slower than the Flight path.

Returns:

If dry_run=True, returns (None, results_df). If dry_run=False, returns (experiment, results_df).

Raises:

RuntimeError – If Flight operations (init/download/upload) fail or return no response.
pa.ArrowInvalid – If converting results to Arrow fails.
Exception – For unexpected errors during Arrow conversion.

Return type:

tuple[Experiment | None, pd.DataFrame]

Evaluators#

class Evaluator(*args: object, **kwargs: object)[source]#

Bases: ABC

A helper super class to guide the implementation of an Evaluator object.

Subclasses must implement either the evaluate or async_evaluate method. Implementing both methods is recommended, but not required.

This Class is intended to be subclassed, and should not be instantiated directly.

Create a new evaluator instance, preventing direct instantiation of abstract class.

Parameters:

args (object)
kwargs (object)

Return type:

Evaluator

property name: str#: Return the name of this evaluator.

property kind: str#: Return the kind of this evaluator (CODE or LLM).

evaluate(*, dataset_row: Mapping[str, JSONSerializable] | None = None, input: ExampleInput = MappingProxyType({}), output: TaskOutput | None = None, experiment_output: TaskOutput | None = None, dataset_output: ExampleOutput = MappingProxyType({}), metadata: ExampleMetadata = MappingProxyType({}), **kwargs: object) → EvaluationResult[source]#

Evaluate the given inputs and produce an evaluation result.

This method should be implemented by subclasses to perform the actual evaluation logic. It is recommended to implement both this synchronous method and the asynchronous async_evaluate method, but it is not required.

Parameters:

dataset_row (Mapping[str, JSONSerializable] | None) – A row from the dataset.
input (ExampleInput) – The input provided for evaluation.
output (TaskOutput | None) – The output produced by the task.
experiment_output (TaskOutput | None) – The experiment output for comparison.
dataset_output (ExampleOutput) – The expected output from the dataset.
metadata (ExampleMetadata) – Metadata associated with the example.
**kwargs (Any) – Additional keyword arguments.

Raises:

NotImplementedError – If the method is not implemented by the subclass.

Return type:

EvaluationResult

async async_evaluate(*, dataset_row: Mapping[str, JSONSerializable] | None = None, input: ExampleInput = MappingProxyType({}), output: TaskOutput | None = None, experiment_output: TaskOutput | None = None, dataset_output: ExampleOutput = MappingProxyType({}), metadata: ExampleMetadata = MappingProxyType({}), **kwargs: object) → EvaluationResult[source]#

Asynchronously evaluate the given inputs and produce an evaluation result.

This method should be implemented by subclasses to perform the actual evaluation logic. It is recommended to implement both this asynchronous method and the synchronous evaluate method, but it is not required.

Parameters:

dataset_row (Mapping[str, JSONSerializable] | None) – A row from the dataset.
input (ExampleInput) – The input provided for evaluation.
output (TaskOutput | None) – The output produced by the task.
experiment_output (TaskOutput | None) – The experiment output for comparison.
dataset_output (ExampleOutput) – The expected output from the dataset.
metadata (ExampleMetadata) – Metadata associated with the example.
**kwargs (Any) – Additional keyword arguments.

Returns:

The result of the evaluation.

Return type:

EvaluationResult

Raises:

NotImplementedError – If the method is not implemented by the subclass.

class CodeEvaluator(*args: object, **kwargs: object)[source]#

Bases: Evaluator, ABC

A convenience super class for defining code evaluators.

There are functionally no differences between this class and the Evaluator class, except that this class has a default _kind attribute for AnnotatorKind.CODE. This class is intended to be subclassed, and should not be instantiated directly.

Create a new code evaluator instance, preventing direct instantiation of abstract class.

Parameters:

args (object)
kwargs (object)

Return type:

CodeEvaluator

class LLMEvaluator(*args: object, **kwargs: object)[source]#

Bases: Evaluator, ABC

A convenience super class for defining LLM evaluators.

There are functionally no differences between this class and the Evaluator class, except that this class has a default _kind attribute for AnnotatorKind.LLM. This class is intended to be subclassed, and should not be instantiated directly.

Create a new LLM evaluator instance, preventing direct instantiation of abstract class.

Parameters:

args (object)
kwargs (object)

Return type:

LLMEvaluator

Response Types#

class AnnotatorKind(value, names=_not_given, *values, module=None, qualname=None, type=None, start=1, boundary=None)[source]#

Bases: Enum

Enum representing the type of annotator used for evaluation.

CODE = 'CODE'#

LLM = 'LLM'#

class EvaluationResult(score: float | None = None, label: str | None = None, explanation: str | None = None, metadata: Mapping[str, JSONSerializable] = <factory>)[source]#

Bases: object

Represents the result of an evaluation.

Parameters:

score (float | None) – The score of the evaluation.
label (str | None) – The label of the evaluation.
explanation (str | None) – The explanation of the evaluation.
metadata (Mapping[str, JSONSerializable]) – Additional metadata for the evaluation.

score: float | None = None#

label: str | None = None#

explanation: str | None = None#

metadata: Mapping[str, JSONSerializable]#

classmethod from_dict(obj: Mapping[str, object] | None) → EvaluationResult | None[source]#

Create an EvaluationResult instance from a dictionary.

Parameters:: obj (Mapping[str, object] | None)
Return type:: EvaluationResult | None

Bases: object

Column names for mapping evaluation results in a pandas.DataFrame.

Parameters:

score (str | None) – Optional name of column containing evaluation scores
label (str | None) – Optional name of column containing evaluation labels
explanation (str | None) – Optional name of column containing evaluation explanations
metadata (dict[str, str | None] | None) – Optional mapping of metadata keys to column names. If a column name is None or empty string, the metadata key will be used as the column name.

Examples

>>> # Basic usage with score and label columns
>>> EvaluationResultColumnNames(
...     score="quality.score", label="quality.label"
... )

>>> # Using metadata with same key and column name
>>> EvaluationResultColumnNames(
...     score="quality.score",
...     metadata={
...         "version": None
...     },  # Will look for column named "version"
... )

>>> # Using metadata with different key and column name
>>> EvaluationResultColumnNames(
...     score="quality.score",
...     metadata={
...         # Will look for "column_in_my_df.version" column and ingest as
...         # "eval.{EvaluatorName}.meatadata.model_version"
...         "model_version": "column_in_my_df.version",
...         # Will look for "column_in_my_df.ts" column and ingest as
...         # "eval.{EvaluatorName}.metadata.timestamp"
...         "timestamp": "column_in_my_df.ts",
...     },
... )

Raises:

ValueError – If neither score nor label column names are specified

Parameters:

score (str | None)
label (str | None)
explanation (str | None)
metadata (dict[str, str | None] | None)

score: str | None = None#

label: str | None = None#

explanation: str | None = None#

metadata: dict[str, str | None] | None = None#

Bases: object

Represents an example in an experiment dataset.

Parameters:

id (str) – The unique identifier for the example.
updated_at (datetime) – The timestamp when the example was last updated.
input (Mapping[str, dict[str, dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | list[dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | str | int | float | bool | None]) – The input data for the example.
output (Mapping[str, dict[str, dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | list[dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | str | int | float | bool | None]) – The output data for the example.
metadata (Mapping[str, dict[str, dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | list[dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | str | int | float | bool | None]) – Additional metadata for the example.
dataset_row (Mapping[str, dict[str, dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | list[dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | str | int | float | bool | None]) – The original dataset row containing the example data.

id: str#

updated_at: datetime#

input: Mapping[str, dict[str, dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | list[dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | str | int | float | bool | None]#

output: Mapping[str, dict[str, dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | list[dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | str | int | float | bool | None]#

metadata: Mapping[str, dict[str, dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | list[dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | str | int | float | bool | None]#

dataset_row: Mapping[str, dict[str, dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | list[dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None] | str | int | float | bool | None]#

classmethod from_dict(obj: Mapping[str, object]) → Example[source]#

Create an Example instance from a dictionary.

Parameters:: obj (Mapping[str, object])
Return type:: Example

class Experiment(*, id: Annotated[str, Strict(strict=True)], name: Annotated[str, Strict(strict=True)], dataset_id: Annotated[str, Strict(strict=True)], dataset_version_id: Annotated[str, Strict(strict=True)], created_at: datetime, updated_at: datetime, experiment_traces_project_id: Annotated[str, Strict(strict=True)] | None = None)[source]#

Bases: BaseModel

Experiments combine a dataset (example inputs/expected outputs), a task (the function that produces model outputs), and one or more evaluators (code or LLM judges) to measure performance. Each run is stored independently so you can compare runs, track progress, and validate improvements over time. See the full definition on the Experiments page. Use an experiment to run tasks on a dataset, attach evaluators to score outputs, and compare runs to confirm improvements.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:

id (Annotated[str, Strict(strict=True)])
name (Annotated[str, Strict(strict=True)])
dataset_id (Annotated[str, Strict(strict=True)])
dataset_version_id (Annotated[str, Strict(strict=True)])
created_at (datetime)
updated_at (datetime)
experiment_traces_project_id (Annotated[str, Strict(strict=True)] | None)

id: StrictStr#

name: StrictStr#

dataset_id: StrictStr#

dataset_version_id: StrictStr#

created_at: datetime#

updated_at: datetime#

experiment_traces_project_id: StrictStr | None#

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

to_str() → str[source]#

Returns the string representation of the model using alias

Return type:: str

to_json() → str[source]#

Returns the JSON representation of the model using alias

Return type:: str

classmethod from_json(json_str: str) → Self | None[source]#

Create an instance of Experiment from a JSON string

Parameters:: json_str (str)
Return type:: Self | None

to_dict() → Dict[str, Any][source]#

Return the dictionary representation of the model using alias.

This has the following differences from calling pydantic’s self.model_dump(by_alias=True):

None is only added to the output dict for nullable fields that were set at model initialization. Other fields with value None are ignored.

Return type:: Dict[str, Any]

classmethod from_dict(obj: Dict[str, Any] | None) → Self | None[source]#

Create an instance of Experiment from a dict

Parameters:: obj (Dict[str, Any] | None)
Return type:: Self | None

class ExperimentEvaluationRun(experiment_run_id: str, start_time: ~datetime.datetime, end_time: ~datetime.datetime, name: str, annotator_kind: str, error: str | None = None, result: ~arize.experiments.evaluators.types.EvaluationResult | None = None, id: str = <factory>, trace_id: str | None = None)[source]#

Bases: object

Represents a single evaluation run of an experiment.

Parameters:

experiment_run_id (str) – The unique identifier for the experiment run.
start_time (datetime) – The start time of the evaluation run.
end_time (datetime) – The end time of the evaluation run.
name (str) – The name of the evaluation run.
annotator_kind (str) – The kind of annotator used in the evaluation run.
error (str | None) – The error message if the evaluation run failed.
result (EvaluationResult | None) – The result of the evaluation run.
id (str) – The unique identifier for the evaluation run.
trace_id (TraceId | None) – The trace identifier for the evaluation run.

experiment_run_id: str#

start_time: datetime#

end_time: datetime#

name: str#

annotator_kind: str#

error: str | None = None#

result: EvaluationResult | None = None#

id: str#

trace_id: str | None = None#

classmethod from_dict(obj: Mapping[str, object]) → ExperimentEvaluationRun[source]#

Create an ExperimentEvaluationRun instance from a dictionary.

Parameters:: obj (Mapping[str, object])
Return type:: ExperimentEvaluationRun

Bases: object

Represents a single run of an experiment.

Parameters:

start_time (datetime) – The start time of the experiment run.
end_time (datetime) – The end time of the experiment run.
experiment_id (str) – The unique identifier for the experiment.
dataset_example_id (str) – The unique identifier for the dataset example.
repetition_number (int) – The repetition number of the experiment run.
output (dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None) – The output of the experiment run.
error (str | None) – The error message if the experiment run failed.
id (str) – The unique identifier for the experiment run.
trace_id (str | None) – The trace identifier for the experiment run.

start_time: datetime#

end_time: datetime#

experiment_id: str#

dataset_example_id: str#

repetition_number: int#

output: dict[str, JSONSerializable] | list[JSONSerializable] | str | int | float | bool | None#

error: str | None = None#

id: str#

trace_id: str | None = None#

classmethod from_dict(obj: Mapping[str, object]) → ExperimentRun[source]#

Create an ExperimentRun instance from a dictionary.

Parameters:: obj (Mapping[str, object])
Return type:: ExperimentRun

class ExperimentTaskFieldNames(example_id: str, output: str)[source]#

Bases: object

Column names for mapping experiment task results in a pandas.DataFrame.

Parameters:

example_id (str) – Name of column containing example IDs. The ID values must match the id of the dataset rows.
output (str) – Name of column containing task results

example_id: str#

output: str#

class ExperimentsList200Response(*, experiments: List[Experiment], pagination: PaginationMetadata)[source]#

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:

experiments (List[Experiment])
pagination (PaginationMetadata)

experiments: List[Experiment]#

pagination: PaginationMetadata#

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

to_str() → str[source]#

Returns the string representation of the model using alias

Return type:: str

to_json() → str[source]#

Returns the JSON representation of the model using alias

Return type:: str

classmethod from_json(json_str: str) → Self | None[source]#

Create an instance of ExperimentsList200Response from a JSON string

Parameters:: json_str (str)
Return type:: Self | None

to_dict() → Dict[str, Any][source]#

Return the dictionary representation of the model using alias.

This has the following differences from calling pydantic’s self.model_dump(by_alias=True):

None is only added to the output dict for nullable fields that were set at model initialization. Other fields with value None are ignored.

Return type:: Dict[str, Any]

classmethod from_dict(obj: Dict[str, Any] | None) → Self | None[source]#

Create an instance of ExperimentsList200Response from a dict

Parameters:: obj (Dict[str, Any] | None)
Return type:: Self | None

to_df(by_alias: bool = False, exclude_none: str | bool = True, json_normalize: bool = False, convert_dtypes: bool = True, expand_field: str = 'additional_properties', expand_prefix: str = '') → pd.DataFrame#

Convert a list of objects to a pandas.DataFrame.

Behavior:

If an item is a Pydantic v2 model, use .model_dump(by_alias=…).
If an item is a mapping (dict-like), use it as-is.
Otherwise, raise a ValueError (unsupported row type).

Parameters:

self (object) – The object instance containing the field to convert.
by_alias (bool) – Use field aliases when dumping Pydantic models.
exclude_none (str | bool) – Control None/NaN column dropping. - False: keep Nones as-is - “all”: drop columns where all values are None/NaN - “any”: drop columns where any value is None/NaN - True: alias for “all”
json_normalize (bool) – If True, flatten nested dicts via pandas.json_normalize.
convert_dtypes (bool) – If True, call DataFrame.convert_dtypes() at the end.
expand_field (str) – If set, look for this field in each row and
columns. (expand its keys into top-level)
expand_prefix (str) – If set, prefix expanded column names with this string.

Returns:

The converted DataFrame.

Return type:

pandas.DataFrame

class ExperimentsRunsList200Response(*, experiment_runs: List[ExperimentRun], pagination: PaginationMetadata)[source]#

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:

experiment_runs (List[ExperimentRun])
pagination (PaginationMetadata)

experiment_runs: List[ExperimentRun]#

pagination: PaginationMetadata#

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

to_str() → str[source]#

Returns the string representation of the model using alias

Return type:: str

to_json() → str[source]#

Returns the JSON representation of the model using alias

Return type:: str

classmethod from_json(json_str: str) → Self | None[source]#

Create an instance of ExperimentsRunsList200Response from a JSON string

Parameters:: json_str (str)
Return type:: Self | None

to_dict() → Dict[str, Any][source]#

Return the dictionary representation of the model using alias.

This has the following differences from calling pydantic’s self.model_dump(by_alias=True):

None is only added to the output dict for nullable fields that were set at model initialization. Other fields with value None are ignored.

Return type:: Dict[str, Any]

classmethod from_dict(obj: Dict[str, Any] | None) → Self | None[source]#

Create an instance of ExperimentsRunsList200Response from a dict

Parameters:: obj (Dict[str, Any] | None)
Return type:: Self | None

to_df(by_alias: bool = False, exclude_none: str | bool = True, json_normalize: bool = False, convert_dtypes: bool = True, expand_field: str = 'additional_properties', expand_prefix: str = '') → pd.DataFrame#

Convert a list of objects to a pandas.DataFrame.

Behavior:

If an item is a Pydantic v2 model, use .model_dump(by_alias=…).
If an item is a mapping (dict-like), use it as-is.
Otherwise, raise a ValueError (unsupported row type).

Parameters:

self (object) – The object instance containing the field to convert.
by_alias (bool) – Use field aliases when dumping Pydantic models.
exclude_none (str | bool) – Control None/NaN column dropping. - False: keep Nones as-is - “all”: drop columns where all values are None/NaN - “any”: drop columns where any value is None/NaN - True: alias for “all”
json_normalize (bool) – If True, flatten nested dicts via pandas.json_normalize.
convert_dtypes (bool) – If True, call DataFrame.convert_dtypes() at the end.
expand_field (str) – If set, look for this field in each row and
columns. (expand its keys into top-level)
expand_prefix (str) – If set, prefix expanded column names with this string.

Returns:

The converted DataFrame.

Return type:

pandas.DataFrame

class AnnotationBatchResult(*, results: List[AnnotateRecordResult])[source]#

Bases: BaseModel

Result of a batch annotation operation. Contains one result entry per annotated record.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: results (List[AnnotateRecordResult])

results: List[AnnotateRecordResult]#

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

to_str() → str[source]#

Returns the string representation of the model using alias

Return type:: str

to_json() → str[source]#

Returns the JSON representation of the model using alias

Return type:: str

classmethod from_json(json_str: str) → Self | None[source]#

Create an instance of AnnotationBatchResult from a JSON string

Parameters:: json_str (str)
Return type:: Self | None

to_dict() → Dict[str, Any][source]#

Return the dictionary representation of the model using alias.

This has the following differences from calling pydantic’s self.model_dump(by_alias=True):

None is only added to the output dict for nullable fields that were set at model initialization. Other fields with value None are ignored.

Return type:: Dict[str, Any]

classmethod from_dict(obj: Dict[str, Any] | None) → Self | None[source]#

Create an instance of AnnotationBatchResult from a dict

Parameters:: obj (Dict[str, Any] | None)
Return type:: Self | None

class AnnotateRecordInput(*, record_id: Annotated[str, Strict(strict=True)], values: Annotated[List[AnnotationInput], MinLen(min_length=1)])[source]#

Bases: BaseModel

A single record to annotate in a batch, identified by its record ID.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:

record_id (Annotated[str, Strict(strict=True)])
values (Annotated[List[AnnotationInput], MinLen(min_length=1)])

record_id: StrictStr#

values: Annotated[List[AnnotationInput], Field(min_length=1)]#

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

to_str() → str[source]#

Returns the string representation of the model using alias

Return type:: str

to_json() → str[source]#

Returns the JSON representation of the model using alias

Return type:: str

classmethod from_json(json_str: str) → Self | None[source]#

Create an instance of AnnotateRecordInput from a JSON string

Parameters:: json_str (str)
Return type:: Self | None

to_dict() → Dict[str, Any][source]#

Return the dictionary representation of the model using alias.

This has the following differences from calling pydantic’s self.model_dump(by_alias=True):

None is only added to the output dict for nullable fields that were set at model initialization. Other fields with value None are ignored.

Return type:: Dict[str, Any]

classmethod from_dict(obj: Dict[str, Any] | None) → Self | None[source]#

Create an instance of AnnotateRecordInput from a dict

Parameters:: obj (Dict[str, Any] | None)
Return type:: Self | None

class AnnotationInput(*, name: Annotated[str, Strict(strict=True)], score: Annotated[float, Strict(strict=True)] | Annotated[int, Strict(strict=True)] | None = None, label: Annotated[str, Strict(strict=True)] | None = None, text: Annotated[str, Strict(strict=True)] | None = None)[source]#

Bases: BaseModel

An annotation value to set on a record, identified by its annotation config name. Omitting a field leaves the existing value unchanged.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:

name (Annotated[str, Strict(strict=True)])
score (Annotated[float, Strict(strict=True)] | Annotated[int, Strict(strict=True)] | None)
label (Annotated[str, Strict(strict=True)] | None)
text (Annotated[str, Strict(strict=True)] | None)

name: StrictStr#

score: StrictFloat | StrictInt | None#

label: StrictStr | None#

text: StrictStr | None#

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

to_str() → str[source]#

Returns the string representation of the model using alias

Return type:: str

to_json() → str[source]#

Returns the JSON representation of the model using alias

Return type:: str

classmethod from_json(json_str: str) → Self | None[source]#

Create an instance of AnnotationInput from a JSON string

Parameters:: json_str (str)
Return type:: Self | None

to_dict() → Dict[str, Any][source]#

Return the dictionary representation of the model using alias.

This has the following differences from calling pydantic’s self.model_dump(by_alias=True):

None is only added to the output dict for nullable fields that were set at model initialization. Other fields with value None are ignored.

Return type:: Dict[str, Any]

classmethod from_dict(obj: Dict[str, Any] | None) → Self | None[source]#

Create an instance of AnnotationInput from a dict

Parameters:: obj (Dict[str, Any] | None)
Return type:: Self | None