Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .secrets.baseline
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"files": "requirements.txt|^.secrets.baseline$",
"lines": null
},
"generated_at": "2026-07-10T10:38:43Z",
"generated_at": "2026-07-30T10:09:44Z",
"plugins_used": [
{
"name": "AWSKeyDetector"
Expand Down Expand Up @@ -122,7 +122,7 @@
"hashed_secret": "2db6d21d365f544f7ca3bcfb443ac96898a7a069",
"is_secret": false,
"is_verified": false,
"line_number": 192,
"line_number": 177,
"type": "Secret Keyword",
"verified_result": null
}
Expand Down
96 changes: 46 additions & 50 deletions docs/developer-guide/creating-actuator-classes.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
<!-- markdownlint-disable first-line-h1 -->

>[!INFO]
>
> A [complete template actuator](https://github.com/IBM/ado/tree/main/plugins/actuators/example_actuator)
> is available.
> This example actuator is functional out-of-the-box
> and can be used as the basis to create new actuators.
!!! tip

A [complete template actuator](https://github.com/IBM/ado/tree/main/plugins/actuators/example_actuator)
is available.
This example actuator is functional out-of-the-box
and can be used as the basis to create new actuators.

[Custom experiments](creating-custom-experiments.md) cover many use cases for
extending `ado` with new experiments. However, sometimes you need more control
Expand All @@ -16,9 +16,8 @@ For such situations developers can write their own
[actuators](../concepts/actuators.md). Actuators allow you to control and
customize the entire experiment submission process giving great flexibility and
power. You can also expose customization options to users via
[`actuatorconfigurations`](#enabling-custom-configuration-of-an-actuator).
Like custom experiments actuator are supplied as plugin **python
packages**.
[`actuatorconfigurations`](#enabling-custom-configuration-of-an-actuator). Like
custom experiments actuator are supplied as plugin **python packages**.

This page gives an overview of how to get started creating your own actuator.
It's not intended to be comprehensive. After reading this page the best resource
Expand All @@ -42,8 +41,8 @@ class that implements a specific interface.
actuator
- implements the `catalog()` method
- _either_
- simple case: override `_experiment_implementations()`
- complex case: overrides `_get_request_executor`
- simple case: override `_experiment_implementations()`
- complex case: overrides `_get_request_executor`

The simple case is for when your actuators experiments are independent of each
other i.e. executing one experiment does not care about other experiments. The
Expand Down Expand Up @@ -73,18 +72,24 @@ function for each `Experiment` entry your `ExperimentCatalog`. The
`_experiment_implementations()` method then returns a dict that maps each
experiment identifier to the corresponding function e.g.

<!-- markdownlint-disable code-block-style -->

```python
def _experiment_implementations(self) -> dict[str, Callable[..., dict[str, Any]]]:

return {"myexperiment": my_experiment_fn()}
```

<!-- markdownlint-enable code-block-style -->

The parameter names of the function must be the same as the input property
identifiers of the `Experiment`. The output of the function must be a dict that
maps the target property identifiers of the Experiment to their measured values.

For example, for an Experiment instance like

<!-- markdownlint-disable code-block-style -->

```yaml
# it's properties which match what is defined here
peptide_mineralization:
Expand All @@ -104,15 +109,21 @@ peptide_mineralization:
- identifier: adsorption_plateau_value
```

<!-- markdownlint-enable code-block-style -->

The function would look like

<!-- markdownlint-disable code-block-style -->

```python
def peptide_mineralization_fn(peptide_identifier, peptide_concentration):

...
return {"adsoprtion_timeseries": timeseries, "adsorption_plateau_value": plateau}
```

<!-- markdownlint-enable code-block-style -->

### Complex case: Experiments with shared state

If your experiments require access to shared state e.g. a queue object, an
Expand All @@ -123,6 +134,8 @@ This method that takes a `MeasurementRequest` instance that describes the
experiment to run. Note, the `use_ray` parameter is used by default
implementation and can be safely ignored when overridden.

<!-- markdownlint-disable code-block-style -->

```python
def _get_request_executor(
self,
Expand All @@ -131,6 +144,8 @@ implementation and can be safely ignored when overridden.
) -> Callable[[], MeasurementRequest]:
```

<!-- markdownlint-enable code-block-style -->

The `get_request_executor` method must return a zero-argument `Callable` that
executes the requested experiment and returns a completed `MeasurementRequest`
(measurements and status set). The function must be picklable.
Expand All @@ -152,13 +167,17 @@ method

e.g.

<!-- markdownlint-disable code-block-style -->

```python
result = actuator.execute(entities,
experiment_reference,
requesterid: "script",
requestIndex: 0,)
```

<!-- markdownlint-enable code-block-style -->

where `entities` is a list of one or more `Entity` instances representing the
points you want to measure, and `experiment_reference`, is an
`ExperimentReference` instance describing the experiment to execute.
Expand All @@ -176,26 +195,7 @@ to the following:
<!-- markdownlint-disable code-block-style -->

```toml
[project]
name = "robotic_lab" # Change to your preferred name, along with the actual package
description = "A template for creating an actuator" # Change to describing your actuator
dependencies = [
"ado-core"
]
dynamic = ["version"]

[project.entry-points."ado.actuators"]
robotic_lab = "robotic_lab_actuator.actuator:RoboticLab"

[build-system]
requires = ["hatchling", "uv-dynamic-versioning>=0.7.0"]
build-backend = "hatchling.build"

[tool.hatch.version]
source = "uv-dynamic-versioning"

[tool.hatch.build.targets.wheel]
packages = ["src/robotic_lab_actuator"]
{% include "../../plugins/actuators/example_actuator/pyproject.toml" %}
```

<!-- markdownlint-enable code-block-style -->
Expand Down Expand Up @@ -726,10 +726,9 @@ at the end of execution

Actuator developers can provide rich, real-time progress output to users running
experiments, using utilities available in
`ado.modules.operators.console_output.py`. This is critical for
long-running operations (such as deployment, environment setup, or
benchmarking), and helps users visually associate progress with specific
requests.
`ado.modules.operators.console_output.py`. This is critical for long-running
operations (such as deployment, environment setup, or benchmarking), and helps
users visually associate progress with specific requests.

### How progress signaling works

Expand All @@ -745,12 +744,11 @@ using provided Rich message helpers:
You should send these messages to the `RichConsoleQueue` actor and update or
stop them when state changes.

> [!INFO]
> Use the `request id` of the MeasurementRequest you're operating on
> as the message `id` (and include it in the message `label`).
> This allows your actuator to support progress for multiple experiments
> running concurrently, and the UI will clearly indicate which progress
> output is tied to which experiment request.
> [!INFO] Use the `request id` of the MeasurementRequest you're operating on as
> the message `id` (and include it in the message `label`). This allows your
> actuator to support progress for multiple experiments running concurrently,
> and the UI will clearly indicate which progress output is tied to which
> experiment request.

### Example usage

Expand Down Expand Up @@ -847,12 +845,10 @@ functions and methods:

- `Experiment.propertyValuesFromEntity` - Get the input values for the
experiment based on the entity and the experiment definition
- `ado.utilities.support.observed_property_values_from_dict` - Extract
the values related to an experiment from a dictionary of measurements and
convert to PropertyValues
- `ado.utilities.support.create_measurement_result` - Create
measurement result
- `ado.utilities.support.compute_measurement_status` - Compute
execution status
- `ado.utilities.async_task_runner.AsyncTaskRunner` - wait for the
completion of an async function and get execution result
- `ado.utilities.support.observed_property_values_from_dict` - Extract the
values related to an experiment from a dictionary of measurements and convert
to PropertyValues
- `ado.utilities.support.create_measurement_result` - Create measurement result
- `ado.utilities.support.compute_measurement_status` - Compute execution status
- `ado.utilities.async_task_runner.AsyncTaskRunner` - wait for the completion of
an async function and get execution result
23 changes: 16 additions & 7 deletions docs/developer-guide/creating-operators.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<!-- markdownlint-disable code-block-style -->
<!-- markdownlint-disable first-line-h1 -->

!!! info end
!!! tip end

A complete example operator is provided
[here](https://github.com/IBM/ado/tree/main/plugins/operators/profile_space).
Expand Down Expand Up @@ -48,11 +48,20 @@ from ado.modules.operators.collections import (


@characterize_operation(
name="my_operator", # The name of your operator.
description="Example operator", # What this operator does
configuration_model=MyOperatorOptions, # A pydantic model that describes your operators input parameters
example_configuration=MyOperatorOptions.example_configuration(), # An example of your operators input parameters
version="1.0", # Version of the operator
name="detect_anomalous_series",
description="""
This operation checks if the behaviour of an observed property versus
an independent (constitutive) property is as expected.

The behaviours that can be checked are: True (all values are 1/True); Constant (all values are the same);
Monotonically Increasing; or Monotonically Decreasing.

The entities in the discovery space can be divided into groups based on a user supplied set of constitutive
properties, other than the selected independent property.
""",
configuration_model=DetectAnomalousSeries,
example_configuration=DetectAnomalousSeries.example_configuration(),
version="1.0.5",
)
def detect_anomalous_series(
discoverySpace: DiscoverySpace,
Expand Down Expand Up @@ -140,7 +149,7 @@ the previous section with the relevant fields called out:
description="Example operator",
configuration_model=MyOperatorOptions, # <- A pydantic model that describes your operators input parameters
example_configuration=MyOperatorOptions(), # <- An example of your operators input parameters
version="1.0",
version="1.0.0",
)
```

Expand Down
24 changes: 16 additions & 8 deletions docs/resources/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,9 +117,10 @@ config: ... # The configuration of the resource - different for each different t
created: "2024-10-03T12:42:35.786484Z" # Creation date
identifier: space-8f1cfb-91ecfb # Resource identifier
kind: discoveryspace # Resource kind
metadata: {} # A field for system metadata. User metadata will be in config.metadata
status: [] # A status field
version: v2 # The version of this resource
metadata: {} # Metadata dictionary
provenance: ... # Package provenance frozen at resource creation time
status: [] # A list of status objects describing notable events on the resource
version: v1 # The version of this resource
```

### Resource status
Expand All @@ -140,16 +141,23 @@ All resources have status updates recorded for the following events:
Here is an example:

```yaml
created: "2024-12-19T10:47:03.931824Z"
identifier: 04535d
config: ...
created: "2026-07-09T14:35:23.264745Z"
identifier: 88341a
kind: samplestore
metadata: {}
provenance:
ado:
distributionName: ado-core
distributionVersion: 2.0.0
status:
- event: created
recorded_at: "2024-12-19T10:47:03.931840Z"
recorded_at: "2026-07-09T14:35:23.264754Z"
- event: added
recorded_at: "2024-12-19T10:47:05.720459Z"
version: v2
recorded_at: "2026-07-09T14:35:23.265368Z"
- event: updated
recorded_at: "2026-07-24T13:56:36.053942Z"
version: v1
```

### Programmatic view of resources
Expand Down
27 changes: 10 additions & 17 deletions docs/resources/metastore.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,23 +85,16 @@ ado get contexts
This will output something like

```terminaloutput
┌───────┬────────────────────────┬─────────┐
│ INDEX │ CONTEXT │ DEFAULT │
├───────┼────────────────────────┼─────────┤
│ 0 │ ap-test │ │
│ 1 │ ap-testing │ │
│ 2 │ caikit-testharness │ │
│ 3 │ developer-testing │ │
│ 4 │ finetuning │ │
│ 5 │ ft-prod │ │
│ 6 │ ft-vela │ │
│ 7 │ llm-d │ │
│ 8 │ local │ ✅ │
│ 9 │ local-test │ │
│ 10 │ mascots2024 │ │
│ 11 │ playground │ │
│ 12 │ resource-store-testing │ │
└───────┴────────────────────────┴─────────┘
┌───────┬─────────────────────────┬────────┐
│ INDEX │ CONTEXT │ ACTIVE │
├───────┼─────────────────────────┼────────┤
│ 0 │ algorithm-nexus │ │
│ 1 │ cplex_mip │ │
│ 2 │ geospatial-benchmarking │ │
│ 3 │ llm-d │ │
│ 4 │ local │ ✅ │
│ 5 │ playground │ │
└───────┴─────────────────────────┴────────┘
```

Note, the name of the context is the name of the associated project.
Expand Down
21 changes: 3 additions & 18 deletions docs/resources/sample-stores.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,26 +127,11 @@ The [Sample Store types](#sample-store-types) section details how to fill the
above fields for the different available Sample Store. Here is an example of
copying data from a CSV file using `CSVSampleStore`:

<!-- markdownlint-disable line-length -->
```yaml
specification:
module:
moduleName: ado.core.samplestore.sql
moduleClass: SQLSampleStore
copyFrom:
- module:
moduleClass: CSVSampleStore
storageLocation:
path: "examples/ml-multi-cloud/ml_export.csv"
parameters:
generatorIdentifier: "multi-cloud-ml"
identifierColumn: "config"
experiments:
- experimentIdentifier: "benchmark_performance"
constitutivePropertyMap:
- cpu_type
observedPropertyMap:
- wallClockRuntime
{% include-markdown "../../examples/ml-multi-cloud/ml_multicloud_sample_store.yaml" %}
```
<!-- markdownlint-enable line-length -->

## Accessing the entities in a sample store

Expand Down
Loading