# Execution Architecture

The execution package exposes a single public entrypoint:

- `Evaluator`
  Result-oriented API that evaluates one metric or a sequence of metrics and
  returns finished `EvaluationResult` or `BenchmarkEvaluationResult` values.

## Public Surface

- `Evaluator.run()` / `Evaluator.run_sync()`
  Evaluate one metric or many metrics and return the completed result.
- `RunConfig`
  A Pydantic config type for execution-only settings such as parallelism and
  sample limits.
- `EvaluationRequest`
  The normalized request object passed into evaluation backends.
- `EvaluationBackend`
  The protocol implemented by result-returning execution backends.

## Backend Selection

```python
# Local SDK execution
evaluator = Evaluator()
result = await evaluator.run(
    metrics=ExactMatchMetric(reference="{{item.reference}}"),
    dataset=[{"reference": "Paris", "output_text": "Paris"}],
)
```

```python
# Evaluator plugin execution with the mounted NeMo Platform evaluator resource.
# This path runs the plugin job locally in-process for now.
client = AsyncNeMoPlatform(workspace="default")
evaluator = Evaluator(client=client.evaluator)
result = await evaluator.run(metrics=metric, dataset=data)
```

## Design Notes

- `Evaluator()` uses `LocalBackend`.
- `Evaluator(client=...)` accepts an evaluator backend object. For NeMo Platform
  execution, pass the mounted evaluator resource, such as `platform.evaluator`.
- The evaluator plugin resource implements `EvaluationBackend` and delegates to
  a private composed backend helper in the plugin package.
- `nemo_platform` and `nemo_evaluator` are optional for local SDK evaluation.
