# engines.toml Schema Reference

The `engines.toml` file tells `simready-benchmark` where the simulation engine is installed. It can also carry fallback `[paths].sr_specs` and `[tests].paths` values, but the recommended source for those is `project_config.toml` (set `SIMREADY_PROJECT_CONFIG` or pass `--project-config`); the resolution orders below show where each `engines.toml` value sits relative to the project config. This reference documents every section and key, verified against the framework source.

## Lookup Order

The framework resolves which `engines.toml` is in effect through a 4-tier lookup. The first match wins.

| Tier | Source | Notes |
|---|---|---|
| 1 | `--engines-toml <FILE>` CLI flag | Explicit. Used as-is. |
| 2 | `$SIMREADY_ENGINES_TOML` environment variable | A missing file here is a hard error, not a fall-through. An empty-string value is treated as unset. |
| 3 | `./engines.toml` | In the current working directory. |
| 4 | `~/.simready-benchmark/engines.toml` | The per-user default. |

When a file is loaded, the framework writes a one-line banner to stderr, `[engines.toml] <path>`, one time per process, so the user can identify which tier won. Set `SIMREADY_QUIET_ENGINES_TOML=1` to suppress it. The `--show-config` command suppresses the banner and renders its own structured output instead.

## Path Resolution Rule

A relative path inside any section of `engines.toml` resolves against the directory that holds `engines.toml`, not against the shell working directory. An absolute path is used as given. TOML strings use forward slashes even on Windows, because a backslash needs escaping inside a TOML string.

## `[kit.<name>]`

One section per Kit or Isaac Sim install. The `<name>` is a user-chosen identifier, for example `isaac_sim`. The same section shape covers Kit and Isaac Sim. The `--runtime <name>` flag selects among multiple `[kit.*]` sections at run time.

| Key | Required | Type | Default | Meaning |
|---|---|---|---|---|
| `executable_path` | yes | string | none | Path to the launcher script, for example `isaac-sim.bat` on Windows or `isaac-sim.sh` on Linux, or a native binary. |
| `version` | yes | string | none | Engine version string, used for engine-versus-test matching, for example `5.1.0`. |
| `tags` | no | list of strings | `["kit"]` | Capability tags the engine provides. Tests declare the tags they need, and the orchestrator matches tests to engines by tag. The conventional value for Isaac Sim is `["isaac", "kit"]`. |
| `custom_args` | no | list of strings | `[]` | Extra arguments passed to the launcher. |
| `session_mode` | no | string | `"batch"` | Session mode for the engine. |
| `driver_compatibility` | no | array of tables | `[]` | Driver-version checks run before launch. Refer to the subsection below. |

The fields map to the `EngineConfig` dataclass in `simready_benchmark/engines/base.py`.

### `[[kit.<name>.driver_compatibility]]`

An array-of-tables. Each entry describes a driver-version constraint for one operating system, checked before the engine launches.

| Key | Required | Type | Default | Meaning |
|---|---|---|---|---|
| `os` | yes | string | none | Operating system the entry applies to, for example `windows` or `linux`. |
| `ranges` | yes | list of strings | none | Version constraint strings, for example `[">=560.00"]`. |
| `message` | no | string | none | Message surfaced to the user when the check fails. |

```toml
[kit.isaac_sim]
executable_path = "C:/Program Files/NVIDIA/IsaacSim/2025.1/isaac-sim.bat"
version = "5.1.0"
tags = ["isaac", "kit"]

[[kit.isaac_sim.driver_compatibility]]
os = "windows"
ranges = [">=560.00"]
message = "Isaac Sim 5.1 requires NVIDIA driver 560 or newer"
```

## `[paths]`

Optional defaults so the user does not have to pass the corresponding flags on every command. A CLI flag always wins over this section.

| Key | Required | Type | Default | Meaning |
|---|---|---|---|---|
| `project_config` | no | string | none | Path to the project's `project_config.toml`. Setting it points the framework at the project config from this one file (no `--project-config` flag or `$SIMREADY_PROJECT_CONFIG` env var), which then supplies content, sr_specs, profiles, features, and tests. The simplest setup. |
| `sr_specs` | no | string | none | The directory that contains the SimReady spec, including feature definitions and `profiles.toml`. Setting it makes `--sr-specs` optional. Usually unnecessary when `project_config` is set. |
| `output_dir` | no | string | none | Where the framework writes `_testing/`. Without it, runs land in `<CWD>/_testing`. |

All values can be absolute or relative. A relative path resolves against the directory that holds `engines.toml`.

### project_config Resolution Order

The first match wins.

1. `--project-config PATH` CLI flag.
2. `$SIMREADY_PROJECT_CONFIG` environment variable.
3. `[paths].project_config` in this file.
4. A repository-relative `sample_content/project_config.toml`.
5. `./project_config.toml` in the current working directory.

### sr_specs Resolution Order

The first match wins. Explicit sources override the project config, which is the project default.

1. `--sr-specs PATH` CLI flag.
2. `$SIMREADY_SR_SPECS` environment variable.
3. `[paths].sr_specs` in this file.
4. `project_config`, when a project config is found (the sr_specs root is derived from its `[validate]` paths, not a separate key).
5. A repository-relative fallback in a developer source tree.
6. `./sr_specs` in the current working directory.

### output_dir Resolution Order

The first match wins.

1. `--output-dir PATH` CLI flag.
2. `project_config.project_root/_testing`, when `--project-config` is given.
3. `[paths].output_dir` in this file.
4. `<CWD>/_testing` fallback.

```toml
[paths]
sr_specs = "D:/simready_next/simready_foundations/nv_core/sr_specs"
output_dir = "D:/simready_runs"
```

## `[tests]`

Optional. Declares on-disk test packs that are not pip-installed. These combine with packs discovered through the `simready_benchmark.tests` entry-point group. They do not replace them.

| Key | Required | Type | Default | Meaning |
|---|---|---|---|---|
| `paths` | no | list of strings | `[]` | One entry per on-disk test pack. Each entry is a package SOURCE directory. |

### The SOURCE-Directory Rule

Each `[tests].paths` entry must point at the package source directory, the one that contains `__init__.py`, not the repository root and not the wheel directory. The framework adds the parent of that directory to `sys.path` and walks the directory for `@test`-decorated functions.

```
<repo>/                          <- repo root  (wrong)
  packages/
    <package_name>/              <- wheel dir  (wrong)
      src/
        <package_name>/          <- SOURCE dir (right, has __init__.py)
```

The source directory name uses underscores, the Python import name, while the repository name typically uses dashes. The directory `simready_benchmark_kit_suite` lives inside the repository `simready-benchmark-kit-suite`.

A relative entry resolves against the directory that holds `engines.toml`. The framework normalizes each entry but does not verify that it exists on disk at load time, so a missing path produces a `[missing]` marker in `--show-config` and an error later in the run.

### Tests-Path Resolution Order

The highest-priority non-empty source wins among the four explicit sources below (no merging among them). Two sources always merge on top of whichever wins: pip-installed entry-point packs, and the bundled FET suite derived from the project config (its fixed location under `nv_core`, when present). So the FET default is found with no path, and an explicit entry adds packs rather than replacing it.

1. `--tests-path PATH ...` CLI flag.
2. `$SIMREADY_BENCHMARK_TESTS_PATH` environment variable, separated by the operating-system path separator.
3. `[tests].paths` in this file.
4. `project_config.toml [tests].paths`, when a project config is found. This is the recommended home for extra on-disk packs and the lowest-priority explicit source, so any of the above overrides it.

```toml
[tests]
paths = [
    "D:/work/simready-benchmark-kit-suite/packages/simready_benchmark_kit_suite/src/simready_benchmark_kit_suite",
    "D:/work/oem-acme-tests/src/oem_acme_tests",
]
```

## `[nvcf.<name>]`

Optional. Declares a remote NVCF worker endpoint. NVCF support is a Technology Preview, and its schema can change in a future version.

| Key | Required | Type | Default | Meaning |
|---|---|---|---|---|
| `url` | yes | string | `""` | The NVCF worker endpoint URL. A config with no URL fails validation. |
| `api_key_env` | no | string | none | Name of the environment variable that holds the API key. If the named variable is unset, the framework warns and calls the endpoint unauthenticated rather than failing. |
| `function_id` | no | string | none | NVCF function identifier. |
| `version_id` | no | string | none | NVCF function version identifier. |
| `version` | no | string | `"nvcf-remote"` | Version string carried on the resulting engine config. |
| `tags` | no | list of strings | `["nvcf", "kit"]` | Capability tags carried on the resulting engine config. |
| `custom_args` | no | list of strings | `[]` | Extra arguments carried on the resulting engine config. |

```toml
[nvcf.remote_worker]
url = "https://nvcf-fn-1.example/v2/nvcf/exec/functions/abc"
api_key_env = "NVCF_API_KEY"
```

## Complete Annotated Example

```toml
# engines.toml
# Relative paths resolve against THIS file's directory.
# Use forward slashes even on Windows.

# --- Engine: one [kit.<name>] section per install. ---
[kit.isaac_sim]
executable_path = "C:/Program Files/NVIDIA/IsaacSim/2025.1/isaac-sim.bat"
version = "5.1.0"
tags = ["isaac", "kit"]

# Optional driver check before launch.
[[kit.isaac_sim.driver_compatibility]]
os = "windows"
ranges = [">=560.00"]
message = "Isaac Sim 5.1 requires NVIDIA driver 560 or newer"

# A second engine, selected at run time with --runtime isaac_sim_4_5.
[kit.isaac_sim_4_5]
executable_path = "C:/Program Files/NVIDIA/IsaacSim/4.5/isaac-sim.bat"
version = "4.5.0"
tags = ["isaac", "kit"]

# --- Default paths, so the matching flags become optional. ---
[paths]
sr_specs = "D:/simready_next/simready_foundations/nv_core/sr_specs"
output_dir = "D:/simready_runs"

# --- On-disk test packs, combined with pip-installed packs. ---
# Each entry is the package SOURCE directory that contains __init__.py.
[tests]
paths = [
    "D:/work/simready-benchmark-kit-suite/packages/simready_benchmark_kit_suite/src/simready_benchmark_kit_suite",
]

# --- NVCF remote worker (Technology Preview; schema can change). ---
[nvcf.remote_worker]
url = "https://nvcf-fn-1.example/v2/nvcf/exec/functions/abc"
api_key_env = "NVCF_API_KEY"
```

## Verify the Result

Run `simready-benchmark --show-config` to confirm the resolution. It prints the framework version and module path, the `engines.toml` file and tier that matched, each configured engine with an `[ok]` or `[missing]` marker on its `executable_path`, the resolved `[paths].sr_specs`, each `[tests].paths` entry with a marker, and the discoverable test packs. Fix any path that shows `[missing]`, then re-run until every path shows `[ok]`.
