---
name: configure-engines-toml
description: Use when a user needs to create or repair their engines.toml so simready-benchmark can find their Kit or Isaac Sim install. The engine binary lives in engines.toml; sr_specs and test packs come from project_config.toml, which engines.toml can override per install. Guides an interactive setup and writes a valid engines.toml.
license: Apache-2.0
compatibility: Requires simready-benchmark (pip install simready-benchmark). For Kit / Isaac Sim engines, also requires a separately-installed runtime to point at.
metadata:
  author: NVIDIA
  library: simready_benchmark
  version: "2026.05"
---

## Confirm the Installed Version

This skill ships inside the `simready-benchmark` wheel, so the copy installed with the framework matches the installed code. If you are working from a cached copy or from memory rather than the installed package, confirm the framework version with `simready-benchmark --version`, and treat `simready-benchmark --help` and `simready-benchmark --show-config` as the authoritative source for flags and configuration. A flag documented here might have changed in a different version.

## How to Use This Skill

Walk the user through the numbered sequence below. Ask one question at a time, wait for the answer, and validate each path on disk before you write it into the file. Do not dump the whole template at the user and ask them to fill it in. Assemble the file from their answers, write it, then verify with `simready-benchmark --show-config`.

The schema for every section and key is in [references/engines-toml-schema.md](references/engines-toml-schema.md). Refer to it whenever you are unsure of a default or a type.

## What Belongs in engines.toml

`engines.toml` carries the engine binary: the `[kit.<name>]` (or `[nvcf.<name>]`) sections, plus optional `[paths].output_dir`. The specifications (`sr_specs`) and the benchmark packs come from `project_config.toml`, which is the single pointer for content, profiles, features, and tests. Point the framework at the project config in any of three ways: `--project-config`, the `SIMREADY_PROJECT_CONFIG` env var, or `[paths].project_config` in this file (the most convenient for a per-install setup, since neither the flag nor the env var is then needed).

So in the common case you only configure the engine here. Configure `[paths].sr_specs` (step 4) and `[tests].paths` (step 5) in `engines.toml` only when the user has no project config, or when this machine needs to override the project config's value. When set, the `engines.toml` value wins over `project_config.toml`, and for tests it replaces the project config's `[tests].paths` rather than adding to it. (The derived FET default and pip-installed entry-point packs still merge on top regardless.) If the user has a `project_config.toml` and does not need a per-machine override, skip those steps and tell them to set `SIMREADY_PROJECT_CONFIG` instead.

## 1. Decide Where to Write the File

The framework finds `engines.toml` through a 4-tier lookup. The first match wins.

1. `--engines-toml <FILE>` CLI flag.
2. `$SIMREADY_ENGINES_TOML` environment variable. If this is set but points at a missing file, the framework raises a hard error rather than falling through to a lower tier.
3. `./engines.toml` in the current working directory.
4. `~/.simready-benchmark/engines.toml`, the per-user default.

Ask the user which destination they want:

- A project-local `./engines.toml` keeps the config next to one project, travels with the repository, and overrides the per-user default when commands run from that directory.
- The per-user default `~/.simready-benchmark/engines.toml` applies to every project the user runs from any directory, so it suits a single workstation with one engine install.

| Situation | Recommended destination |
|---|---|
| One workstation, one engine, shared across all projects | `~/.simready-benchmark/engines.toml` |
| Config should travel with a repository, or per-project test packs differ | `./engines.toml` in the project root |
| Continuous integration that sets an explicit path | A custom path passed through `--engines-toml` or `$SIMREADY_ENGINES_TOML` |

The recommended path is the setup wizard, `simready-benchmark --init-engines-toml`. It acquires `simready_foundations`, detects or installs Isaac Sim, and writes a complete `engines.toml` with no placeholders to edit. With no argument it writes `~/.simready-benchmark/engines.toml`. Pass `./` to write `./engines.toml`, or a full path. It overwrites an existing file by default; pass `--no-overwrite` to refuse, and on a terminal it confirms first.

```bash
simready-benchmark --init-engines-toml                  # writes ~/.simready-benchmark/engines.toml
simready-benchmark --init-engines-toml ./               # writes ./engines.toml
simready-benchmark --init-engines-toml --no-overwrite   # keep an existing file
```

The wizard prompts before it clones foundations and before the multi-gigabyte Isaac install, and it accepts `--foundations-path`, `--clone-foundations`, `--install-isaac`, `--executable-path`, and `--project-config` for unattended runs. The steps below describe building or repairing the file by hand, which is what you do when you add a second engine, configure NVCF, or set a per-machine path override.

## 2. Configure the Engine

Each engine install is one `[kit.<name>]` section. The `<name>` is yours to choose, for example `isaac_sim`. Ask the user:

1. Which engine they are pointing at, Kit or Isaac Sim. Both use the same `[kit.<name>]` section shape.
2. The path to the launcher script. On Windows this is `isaac-sim.bat`, and on Linux it is `isaac-sim.sh`. Common locations:
   - Windows: `C:/Program Files/NVIDIA/IsaacSim/2025.1/isaac-sim.bat`
   - Linux: `/opt/nvidia/isaac-sim/isaac-sim.sh`
3. The engine version string, for example `5.1.0`. The framework uses this for engine-versus-test matching.

Validate that the launcher path exists on disk before you write it. If it does not exist, stop and ask the user for the correct path rather than writing a path that will fail at launch.

Set `tags` to the capability tags the engine provides. If omitted, the default is `["kit"]`; the conventional value for Isaac Sim is `["isaac", "kit"]`. Tests declare the tags they need, and the orchestrator matches tests to engines by tag.

```toml
[kit.isaac_sim]
executable_path = "C:/Program Files/NVIDIA/IsaacSim/2025.1/isaac-sim.bat"
version = "5.1.0"
tags = ["isaac", "kit"]
```

Ask whether the user wants a driver-compatibility check before launch. If yes, add one or more `[[kit.<name>.driver_compatibility]]` array-of-tables entries. Each entry has `os`, `ranges` (a list of version constraint strings), and an optional `message`.

```toml
[[kit.isaac_sim.driver_compatibility]]
os = "windows"
ranges = [">=560.00"]
message = "Isaac Sim 5.1 requires NVIDIA driver 560 or newer"
```

## 3. Multiple Engines

Ask whether the user has more than one engine instance, for example two Kit versions side by side. If so, add a second `[kit.<name>]` section with its own distinct name, path, and version. Repeat the engine block from step 2.

```toml
[kit.isaac_sim]
executable_path = "C:/Program Files/NVIDIA/IsaacSim/2025.1/isaac-sim.bat"
version = "5.1.0"
tags = ["isaac", "kit"]

[kit.isaac_sim_4_5]
executable_path = "C:/Program Files/NVIDIA/IsaacSim/4.5/isaac-sim.bat"
version = "4.5.0"
tags = ["isaac", "kit"]
```

When more than one `[kit.*]` engine is declared, the user selects among them at run time with `--runtime <name>`. The flag accepts a list, so a continuous-integration matrix can pass several names. A name that is not present in `engines.toml` is a hard error, so a typo cannot silently fall back to a different engine. Omitting `--runtime` uses every declared engine.

## 4. Set Default Paths

The optional `[paths]` section sets defaults so the user does not have to pass the corresponding flags on every command. A CLI flag always wins over this section.

If the user has a `project_config.toml`, the most convenient setup is to name it with `[paths].project_config`. The framework then resolves it from this one file with no `--project-config` flag and no `SIMREADY_PROJECT_CONFIG` env var, and the project config supplies content, sr_specs, profiles, features, and tests. Validate the path exists before writing it.

```toml
[paths]
project_config = "C:/simready_testing/simready_foundations/sample_content/project_config.toml"
```

It resolves below `--project-config` and `$SIMREADY_PROJECT_CONFIG`, above the repo-relative and CWD fallbacks.

Ask whether to set `[paths].output_dir`. Without it, and without `--output-dir` or `--project-config`, the framework writes `_testing/` into the current working directory, which depends on where the shell happens to be. Setting it fixes where runs land.

```toml
[paths]
output_dir = "D:/simready_runs"
```

`[paths].sr_specs` overrides the project config's sr_specs for this install (and is how you set it when there is no project config). It sits above `project_config` but below `--sr-specs` and `$SIMREADY_SR_SPECS`. Set it ONLY when the user has no `project_config.toml`, or needs a per-machine override; otherwise `sr_specs` comes from the project config (refer to "What Belongs in engines.toml" above). If you do set it, point it at the directory that contains the SimReady spec (feature definitions and `profiles.toml`), and validate that it exists before writing it.

```toml
[paths]
output_dir = "D:/simready_runs"
sr_specs = "D:/simready_next/simready_foundations/nv_core/sr_specs"   # only without a project config
```

Both `[paths]` values can be absolute or relative. A relative path resolves against the directory that holds `engines.toml`, not the shell working directory.

## 5. Test Packs

Test packs that the user installed with `pip` need no configuration (entry-point discovery). And when the user points at a foundations `project_config.toml`, the bundled FET suite is discovered automatically from its fixed location under `nv_core` (the sibling of `sr_specs`) with no `[tests].paths` anywhere. So a user with a project config usually configures no tests here at all.

Configure `[tests].paths` in `engines.toml` only when the user has no project config, or to override the project config's list on this machine (for example a `simready-benchmark-kit-suite` checkout used before the PyPI release, or extra OEM test folders). For each one, add a `[tests].paths` entry. The lookup is first-non-empty-wins in the order `--tests-path`, `SIMREADY_BENCHMARK_TESTS_PATH`, `engines.toml [tests].paths`, then `project_config.toml [tests].paths`, so an `engines.toml` list overrides the project config's `[tests].paths` and replaces it. The derived FET default (under the project config's `nv_core`) and pip-installed entry-point packs still merge on top.

Enforce the SOURCE-directory rule for every entry. The path must end at the package source directory, the one that contains `__init__.py`, not the repository root and not the wheel directory. The framework adds the parent of that directory to `sys.path` and walks the directory for `@test`-decorated functions. A typical layout shows where the right directory sits:

```
<repo>/                          <- repo root  (wrong)
  packages/
    <package_name>/              <- wheel dir  (wrong)
      src/
        <package_name>/          <- SOURCE dir (right, has __init__.py)
```

Note that the directory name uses underscores, the Python import name, while the repository name typically uses dashes. The source directory `simready_benchmark_kit_suite` lives inside the repository `simready-benchmark-kit-suite`.

```toml
[tests]
paths = [
    "D:/work/simready-benchmark-kit-suite/packages/simready_benchmark_kit_suite/src/simready_benchmark_kit_suite",
    "D:/work/oem-acme-tests/src/oem_acme_tests",
]
```

Validate that each resolved path exists and contains `__init__.py` before you write it. Path-discovered packs combine with entry-point-discovered packs. They do not replace them.

## 6. NVCF (Technology Preview)

NVCF support is a Technology Preview. Its schema can change in a future version, so confirm the keys against `simready-benchmark --show-config` after you write the file.

Ask whether the user wants to dispatch tests to remote NVCF workers. If yes, add a `[nvcf.<name>]` section. The required key is `url`. Set `api_key_env` to the name of the environment variable that holds the API key. If the named variable is unset, the framework warns and calls the endpoint unauthenticated rather than failing, which suits local emulators.

```toml
[nvcf.remote_worker]
url = "https://nvcf-fn-1.example/v2/nvcf/exec/functions/abc"
api_key_env = "NVCF_API_KEY"
```

## 7. Write the File

Assemble the file from the answers and write it to the destination chosen in step 1. TOML strings use forward slashes even on Windows, because a backslash needs escaping inside a TOML string.

The common case, where the user has a `project_config.toml` for sr_specs and tests, is engine-only:

```toml
# engines.toml

[kit.isaac_sim]
executable_path = "C:/Program Files/NVIDIA/IsaacSim/2025.1/isaac-sim.bat"
version = "5.1.0"
tags = ["isaac", "kit"]

[[kit.isaac_sim.driver_compatibility]]
os = "windows"
ranges = [">=560.00"]
message = "Isaac Sim 5.1 requires NVIDIA driver 560 or newer"

[paths]
output_dir = "D:/simready_runs"
```

Only when the user has no project config do you add the `[paths].sr_specs` and `[tests].paths` fallbacks:

```toml
[paths]
output_dir = "D:/simready_runs"
sr_specs = "D:/simready_next/simready_foundations/nv_core/sr_specs"

[tests]
paths = [
    "D:/work/simready-benchmark-kit-suite/packages/simready_benchmark_kit_suite/src/simready_benchmark_kit_suite",
]
```

## 8. Verify

Run `simready-benchmark --show-config`. It prints which file matched, which tier produced it, and the configured engines, with `[ok]` or `[missing]` markers on each path.

```bash
simready-benchmark --show-config
```

Confirm that:

- The `engines.toml` section reports `loaded from` the file you just wrote, on the expected tier.
- Each `[kit.<name>]` engine is listed, and its `executable_path` shows `[ok]`.
- `sr_specs` resolves and shows `[ok]`, whether it came from the project config or the `[paths].sr_specs` fallback.
- The discovered test packs include the expected source, whether from the project config or a fallback.

If any path shows `[missing]`, fix the offending value. A `[missing]` launcher or `sr_specs` is usually a typo, a backslash that needs to be a forward slash, or a relative path resolved against the wrong directory. If `sr_specs` or the test packs are empty, the usual cause is that `SIMREADY_PROJECT_CONFIG` is unset; set it (or pass `--project-config`) and re-run. Re-run `--show-config` after each fix until every path shows `[ok]`.

## Key Rules

- The 4-tier lookup stops at the first match: CLI flag, `$SIMREADY_ENGINES_TOML`, `./engines.toml`, then `~/.simready-benchmark/engines.toml`. A `$SIMREADY_ENGINES_TOML` that points at a missing file is a hard error, not a fall-through.
- Relative paths inside `engines.toml` resolve against the file's own directory, not the shell working directory.
- TOML strings use forward slashes. A backslash needs escaping.
- Every `[tests].paths` entry must point at the package SOURCE directory that contains `__init__.py`, not the repository root or the wheel directory.
- `sr_specs` and test packs are recommended to live in `project_config.toml` (set `SIMREADY_PROJECT_CONFIG` or pass `--project-config`). The `project_config.toml` is the lowest-priority source: `engines.toml` `[paths].sr_specs` / `[tests].paths`, the environment variables, and the CLI flags all override it.
- `[paths].sr_specs` and `[tests].paths` are overridden only by the matching CLI flag or environment variable; they themselves override `project_config.toml`. Resolution is first-non-empty-wins with no merging, except two always-merged test sources: pip-installed entry-point packs and the derived FET default (the suite's location under the project config's `nv_core`).
- NVCF is a Technology Preview. Its schema can change.
- Validate every path on disk before writing it, then confirm the whole file with `simready-benchmark --show-config`.

For the full schema, lookup order, and an annotated example, refer to [references/engines-toml-schema.md](references/engines-toml-schema.md).
