---
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

title: "Deploy Agents"
description: ""
---

Deploy a registered agent as a running service and invoke it through the
Agents gateway. An agent can run in one of three modes — as a local
subprocess (the default), or as a durable container on Docker or Kubernetes.

<Note>

Resource names for agents and deployments must contain only letters (a-z,
A-Z), digits (0-9), underscores, hyphens, and dots. For example:
`calculator-agent`, `my-agent`, `react-agent`.

</Note>

<Markdown src="/snippets/_snippets/tutorials/cli-sdk-setup.mdx" />
---

## Deployment Modes

`nemo agents deploy --mode <mode>` selects the runtime backend for a
deployment. The default is `subprocess`; `docker` and `k8s` run the agent as a
durable container through the deployments plugin.

| Mode | Runs as | Survives a platform restart | Requires |
|------|---------|-----------------------------|----------|
| `subprocess` (default) | A local FastAPI server on the platform host | No | Nothing extra |
| `docker` | A Docker container | Yes | A container image and a configured `docker` executor |
| `k8s` | A Kubernetes Deployment + Service | Yes | A container image and a configured `k8s` executor |

In every mode the agent is reached the same way — through the Agents gateway,
which resolves the agent's active deployment and proxies the request, so
clients do not need to know which mode the agent runs in.

---

## Subprocess Mode (Default)

The simplest path: the platform launches a FastAPI server for the
agent on its own host, assigns a port, watches its health, and tears it down
on `nemo agents undeploy`. No image or executor configuration is required.

<Tabs>

<Tab title="CLI">

```bash
# Confirm that the local Platform instance is ready
export NMP_BASE_URL=http://localhost:8080

curl -fsS --connect-timeout 2 --max-time 5 \
  "$NMP_BASE_URL/health/ready" >/dev/null || {
  echo "NeMo Platform is not ready at $NMP_BASE_URL"
  exit 1
}

# Register the calculator agent from agent.yaml
nemo agents create \
  --name calculator-agent \
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml

# Deploy as a local subprocess (waits for "running" by default)
nemo agents deploy \
  --agent calculator-agent \
  --name calculator-agent-deployment \
  --mode subprocess

# Invoke through the Agents gateway
nemo agents invoke \
  --agent-deployment calculator-agent-deployment \
  --input "What is 12 multiplied by 8?"
```

</Tab>
<Tab title="Python SDK">

```python
import os

import yaml
from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

# Register the agent from an agent.yaml
with open(
    "plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml"
) as f:
    config = yaml.safe_load(f)
client.agents.create(
    name="calculator-agent",
    config=config,
    config_format=config["config_format"],
)

# Deploy as a local subprocess
client.agents.deployments.create(
    agent="calculator-agent",
    name="calculator-agent-deployment",
)

# Invoke through the Agents gateway
response = client.agents.invoke(
    deployment="calculator-agent-deployment",
    input="What is 12 multiplied by 8?",
)
```

</Tab>

</Tabs>

---

## Container Modes (Docker / Kubernetes)

Container modes give an agent a durable deployment that survives a platform
restart. Instead of a local process, the platform compiles the agent into a
generic deployment and hands it to the **deployments plugin**, which runs it on
the configured executor (Docker or Kubernetes) and projects the running
container's address back onto the agent deployment. The Agents gateway then
routes to that projected address.

### Prerequisites

**1. A container image for the agent.** Container modes run a packaged agent
runtime with the selected harness adapters and the agent's dependencies. Build
one with `nemo agents package`; the command detects `nemo-agents-spec-v1` and
selects the Platform agent image pipeline automatically. Image building
requires the `container` extra. Install it with:

```bash
uv sync --package nemo-agents-plugin --extra container
```

The examples below use the calculator agent that ships with the source
checkout. Run them from the repository root. Its config is located at
`plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml`.

<Tabs>

<Tab title="CLI">

```bash
# Build the calculator agent image and tag it locally
nemo agents package \
  --agent plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml \
  --tag calculator-agent:local

# For k8s, publish to a registry your cluster can pull from
nemo agents package \
  --agent plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml \
  --tag calculator-agent:1.0.0 \
  --publish \
  --registry <your-registry>
```

</Tab>

</Tabs>
<Note>

`--mode docker` needs an image the platform's Docker daemon can run.
`--mode k8s` needs an image the cluster nodes can pull (a registry image, or an
image pre-loaded onto the nodes) — the k8s backend does not use image pull
secrets. Pass the image with `--image`, or set
`agents.deployments.default_image` in the platform configuration.

</Note>

**2. A configured deployments executor.** The platform operator defines named
executors in the platform configuration and points the agents plugin at them.
A minimal Docker + Kubernetes configuration:

```yaml
agents:
  deployments:
    # Names below must match deployments.executors[].name
    docker_executor: local-docker
    k8s_executor: k8s-local
    # Optional: the image used when --image is omitted
    default_image: ""
    # Container port the packaged agent runtime binds (and the readiness probe target)
    container_port: 8000

deployments:
  default_executor: local-docker
  executors:
    - name: local-docker
      backend: docker
      config:
        # Set false when running locally-built (not registry) images
        pull_images: false
    - name: k8s-local
      backend: k8s
      config:
        # Omit kubeconfig_path to use in-cluster ServiceAccount auth
        default_namespace: default
```

<Note>

The Kubernetes backend requires the `kubernetes` Python client in the platform
image and a ServiceAccount with permission to manage Deployments, Services,
ConfigMaps, and Pods in the target namespace. In the packaged Helm chart these
run in the core controller, whose Role already grants those permissions.

</Note>

### Deploy on Docker

<Tabs>

<Tab title="CLI">

```bash
# Run from the repository root (see Prerequisites)
nemo agents create \
  --name calculator-agent \
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml

# --mode docker compiles to the deployments plugin's docker executor
nemo agents deploy \
  --agent calculator-agent \
  --name calculator-agent-docker \
  --mode docker \
  --image calculator-agent:local

# Reached through the Agents gateway, exactly like subprocess mode
nemo agents invoke \
  --agent-deployment calculator-agent-docker \
  --input "What is 12 multiplied by 8?"
```

</Tab>
<Tab title="Python SDK">

```python
import os

import yaml
from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

with open(
    "plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml"
) as f:
    config = yaml.safe_load(f)
client.agents.create(
    name="calculator-agent",
    config=config,
    config_format=config["config_format"],
)

client.agents.deployments.create(
    agent="calculator-agent",
    name="calculator-agent-docker",
    deployment_mode="docker",
    image="calculator-agent:local",
)

response = client.agents.invoke(
    deployment="calculator-agent-docker",
    input="What is 12 multiplied by 8?",
)
```

</Tab>

</Tabs>

### Deploy on Kubernetes

Deployment is identical apart from `--mode k8s`. The deployments plugin creates
a Kubernetes Deployment and a ClusterIP Service, and projects the Service's
in-cluster DNS address (`<service>.<namespace>.svc.cluster.local:<port>`) onto
the agent deployment. Because the Agents gateway runs in-cluster, it routes to
that address directly.

<Tabs>

<Tab title="CLI">

```bash
# Run from the repository root (see Prerequisites)
nemo agents create \
  --name calculator-agent \
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml

nemo agents deploy \
  --agent calculator-agent \
  --name calculator-agent-k8s \
  --mode k8s \
  --image <registry>/calculator-agent:1.0.0

nemo agents invoke \
  --agent-deployment calculator-agent-k8s \
  --input "What is 12 multiplied by 8?"
```

</Tab>
<Tab title="Python SDK">

```python
import os

import yaml
from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

with open(
    "plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml"
) as f:
    config = yaml.safe_load(f)
client.agents.create(
    name="calculator-agent",
    config=config,
    config_format=config["config_format"],
)

client.agents.deployments.create(
    agent="calculator-agent",
    name="calculator-agent-k8s",
    deployment_mode="k8s",
    image="<registry>/calculator-agent:1.0.0",
)

response = client.agents.invoke(
    deployment="calculator-agent-k8s",
    input="What is 12 multiplied by 8?",
)
```

</Tab>

</Tabs>
---

## Model Access from a Deployed Agent

Regardless of mode, model traffic from inside the agent routes back through the
[Inference Gateway](/documentation/models-and-inference). The platform injects
the gateway URL when it deploys the agent, and the gateway resolves model
entity names to upstream providers and supplies their credentials. Two
conventions apply to `agent.yaml`:

- **Set `models.default.model` to the Inference Gateway entity name.** The
  models controller creates these names by replacing slashes and dots with
  hyphens (`nvidia/nemotron-3-nano-30b-a3b` becomes
  `nvidia-nemotron-3-nano-30b-a3b`).
- **Leave `base_url` unset for a Platform-routed model.** When `provider` is
  `nvidia`, `openai`, or `openai-compatible`, the deployment supplies the
  Inference Gateway URL. `api_key_env` names the environment variable expected
  by the selected harness; it does not contain a credential.

The calculator agent uses:

```yaml
models:
  default:
    provider: nvidia
    model: nvidia-nemotron-3-nano-30b-a3b
    api_key_env: NVIDIA_API_KEY
```

To make an external model available to the agent, register a provider first —
see [Deploy Models](/documentation/models-and-inference/tutorials/deploy-models#add-external-providers)
for NVIDIA Build, OpenAI, and Anthropic examples.

<Note>

A deployed agent needs to reach the platform from **inside its container**. The
deployment handles this automatically for both Docker and Kubernetes, so you
normally don't need to configure anything. If an agent can't reach the platform,
set `agents.deployments.gateway_url_override` to a URL that is reachable from
inside the container.

</Note>

#### Docker mode on Linux

On **Linux**, `host.docker.internal` doesn't resolve inside containers, so agent
invokes can fail with `openai.APIConnectionError`. Point the deployment at the
Docker bridge address (`172.17.0.1` by default) in `config.yaml`, then start the
platform bound to all interfaces:

```yaml
agents:
  deployments:
    gateway_url_override: http://172.17.0.1:8080
```

```bash
nemo services run --host 0.0.0.0 --port 8080
```

If your Docker bridge uses a non-default subnet, substitute its gateway address
(`docker network inspect bridge --format '{{ (index .IPAM.Config 0).Gateway }}'`).

---

## Inspect a Deployment

<Tabs>

<Tab title="CLI">

```bash
# Block until the deployment is running or failed
nemo agents deployments wait --agent calculator-agent

# List / inspect deployments
nemo agents deployments list
nemo agents deployments get calculator-agent-deployment
```

</Tab>
<Tab title="Python SDK">

```python
import os

from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

deployment = client.agents.deployments.get("calculator-agent-deployment")
print(deployment["deployment_mode"], deployment["status"], deployment["endpoints"])
```

</Tab>

</Tabs>

For a container-mode deployment, the deployment reports `deployment_mode`
(`docker` or `k8s`), a `status` of `running` once ready, and an `endpoints`
list carrying the container's routable address. Subprocess deployments carry a
loopback `endpoint` instead. The Agents gateway uses whichever the deployment's
mode provides, so invocation is identical across modes.

---

## Deployment Cleanup

<Tabs>

<Tab title="CLI">

```bash
# Stop the running deployment (removes the process/container/k8s objects)
nemo agents undeploy calculator-agent-deployment

# Remove the agent entity
nemo agents delete calculator-agent
```

</Tab>
<Tab title="Python SDK">

```python
import os

from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

# Stop the running deployment (removes the process/container/k8s objects)
client.agents.deployments.delete("calculator-agent-deployment")

# Remove the agent entity
client.agents.delete("calculator-agent")
```

</Tab>

</Tabs>
