---
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

title: "GPT-OSS Models"
description: ""
---

This page provides detailed technical specifications for the OpenAI GPT-OSS model family supported by NeMo Customizer. For supported features and capabilities, refer to [Tested Models](/documentation/customizer-reference/models/model-catalog).

## Before You Start

These models require a Hugging Face token to download. Create a secret with your Hugging Face API key, then create a FileSet and Model Entity referencing the model. See [index](/documentation/customizer-reference/manage-model-entities/overview) for setup instructions.

---

## GPT-OSS 20B

| Property | Value |
|----------|-------|
| Creator | OpenAI |
| Architecture | Mixture of Experts (MoE) Transformer |
| Description | GPT-OSS 20B provides lower latency for local or specialized use cases, featuring full chain-of-thought reasoning and agentic capabilities. |
| Context Length | 131,072 tokens |
| Parameters | 21B parameters (3.6B active parameters) |
| Training Data | Trained on harmony response format |
| Checkpoint Quantization | MXFP4 quantization of the MoE weights |
| Inference Memory | The official checkpoint can run within 16 GB of memory; Customizer training requires the GPU configurations below |
| Default Name | openai/gpt-oss-20b |
| Hugging Face | [openai/gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b) |

### Training Options (20B)

The Automodel contract matrix validates these configurations:

| Fine-Tuning | Dataset Format | GPUs | Sequence Packing |
| --- | --- | --- | --- |
| LoRA | Prompt-completion | 1 | Supported |
| Full SFT | Prompt-completion or chat | 8 | Not supported in the tested configurations |

For the tested Full SFT configuration, tensor and pipeline parallel sizes are `1` and expert parallel size is `8`. The tested LoRA configuration uses one GPU without expert parallelism.

Default training max sequence length: 4096.

### Deployment Configuration

- **LoRA**:
 - NIM Image: `nvcr.io/nim/openai/gpt-oss-20b:1.6.1-variant`
 - GPU Count: 2x 80GB
 - Additional Environment Variables:
 - `NIM_DISABLE_MODEL_DOWNLOAD`: `1`
 - `NIM_WORKSPACE`: `/model-store`
- **Full SFT**:
 - NIM Image: `nvcr.io/nim/nvidia/llm-nim:1.15.5`
 - GPU Count: 2x 80GB
 - Additional Environment Variables:
 - `NIM_MODEL_PROFILE`: `vllm`

## Usage Recommendations

### Reasoning Levels

GPT-OSS models support configurable reasoning levels that you can set in system prompts:

- **Low**: Fast responses for general dialogue
- **Medium**: Balanced speed and detail
- **High**: Deep and detailed analysis

Example: Set reasoning level using "Reasoning: high" in the system prompt.

### MoE Model Parallelization

GPT-OSS models use a Mixture of Experts (MoE) architecture and benefit from specialized parallelization across expert layers for optimal performance. 

<Note>

**MoE Parallelism Constraints**

MoE models only support expert parallelism for distributing experts across GPUs. When `expert_parallel_size > 1`, `tensor_parallel_size` must be set to 1. Additionally, `expert_parallel_size` must evenly divide the number of GPUs. These constraints apply to training parallelism only and NIM deployment may use different GPU counts optimized for inference.

</Note>
### Model Selection Guidelines

- **GPT-OSS 20B**: Ideal for lower latency requirements, local deployment, specialized use cases, and consumer hardware (with Ollama support)

### Important Usage Note

GPT-OSS models use the harmony response format and require this format for proper functionality.

<Note>

For GPT-OSS 20B, sequence packing is supported by the tested Automodel LoRA configuration with prompt-completion data. It is not supported by the tested Full SFT or chat configurations.

</Note>
