---
title: "Benchmark Job Management"
description: ""
---

Manage your evaluation job.

<Note>

**Performance Tuning**: You can improve evaluation performance by setting `job.params.parallelism` to control the number of concurrent requests. A typical default value is 16, but you might need to adjust based on your model capacity and rate limits.

</Note>
## Monitor Job

Monitor the status and progress of a job in real-time.

```python
job_status = client.evaluation.benchmark_jobs.get_status(name=job.name)
while job_status.status in ("active", "pending", "created"):
    time.sleep(10)
    job_status = client.evaluation.benchmark_jobs.get_status(name=job.name)
    print("status:", job_status.status, job_status.status_details)
print(job_status)
```

Refer to [Evaluator Troubleshooting](/documentation/reference/troubleshooting/evaluator) for help troubleshooting job failures.

## Fetch Job Logs

Get JSON logs with pagination. Logs are available for an active job and after the job terminates.

```python
logs_response = client.evaluation.benchmark_jobs.get_logs(name=job.name)
for log_entry in logs_response.data:
    print(f"[{log_entry.timestamp}] {log_entry.message.strip()}")

# Handle pagination
while logs_response.next_page:
    logs_response = client.evaluation.benchmark_jobs.get_logs(
        name=job.name, page_cursor=logs_response.next_page
    )
    for log_entry in logs_response.data:
        print(f"[{log_entry.timestamp}] {log_entry.message.strip()}")
```

## View Evaluation Results

Evaluation results are available after the evaluation job completes. Refer to [Benchmark Results](/documentation/evaluator-benchmark-reference/benchmarks/benchmark-results) for details on fetching evaluation results to analyze the job output.

### Aggregate Scores

Get the overall scores for each metric:

```python
aggregate = client.evaluation.benchmark_jobs.results.aggregate_scores.download(
    job=job.name,
)
print(aggregate)  # Returns a typed AggregatedMetricResult model
```

The aggregate scores provide statistical summaries including mean, standard deviation, min, max, and percentiles for each metric in the benchmark.

### Row-Level Scores

Get per-sample scores for detailed analysis:

```python
row_scores = client.evaluation.benchmark_jobs.results.row_scores.download(
    job=job.name,
)

for row in row_scores:
    print(row)  # Each row contains item, sample, and metrics
```

Row-level scores are returned in JSONL format, with each line containing the original dataset row, model output (for online jobs), and scores for each metric.

## List Available Results

List all result files generated by a job:

```python
results_list = client.evaluation.benchmark_jobs.results.list(name=job.name)

for result in results_list.data:
    print(f"Available: {result.name}")
```

Common result files include:
- `aggregate-scores` - Aggregate scores with statistics
- `row-scores` - Per-row scores in JSONL format
- `artifacts` - Tarball of job artifacts

## Download Job Artifacts

Download artifacts the job produced during evaluation to a tarball.

```python
artifacts = client.evaluation.benchmark_jobs.results.artifacts.download(job=job.name)
artifacts.write_to_file("evaluation_artifacts.tar.gz")
print("Saved artifacts to evaluation_artifacts.tar.gz")
```

Extract files from the tarball with the following command and an `artifacts` directory will be created.

```bash
tar -xf evaluation_artifacts.tar.gz
```
