> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://nemo-platform.docs.buildwithfern.com/nemo/platform/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://nemo-platform.docs.buildwithfern.com/nemo/platform/_mcp/server.
# Run an Audit Locally
This tutorial walks through running a single audit end-to-end with the NeMo Auditor plugin SDK. You will persist an audit configuration and a target, run the audit in-process, and inspect the resulting report artifacts.
**What you will learn:**
* Initialize the NeMo Platform SDK and reach the `client.auditor` resource.
* Create an `AuditConfig` selecting which garak probes to run.
* Create an `AuditTarget` pointing at a model through the Inference Gateway.
* Execute the audit locally with `client.auditor.run(...)`.
* Read the JSONL, HTML, and hitlog report artifacts the run produces.
This tutorial takes approximately **10 minutes** to complete, plus however long garak takes to run the selected probes against your target.
## Prerequisites
* Install and start NeMo Platform using the [Setup guide](/documentation/get-started).
* Install garak in a Python virtual environment so the plugin can shell out to it. By default the plugin invokes `~/.auditor/.venv/bin/python -m garak`. Override the interpreter path with `NEMO_AUDITOR_GARAK_PYTHON` if your install lives elsewhere.
* Configure at least one Inference Gateway provider — this tutorial uses a `build` provider named `build` that routes to NVIDIA-build models, but any chat-completion-compatible provider works. See [Inference Gateway](/documentation/vulnerability-scanning/targets/inference-gateway) for details on the `nmp_uri_spec` block used below.
***
## Key Concepts
* **`AuditConfig`** — Selects probes, detectors, and reporting settings for a garak run. Persisted in the entity store.
* **`AuditTarget`** — Identifies the model under test (generator class, model identifier, endpoint options). Persisted in the entity store.
* **In-process run** — `client.auditor.run(...)` shells out to garak on the host running the SDK call, writes report artifacts under a temporary directory, and returns their file paths. There is no remote job submission.
***
## 1. Initialize the SDK
Connect to the running platform and reach the auditor sub-resource:
```python
import os
from nemo_platform import NeMoPlatform
client = NeMoPlatform(
base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
workspace="default",
)
auditor = client.auditor
print(auditor.plugin_status())
# {'plugin': 'auditor', 'status': 'ok', ...}
```
If `plugin_status()` raises a connection error, the platform is not running or `NMP_BASE_URL` is misconfigured. Refer back to [Setup](/documentation/get-started).
***
## 2. Create an Audit Configuration
Build a configuration that runs a small slice of garak's latent-injection probes with three generations per probe:
```python
from nemo_auditor.entities import (
AuditPluginsData,
AuditReportData,
AuditRunData,
AuditSystemData,
)
config = auditor.configs.create(
workspace="default",
name="quick-scan",
description="Lite latentinjection scan, 3 generations per probe.",
system=AuditSystemData(lite=True, parallel_attempts=4),
run=AuditRunData(generations=3),
plugins=AuditPluginsData(probe_spec="latentinjection", detector_spec="auto"),
reporting=AuditReportData(report_prefix="quick-scan"),
)
print(config.model_dump_json(indent=2))
```
For the full set of options on each sub-block, see [Configuration Schema](/documentation/vulnerability-scanning/configurations/schema). For probe selection syntax, see [Selecting Probes](/documentation/vulnerability-scanning/configurations/selecting-probes).
***
## 3. Create an Audit Target
Define the model under test. This example uses garak's `nim.NVOpenAIChat` generator pointed at a provider registered in the Inference Gateway:
```python
target = auditor.targets.create(
workspace="default",
name="llama-31-8b",
type="nim.NVOpenAIChat",
model="meta/llama-3.1-8b-instruct",
options={
"nim": {
"max_tokens": 1024,
"nmp_uri_spec": {
"inference_gateway": {
"workspace": "default",
"provider": "build",
},
},
},
},
)
print(target.model_dump_json(indent=2))
```
The `nmp_uri_spec` sentinel inside `options.nim` tells the plugin to resolve a concrete URI from the Inference Gateway provider at run time. See [Inference Gateway](/documentation/vulnerability-scanning/targets/inference-gateway) for the full conflict rules and resolution behavior.
***
## 4. Run the Audit Locally
`run()` accepts either inline entities or name strings that reference entities in the entity store. Pass the names you persisted above:
```python
result = auditor.run(
config="quick-scan",
target="llama-31-8b",
workspace="default",
)
print(result["status"], result["returncode"])
# completed 0
for name, ref in result["results"].items():
print(f"{name}: {ref['artifact_url']}")
# report-jsonl: file:///var/folders/.../results/report-jsonl
# report-html: file:///var/folders/.../results/report-html
# report-hitlog-jsonl: file:///var/folders/.../results/report-hitlog-jsonl
```
If you prefer to skip the entity-store roundtrip, pass the inline `AuditConfig` and `AuditTarget` objects from the previous two steps directly:
```python
result = auditor.run(config=config, target=target, workspace="default")
```
***
## 5. Read the Results
`run()` returns `file://` URLs for whichever of the three report types garak produced. Parse them with `urllib.parse.urlparse` to recover local paths, then load them as needed.
Load the JSONL probe-by-probe summary:
```python
import json
from pathlib import Path
from urllib.parse import urlparse
def url_to_path(url: str) -> Path:
return Path(urlparse(url).path)
jsonl_path = url_to_path(result["results"]["report-jsonl"]["artifact_url"])
records = [json.loads(line) for line in jsonl_path.read_text().splitlines() if line.strip()]
print(f"Loaded {len(records)} report entries.")
```
Summarize hits per probe from the hitlog:
```python
from collections import Counter
hitlog_ref = result["results"].get("report-hitlog-jsonl")
if hitlog_ref is None:
print("No hits recorded — every probe passed.")
else:
hitlog_path = url_to_path(hitlog_ref["artifact_url"])
hits = [json.loads(line) for line in hitlog_path.read_text().splitlines() if line.strip()]
by_probe = Counter(entry.get("probe") for entry in hits)
for probe, count in by_probe.most_common():
print(f"{probe}: {count} hits")
```
The HTML report at `result["results"]["report-html"]["artifact_url"]` is the most human-friendly summary — open it in a browser to see grouped pass/fail counts and per-probe details.
***
## 6. Clean Up
Delete the entities you created:
```python
auditor.configs.delete(workspace="default", name="quick-scan")
auditor.targets.delete(workspace="default", name="llama-31-8b")
```
Report artifacts are left in place under the scheduler's temporary directory; the OS reaps them eventually.
***
## Troubleshooting
**`FileNotFoundError: garak interpreter not found at ...`**
: The plugin couldn't find a garak install. Either install garak at `~/.auditor/.venv/bin/python`, or set `NEMO_AUDITOR_GARAK_PYTHON` to the absolute path of a Python interpreter that has garak installed.
**`RuntimeError: Failed to resolve inference gateway provider '/'`**
: The `nmp_uri_spec` block in your target's options references an Inference Gateway provider that doesn't exist. List your providers with `client.inference.providers.list(workspace="default")` and update the target to reference an existing one.
**`returncode != 0` with empty `results`**
: garak started but failed early, usually because the target endpoint is unreachable. Inspect `result["stderr_tail"]` for the error message, and verify the model is reachable through the provider you configured.
**Where do the report files live?**
: The scheduler writes garak output under `/garak//.*`, then copies the produced files into the local results directory referenced by each `artifact_url`. For a local run this is a temporary directory under the system's `$TMPDIR`.
***
## Summary and Next Steps
You created an `AuditConfig` and `AuditTarget`, ran a single audit locally with `client.auditor.run(...)`, and loaded the resulting reports.
* For the full SDK surface — including async variants — see [SDK Resources](/documentation/vulnerability-scanning/sdk-resources).
* For more probe selection options, see [Selecting Probes](/documentation/vulnerability-scanning/configurations/selecting-probes).
* For other generator types (OpenAI-compatible endpoints, REST targets, test targets), see [Audit Targets](/documentation/vulnerability-scanning/targets).