DocumentationVulnerability Scanning

Manage Audit Targets

View as Markdown

An AuditTarget identifies the model under test. It pairs a garak generator class (type) with a model identifier (model) and a generator-specific options dict (options). Targets are persisted in the NeMo Platform entity store and referenced by name from client.auditor.run(...).

What an AuditTarget Holds

FieldDescription
typeA fully-qualified garak generator class, such as nim.NVOpenAIChat, openai.OpenAIGenerator, rest.RestGenerator, or test.Blank.
modelThe provider’s model identifier passed through to garak (for example, meta/llama-3.1-8b-instruct).
optionsA nested dict whose top-level key is the generator namespace (nim, openai, rest, …). Contents are passed through to garak unchanged, with one exception: the nmp_uri_spec sentinel is resolved to a concrete uri at run time.
descriptionOptional free-form description shown in listings.

See Target Schema for the full field reference.

Common Target Types

NIM via Inference Gateway

Audit a NIM (or any OpenAI-compatible chat endpoint) routed through a NeMo Platform provider:

1target = client.auditor.targets.create(
2    workspace="default",
3    name="llama-31-8b",
4    type="nim.NVOpenAIChat",
5    model="meta/llama-3.1-8b-instruct",
6    options={
7        "nim": {
8            "max_tokens": 1024,
9            "nmp_uri_spec": {
10                "inference_gateway": {
11                    "workspace": "default",
12                    "provider": "build",
13                },
14            },
15        },
16    },
17)

The nmp_uri_spec block resolves to a concrete uri at run time. See Inference Gateway for details.

OpenAI-compatible Endpoint

For a vanilla OpenAI-compatible endpoint routed through a provider:

1target = client.auditor.targets.create(
2    workspace="default",
3    name="gpt-4o-audit",
4    type="openai.OpenAIGenerator",
5    model="gpt-4o",
6    options={
7        "openai": {
8            "max_tokens": 1024,
9            "seed": 42,
10            "nmp_uri_spec": {
11                "inference_gateway": {
12                    "workspace": "default",
13                    "provider": "openai-provider",
14                },
15            },
16        },
17    },
18)

Blank Target (Smoke Test)

test.Blank returns the empty string for every request — useful for verifying that an audit configuration is well-formed without making network calls:

1target = client.auditor.targets.create(
2    workspace="default",
3    name="smoke",
4    type="test.Blank",
5    model="ignored",
6)

test.Blank ignores model, but the field is still required by the schema.

List Targets

1listing = client.auditor.targets.list(workspace="default")
2for entry in listing["data"]:
3    print(entry["name"], entry["type"], entry["model"])

list() returns the standard envelope (data, pagination, sort); pass page, page_size, and sort keyword arguments for pagination.

Retrieve a Target

1target = client.auditor.targets.get(workspace="default", name="llama-31-8b")

Update a Target

update() replaces every field — pass the fields you want to keep alongside the ones you are changing.

1updated = client.auditor.targets.update(
2    workspace="default",
3    name="llama-31-8b",
4    type="nim.NVOpenAIChat",
5    model="meta/llama-3.3-70b-instruct",
6    options={
7        "nim": {
8            "max_tokens": 2048,
9            "nmp_uri_spec": {
10                "inference_gateway": {"workspace": "default", "provider": "build"},
11            },
12        },
13    },
14)

Delete a Target

1client.auditor.targets.delete(workspace="default", name="llama-31-8b")