Create ModelDeploymentConfig

POST

https://host.com/apis/models/v2/workspaces/:workspace/deployment-configs

POST

/apis/models/v2/workspaces/:workspace/deployment-configs

$ curl -X POST https://host.com/apis/models/v2/workspaces/workspace/deployment-configs \
>      -H "Content-Type: application/json" \
>      -d '{
>   "name": "nim-config-v1",
>   "nim_deployment": {
>     "gpu": 2
>   }
> }'

1 {
2   "name": "llama-3.1-8b",
3   "workspace": "research-team-alpha",
4   "created_at": "2024-01-15T09:30:00Z",
5   "updated_at": "2024-01-15T09:30:00Z",
6   "entity_version": 1,
7   "nim_deployment": {
8     "gpu": 2,
9     "model_type": "llm",
10     "lora_enabled": false,
11     "disk_size": "100Gi",
12     "image_name": "nvcr.io/nemo/multi-llm",
13     "image_tag": "v1.2.3",
14     "model_namespace": "nvidia",
15     "model_name": "llama-3-8b",
16     "model_revision": "main",
17     "model_provider": "hf",
18     "chat_template": "default_chat_template.j2",
19     "tool_call_config": {
20       "tool_call_parser": "openai",
21       "tool_call_plugin": "research-team-alpha/custom-tool-plugin",
22       "auto_tool_choice": true
23     },
24     "additional_envs": {},
25     "k8s_nim_operator_config": {
26       "resources": {},
27       "tolerations": [
28         {}
29       ],
30       "node_selector": {},
31       "startup_probe_grace_seconds": 600
32     },
33     "override_config": {}
34   },
35   "id": "d4f8a9b2-3c7e-4f1a-9b6d-2e5f7a8c9d01",
36   "project": "projects/nemo-platform/llm-deployments",
37   "description": "Deployment config for LLaMA 3 8B model with 2 GPUs",
38   "model_entity_id": "model-entity-12345"
39 }

Create a new ModelDeploymentConfig (version 1).

Path parameters

workspacestringRequired

Request

This endpoint expects an object.

namestringRequiredformat: "^[\w\-.]+$"<=255 characters

Name of the deployment configuration. Allowed characters: letters (a-z, A-Z), digits (0-9), underscores, hyphens, and dots.

nim_deploymentobjectRequired

Configuration for NIM-based deployment

projectstringOptionalformat: "^[\w\-./]+$"<=255 characters

The URN of the project associated with this deployment configuration

descriptionstringOptional<=1000 characters

Optional description of the deployment configuration

model_entity_idstringOptional<=255 characters

Optional reference to the base model entity ID for this deployment

Response

Create a new model deployment configuration

namestringformat: "^[\w\-.]+$"<=255 characters

Name of the entity. Name/workspace combo must be unique across all entities. Allowed characters: letters (a-z, A-Z), digits (0-9), underscores, hyphens, and dots.

workspacestringformat: "^[\w\-.]+$"<=255 characters

The workspace of the entity. Allowed characters: letters (a-z, A-Z), digits (0-9), underscores, hyphens, and dots.

created_atdatetime

The timestamp of model entity creation

updated_atdatetime

The timestamp of the last model entity update

entity_versioninteger

Version of this deployment config. Automatically managed.

nim_deploymentobject

Configuration for NIM-based deployment

idstring

Unique identifier for the deployment config

projectstringformat: "^[\w\-./]+$"<=255 characters

The URN of the project associated with this entity.

descriptionstring<=1000 characters

Optional description of the deployment configuration

model_entity_idstring<=255 characters

Optional reference to the base model entity ID for this deployment

Errors

422

Unprocessable Entity Error