Cluster
updates automatically
Loading…
Start a model
Model
Nodes
Automatic
Choose myself
Deployment name
Used as the API/LiteLLM model name. Auto-filled from the selected model - change it to run more than one deployment of the same model at once.
Start model
Starting takes several minutes. You can close this page - it keeps running.
Running models
updates automatically
Loading…
Model catalogue
Add model
Changes are saved straight to the cluster - no
kubectl
required.
Add model
Name
Lowercase. This is what users ask for at the endpoint.
Hugging Face repository
Size (GB)
Decides how many nodes are needed.
Minimum nodes
Container image
Pre-filled with the cluster default. Change only if this model needs a different image.
Advanced (optional)
Multi-node parallelism
Pipeline parallel (default)
Tensor parallel
Only applies when a deployment of this model spans more than one node.
Extra vLLM arguments (one per line)
Environment variables (KEY=value, one per line)
Cancel
Save