AIM Profiles#
An AIM profile is a self-contained runtime configuration for an inference workload. It answers five questions about a deployment without consulting any other resource:
Model architecture — What model family does this serve? (
aimId)Accelerator — What hardware is required? (
acceleratorVendor,acceleratorType,acceleratorModel,acceleratorCount)Engine — How is the inference engine configured? (
engineArgs,engineEnv)Container image — What image runs the workload? (
image)Optimization target — What was this profile tuned for? (
metric,precision,type)
AIMProfile and AIMClusterProfile carry this configuration. An AIMService resolves to exactly one of them and deploys it.
v1alpha2
Profiles are part of aim.eai.amd.com/v1alpha2. They replace v1alpha1 Service Templates, which are deprecated.
Where profiles come from#
Most profiles aren’t hand-authored. They’re produced by model discovery, model-ID profile generation, or profile derivation:
Source |
Profile origin |
Typical labels |
|---|---|---|
Image discovery on an official AIM image |
|
|
Image discovery on a base AIM image |
|
|
|
|
|
|
|
|
Hand-authored by a user |
|
(none of the |
See AIM Models for the four model flows that produce these profiles.
Cluster vs namespace scope#
Resource |
Scope |
Caching support |
|---|---|---|
|
Cluster |
No |
|
Namespace |
|
When both a namespace-scoped and cluster-scoped profile match a service’s selector, the namespace-scoped profile takes precedence.
Deployable vs base#
A profile is deployable when it carries both spec.aimId and a non-empty spec.modelSources. The model controller stamps this onto status.deployable: true and labels the profile aim.eai.amd.com/profile-role=deployable.
A profile is a base profile when both aimId and modelSources are empty. Base profiles carry status.deployable: false and profile-role=base. They exist only as derivation sources for custom-model AIMModels — they cannot back an AIMService directly.
Mixed spec (one of aimId / modelSources set, the other empty) is rejected at admission so status.deployable is always derivable from spec.
Provenance labels#
v1alpha2 stamps a small set of canonical labels on every operator-produced profile. These are the labels selectors key off, both inside the AIMService resolver and inside AIMProfileSet derivation.
Label |
Values |
|---|---|
|
|
|
|
|
Name of the owning |
|
|
Two provenance markers are annotations, not labels — nothing selects on them, so a label selector will never match:
Annotation |
Values |
|---|---|
|
|
|
|
The matching status fields mirror the labels for kubectl-friendly access:
status.originmirrorsprofile-originstatus.sourceModel.{name,kind,namespace}mirrorssource-model+source-model-scopestatus.deployablederives from spec (also implicit fromprofile-role)
See Naming and Labels → Profile labels for the canonical list (including label setters and selector recipes).
Projected profile YAML compatibility#
AIMProfile.spec is the canonical control-plane model used for profile selection, scheduling, derivation, and overrides. The YAML mounted into an AIM runtime is a versioned wire format. AIM Engine decodes source YAML into the canonical CRD and later uses the recorded source codec to encode it back into a format accepted by that runtime. Detection comes from the source profile document, never from the image repository or tag.
The current aim-profile/v1 codec covers the legacy, transitional, and strict accelerator field variants:
Source YAML field presence |
Projected metadata |
|---|---|
|
|
|
|
|
|
Discovery records one controller-owned aim.eai.amd.com/profile-yaml-contract JSON annotation containing:
the codec identifier;
the source presence of fields AIM Engine models; and
source-owned metadata or top-level extensions AIM Engine does not model.
Profile sets, custom-model derivation, and AIMService overlays inherit the complete contract, so eager and lazy runtime projection reproduce the same field family and retain runtime-only additions such as metadata.capabilities. AIM Engine’s normalized CRD values remain authoritative: opaque extensions cannot replace identity, hardware, engine, metric, precision, type, engine_args, or env_vars.
metadata.features and metadata.capabilities are intentionally distinct. Features advertise runtime/container integration contracts, currently LoRA adapter support (adapters or adapters-scale-only). AIM Engine models features in AIMProfile.spec.features because service composition depends on them. Capabilities describe model/API behavior such as tool calling, structured outputs, and reasoning. AIM runtime validation consumes those flags, while AIM Engine currently preserves the complete capabilities block as an opaque source-owned extension. A profile may declare either field, both, or neither.
If a later AIM Engine release starts modeling a field that an older contract stored as an extension, the existing opaque value remains authoritative until the source profile is inspected again. Re-inspection moves the value into the canonical CRD and records modeled field presence. This avoids replacing preserved future data with a zero-valued typed field during an upgrade.
Source profiles may eventually declare a top-level profile_schema_version. Version 1 selects the current codec; profiles that omit it remain compatible with the established v1 document shape. Unknown explicit versions and unknown persisted codec identifiers fail explicitly rather than silently dropping fields. New structural formats can therefore add new codecs while old profiles continue using the codec recorded when they were discovered.
Generated and hand-authored profiles that have no source YAML provenance use the strict canonical accelerator_* v1 format. Discovered and derived profiles must carry their inspected contract; projection waits for producer backfill rather than guessing a source schema from annotation absence or an image tag. The annotation is diagnostic controller state, not a normal user-facing configuration knob.
Contracts are bounded to 64 KiB so they cannot consume Kubernetes’ aggregate annotation budget. Unknown contract-envelope fields, oversized contracts, unknown codecs, and malformed source metadata fail explicitly. Opaque extension keys remain unrestricted within the bounded extensions maps.
manual_selection_only is preserved only when the source schema contained that field. This is runtime-schema compatibility, not resolver behavior: spec.manualSelectionOnly remains deprecated and ignored for profile selection.
Filtering profiles by aimId (field selector)#
AIMProfile and AIMClusterProfile expose spec.aimId as a selectable field, so clients can filter profiles by model architecture server-side with a field selector instead of listing everything and filtering client-side:
# Cluster profiles for one model architecture
kubectl get aimclusterprofile --field-selector spec.aimId=qwen/qwen3-32b
# Namespace profiles for one model architecture
kubectl get aimprofile -n ml-team --field-selector spec.aimId=qwen/qwen3-32b
This is the recommended way for the management UI (and any API consumer) to scope a profile listing to a single model — it pushes the filter to the API server, avoiding a full list + client-side scan. Selectable fields on custom resources require Kubernetes 1.32+ (the CustomResourceFieldSelectors feature), which matches the project’s minimum supported version (see Prerequisites).
Only spec.aimId is wired as a selectable field today. Other axes (spec.engine, spec.precision, spec.acceleratorModel, status.deployable, …) are natural candidates and can be added the same way (Kubernetes allows up to 8 selectable fields per CRD version).
Profile specification#
apiVersion: aim.eai.amd.com/v1alpha2
kind: AIMClusterProfile
metadata:
name: qwen-qwen3-32b-mi300x-lat-fp8
spec:
aimId: qwen/qwen3-32b
modelId: qwen/qwen3-32b-fp8
engine: vllm
metric: latency
precision: fp8
type: optimized
primary: true
acceleratorVendor: amd
acceleratorModel: MI300X
acceleratorType: gpu
acceleratorCount: 1
resources:
requests:
cpu: "4"
memory: 32Gi
image: amdenterpriseai/aim-qwen-qwen3-32b:0.8.5
engineArgs:
distributed_executor_backend: mp
gpu-memory-utilization: "0.95"
engineEnv:
VLLM_DO_NOT_TRACK: "1"
modelSources:
- modelId: qwen/qwen3-32b-fp8
sourceUri: hf://qwen/qwen3-32b-fp8
Spec fields#
Field |
Description |
|---|---|
|
Model architecture identifier (e.g. |
|
Specific model variant or weights identifier (e.g. |
|
On-disk profile identifier from the AIM image (e.g. |
|
Inference engine ( |
|
Optional runtime recipe suffix for profiles that otherwise share the same engine/hardware/precision/count/metric tuple (for example |
|
Optimization target: |
|
Numeric precision: |
|
Optimization level. Hierarchy: |
|
Automatic-selection policy: |
|
Marks the recommended default for this model + hardware combination. Boosts ranking during automatic selection. Default |
|
Deprecated and ignored by the resolver. Use |
|
Accelerator device-plugin contract: |
|
Accelerator identifier for node selection (e.g. |
|
|
|
Number of accelerator units required. Combined with |
|
Optional override for K8s |
|
Required. Deployment container image. For purpose-built profiles: the full AIM image. For overlay-produced custom-model profiles: the base image. |
|
Optional inference-engine tuning and overrides as a free-form JSON object. Typed values (ints, floats, booleans, strings) are preserved and projected into engine-specific CLI flags. |
|
Environment variables passed to the inference engine subprocess (distinct from |
|
Model artifact sources with download URIs. |
|
Container-level env vars on the pod spec. |
|
Secrets for pulling the deployment image. |
|
Workload service account. |
For engine-specific behavior—including shared-memory sizing and vLLM-Omni USP
variants—see Inference Engines. To require a specific
variant, select the profile by spec.profile.name; automatic selection does not
filter or rank on variant.
Namespace-scoped AIMProfile adds one more field:
Field |
Description |
|---|---|
|
When |
Accelerator and node affinity#
The accelerator fields jointly describe the hardware AIM Engine schedules onto. acceleratorVendor is sufficient to target any GPU from one vendor; acceleratorModel is an optional additional constraint for profiles that require a specific card.
spec.acceleratorVendor: amd
→ feature.node.kubernetes.io/aim-accelerator.vendor.GPU.AMD (Exists)
spec.acceleratorModel: MI300X
→ feature.node.kubernetes.io/aim-accelerator.MI300X (Exists)
spec.acceleratorType: gpu
spec.acceleratorCount: 1
→ AIM Engine computes amd.com/gpu: "1"
spec.acceleratorVendor: nvidia
spec.acceleratorType: gpu
spec.acceleratorCount: 1
→ AIM Engine computes nvidia.com/gpu: "1"
The AcceleratorDetector DaemonSet labels each node with its vendor and applicable identifiers (specific GPU model, CPU architecture). A generic profile with acceleratorVendor: nvidia matches NVIDIA GPU nodes of any model. Adding acceleratorModel: H100 narrows that profile to H100 nodes.
The label value (count) is informational only — the selector operator is Exists. Actual capacity is enforced via the computed device resource request.
Hardware support#
Optimized AIM images declare the concrete AMD hardware family they support:
Family |
Example |
|---|---|
AMD Instinct |
|
AMD Radeon |
|
AMD EPYC |
|
The hardware a profile targets is declared on the profile itself, via AIMProfile.spec.acceleratorVendor and, when required, AIMProfile.spec.acceleratorModel:
# AIMProfile (or AIMClusterProfile)
spec:
acceleratorVendor: amd
acceleratorModel: MI300X
You don’t normally author these profiles by hand — the AIMModel discovers the AIM image and publishes one profile per supported (hardware, precision, metric) combination, each stamped with the spec.acceleratorModel it was built for. See Where profiles come from. To target specific hardware you therefore pick among the already-discovered profiles rather than inventing a new accelerator value.
An AIMService selects a profile with that value through spec.profile.selector.acceleratorModel — the selector matches against the profiles’ spec.acceleratorModel and resolves the service to a profile carrying it:
# AIMService
spec:
model:
name: qwen-qwen3-32b
profile:
selector:
acceleratorModel: MI300X
Make sure the chosen model image actually supports the target hardware — an image built for AMD Instinct GPUs will not run on Radeon or EPYC. Profile resolution only matches an image to a node whose detected labels satisfy the profile’s spec.acceleratorModel; it does not transcode an unsupported image onto incompatible hardware. See Deploying Services — Model + selector for the full selector resolution flow.
Generic NVIDIA vLLM profiles#
A hand-authored profile can run an upstream vLLM image on any NVIDIA GPU model without pre-creating a profile per card:
apiVersion: aim.eai.amd.com/v1alpha2
kind: AIMProfile
metadata:
name: qwen2-5-0-5b-nvidia-vllm
spec:
aimId: Qwen/Qwen2.5-0.5B-Instruct
modelId: Qwen/Qwen2.5-0.5B-Instruct
engine: vllm
type: unoptimized
acceleratorVendor: nvidia
acceleratorType: gpu
acceleratorCount: 1
image: vllm/vllm-openai:v0.16.0
modelSources:
- modelId: Qwen/Qwen2.5-0.5B-Instruct
sourceUri: hf://Qwen/Qwen2.5-0.5B-Instruct
For this direct NVIDIA vLLM contract, AIM Engine generates vllm serve, points it at the locally mounted model artifact, sets the served model name, host, port, and tensor-parallel size, and forces Hugging Face offline mode. engineArgs is optional and is only needed for additional vLLM tuning or to override generated tuning values such as served-model-name and tensor-parallel-size; the model path, host, and port remain controller-managed.
The profile itself does not need spec.caching. An AIMService selecting the profile creates or reuses the cache according to the service’s caching mode and does not create the InferenceService until the model artifact is ready. Because type: unoptimized is below the default profile-selection floor, a selector using this reference profile must explicitly set minimumType: unoptimized (or any).
Partitioned GPUs#
The vendor resolver is also the extension point for partition-specific device-plugin resources. In the current implementation, AMD partition modes still derive amd.com/gpu; a later resource-naming strategy can map a concrete mode such as CPX-NPS4 to amd.com/cpx_nps4 without changing profile selection or runtime projection.
For recognized AMD compute partition modes, generated host CPU and memory
defaults scale with slice density: SPX=1, DPX=2, QPX=4, and CPX=8 slices per
physical GPU. For example, eight CPX-NPS4 accelerator units receive the same
aggregate host defaults as one unpartitioned GPU. The geometry-agnostic
partitioned value and unknown schemes do not receive host defaults; specify
resources explicitly for those profiles.
Until partition-specific resource mapping is enabled, a cluster that advertises such a resource can override the derived device resource explicitly:
spec:
acceleratorVendor: amd
acceleratorModel: MI300X
acceleratorType: gpu
acceleratorCount: 0
resources:
requests:
amd.com/cpx_nps4: "4"
acceleratorCount: 0 suppresses the default amd.com/gpu derivation so the override stands alone.
CPU-only profiles#
Profiles without accelerator fields are treated as CPU-only and are always Ready (no node-affinity gating).
spec:
aimId: microsoft/phi-2
image: amdenterpriseai/aim-phi-2:0.8.5
engine: vllm
metric: latency
precision: fp32
Profile status#
Field |
Description |
|---|---|
|
|
|
|
|
|
|
Base image reference extracted from the discovery-cache metadata ( |
|
|
|
Extracted from |
|
Count of cluster nodes matching the accelerator label and resource requests |
|
Human-readable summary ( |
|
Definitive |
|
Computed node affinity rules |
|
Standard Kubernetes conditions |
Status lifecycle#
Pending — profile created, not yet reconciled
Ready — at least one cluster node matches the accelerator label and has sufficient resource capacity. CPU-only profiles are always
Ready.Degraded — transient error (e.g. failed to list nodes); will be retried.
NotAvailable — no cluster nodes match the requirements. Auto-recovers when matching nodes are added.
Conditions#
HardwareAvailable reports whether the cluster has nodes matching the profile’s requirements:
Status |
Reason |
Description |
|---|---|---|
|
|
Matching nodes present |
|
|
No accelerator requirements — always available |
|
|
No matching nodes |
The profile controller watches node events and re-evaluates hardware availability whenever node labels change.
Primary profiles#
spec.primary: true marks a profile as the recommended default for its (model + accelerator + precision + metric) combination. AIM image authors stamp this on the profile they consider the production sweet spot for the hardware.
When a service uses spec.model.name resolution (which produces multiple candidate profiles), primaries are ranked above non-primaries during selection. Non-primary profiles remain addressable by explicit name.
Optimization tier and the minimumType floor#
A profile’s spec.type (optimized > general > preview > unoptimized) is both
a ranking signal and a selection gate. Automatic selection
(spec.model.name / selector resolution) first considers profiles whose type
is at or above the selector’s spec.profile.selector.minimumType floor. For
AIMServices that floor defaults to optimized, so lower-tier profiles are
excluded from the optimized pass unless the service opts in:
# AIMService — accept lower tiers (e.g. an EPYC unoptimized profile)
spec:
model:
name: aim-epyc-llama-3-2-1b-zen4-lowmem
profile:
selector:
minimumType: any # or "unoptimized"; default would be "optimized"
Any profile — including lower tiers — also remains addressable by explicit spec.profile.name, which bypasses the floor entirely.
spec.autoSelectionPolicy: any is a platform/profile-publisher opt-in for
automatic selection regardless of optimization tier. If the service did not
explicitly set minimumType and no optimized profile matched in either scope,
the resolver performs a second pass containing only any profiles. This allows
a generic runtime such as the chart-provided NVIDIA vLLM profile to remain
truthfully unoptimized without requiring every service to lower its floor.
The default policy is optimized, so existing AMD preview and unoptimized
profiles remain excluded.
Warning
manualSelectionOnly is deprecated
spec.manualSelectionOnly is no longer honored. Use type + minimumType for
service-authored optimization-tier consent and autoSelectionPolicy for
platform-authored automatic eligibility. The field is still accepted for
backward compatibility but has no effect and will be removed in a future API
version.
Examples#
Cluster profile — latency-tuned#
apiVersion: aim.eai.amd.com/v1alpha2
kind: AIMClusterProfile
metadata:
name: qwen-qwen3-32b-mi300x-lat-fp8
spec:
aimId: qwen/qwen3-32b
modelId: qwen/qwen3-32b-fp8
engine: vllm
metric: latency
precision: fp8
type: optimized
primary: true
acceleratorModel: MI300X
acceleratorType: gpu
acceleratorCount: 1
image: amdenterpriseai/aim-qwen-qwen3-32b:0.8.5
engineArgs:
gpu-memory-utilization: "0.95"
Namespace profile — custom weights with caching#
apiVersion: aim.eai.amd.com/v1alpha2
kind: AIMProfile
metadata:
name: my-finetuned-qwen-mi300x-lat-fp8
namespace: ml-team
spec:
aimId: qwen/qwen3-32b
modelId: my-org/qwen-finetuned-fp8
engine: vllm
metric: latency
precision: fp8
type: general
acceleratorModel: MI300X
acceleratorType: gpu
acceleratorCount: 2
resources:
requests:
cpu: "8"
memory: 64Gi
image: amdenterpriseai/aim-base:0.8.5
engineArgs:
tensor-parallel-size: "2"
modelSources:
- modelId: my-org/qwen-finetuned-fp8
sourceUri: s3://my-bucket/fp8-weights/
caching:
enabled: true
Troubleshooting#
Profile stuck in NotAvailable#
The profile’s accelerator requirements don’t match any cluster node.
kubectl get aimclusterprofile <name> -o jsonpath='{.status.resolvedNodeAffinity}' | jq
kubectl get aimclusterprofile <name> -o jsonpath='{.status.resources}' | jq
kubectl get aimclusterprofile <name> -o jsonpath='{.status.matchingNodes}'
# Nodes carrying the expected accelerator label
kubectl get nodes -l feature.node.kubernetes.io/aim-accelerator.MI300X
Common causes: GPU nodes not yet added, AcceleratorDetector not labeling nodes, wrong acceleratorModel value.
source-model label disappears after reconcile#
Expected for user-authored profiles. See Hand-authored profiles cannot fake provenance above.
Profile is Ready but a service can’t resolve to it#
If the service uses spec.profile.selector or spec.model.name, the selector may not match this profile. Check:
The profile’s
status.deployableistrue(base profiles are never returned).The profile carries
profile-role: deployable.For
modelRef.nameselectors, the profile carriessource-model: <model-name>. User-authored profiles don’t carry this label — they’re reachable only byaimId/modelId.