Custom Models

Custom Models#

A custom model lets you deploy a model whose architecture isn’t published in the AMD AIM catalog. You combine a generic AIM base image — which supplies the runtime, engine, and tested accelerator profiles — with your own model weights pulled from S3, HuggingFace, or any other supported source.

v1alpha2

Custom models use the aim.eai.amd.com/v1alpha2 API. The v1alpha1 spec.custom / spec.modelSources flow on AIMModel is deprecated and removed from v1alpha2.

When to use this flow#

Goal

Use this flow?

Deploy a model whose architecture is in the AMD AIM catalog (with your own fine-tune weights)

No — use Fine-Tuned Models

Deploy a model whose architecture is not in the AMD AIM catalog

Yes

Deploy a published AIM image without modification

No — apply the AIMModel and you’re done

Mental model#

The flow has two AIMModels working together:

Custom-model flow: a base-image AIMModel's base profiles are consumed by a custom-model AIMModel that emits deployable profiles backing an AIMService. Custom-model flow: a base-image AIMModel's base profiles are consumed by a custom-model AIMModel that emits deployable profiles backing an AIMService.

  1. Base-image model — points spec.image at an AIM base image and produces base AIMProfiles: one per accelerator/precision combination the base image ships, with no aimId or modelSources populated. Base profiles are not deployable on their own — they exist only as derivation sources.

  2. Custom-model AIMModel — uses spec.profiles.derivedFrom to select those base profiles and overlay your model identity and weights. The result is a set of regular deployable AIMProfiles that an AIMService can resolve through.

The base-image model is typically applied once by the platform team and reused across many custom-model derivations.

Prerequisites#

To use a container as a base image for custom models, it must:

  • Ship profile YAMLs under /workspace/aim-runtime/profiles/general/. The v1alpha2 discovery walker only scans general/ when the image’s AIM_ID is empty (which is the case for a generic base image).

  • Leave aim_id, model_id, and model sources empty in those YAMLs. A materialized base AIMProfile must have both spec.aimId and spec.modelSources empty; setting only one side is rejected by the profile CRD. The derivation supplies the target identity through overrides.aimId / overrides.modelId and the weights through overrides.modelSources (see Identity stamping below).

Base profile YAMLs do carry precision, metric, accelerator metadata, and engine_args — these are intrinsic tuning artefacts of the base image and the whole reason a base profile is useful as a derivation source. Accelerator metadata may use legacy gpu / gpu_count fields or canonical accelerator_model / accelerator_type / accelerator_count fields. Discovery normalizes either shape into AIMProfile.spec while recording the source codec, modeled-field presence, and any runtime-only extensions. Derivation copies both the normalized values and that YAML contract, so custom profiles retain additions such as metadata.capabilities as well as the accelerator format accepted by the base runtime.

AMD publishes amdenterpriseai/aim-base as a ready-to-use base image. You can also build your own following the layout described above.

Step 1: Apply the base-image AIMModel#

apiVersion: aim.eai.amd.com/v1alpha2
kind: AIMModel
metadata:
  name: aim-base-vllm
  namespace: ml-team
spec:
  image: amdenterpriseai/aim-base:0.11
  imagePullSecrets:
    - name: registry-creds
  serviceAccountName: aim-runtime

Discovery runs as a Kubernetes Job that inspects the image and writes its profile catalog into a ConfigMap (referenced by status.discoveryCacheRef). For each supported profile in the catalog, the model controller creates a base AIMProfile.

Wait for discovery to complete and base profiles to materialise:

kubectl -n ml-team get aimmodel aim-base-vllm \
  -o jsonpath='{.status.managedProfiles}'
# {"deployable":0,"notAvailable":0,"ready":8,"base":8,"total":8}

base > 0 and deployable == 0 is the canonical “this is a base-image model” signal.

The base profiles themselves carry distinct labels and status:

kubectl -n ml-team get aimprofile \
  -l aim.eai.amd.com/source-model=aim-base-vllm \
  -o custom-columns=NAME:.metadata.name,ROLE:.metadata.labels.aim\\.eai\\.amd\\.com/profile-role,DEPLOYABLE:.status.deployable
# NAME                                 ROLE  DEPLOYABLE
# aim-base-vllm-general-fp8-lat-...    base  false
# aim-base-vllm-general-fp16-lat-...   base  false
# ...

Step 2: Apply the custom-model AIMModel#

apiVersion: aim.eai.amd.com/v1alpha2
kind: AIMModel
metadata:
  name: acme-custom-transformer
  namespace: ml-team
spec:
  profiles:
    derivedFrom:
      selector:
        role: base
        modelRef:
          name: aim-base-vllm
          scope: Namespace
    versionPolicy: all
    overrides:
      aimId: acme/custom-transformer
      modelId: acme/custom-transformer
      modelSources:
        - modelId: acme/custom-transformer
          sourceUri: s3://acme-models/custom-transformer
  imagePullSecrets:
    - name: registry-creds
  serviceAccountName: aim-runtime

The spec splits cleanly into “what to filter” (selector) and “what to stamp on the derived profile” (overrides):

Field

Half

Purpose

selector.role: base

filter

Match only base profiles. Required — without this, only already-deployable profiles are considered and no base profiles match.

selector.modelRef.name

filter

Pin the derivation to one specific base-image AIMModel. Without this, every base-image model in scope contributes candidates.

overrides.aimId

stamp

Target architecture identifier for the derived profile. Required by CEL when selector.role=base — base profiles ship with empty identity.

overrides.modelId

stamp

Target model identifier. Required by CEL when selector.role=base.

overrides.modelSources

stamp

The BYO weights that complete the base into a deployable profile.

CEL validation on AIMModelProfilesSpec enforces the split: setting selector.aimId, selector.modelId, or selector.profileId when selector.role=base is rejected at admission time. Identity fields on the selector were a filter on the source (matching the source’s identity); base profiles by design have no source identity, so the field is meaningless there.

When overrides.modelId is unset and overrides.modelSources[0].modelId is, the derived profile’s spec.modelId auto-derives from the latter. The explicit override always wins, and is required for role=base regardless of modelSources.

Wait for the derivation to produce deployable profiles:

kubectl -n ml-team get aimmodel acme-custom-transformer \
  -o jsonpath='{.status.managedProfiles}'
# {"deployable":8,"notAvailable":0,"ready":8,"base":0,"total":8}

kubectl get aimmodel shows the resolved flow under the KIND column — a correctly-configured custom model surfaces as Custom. If you see Image there, your spec is using spec.image (the official flow) rather than spec.profiles.derivedFrom; if you see Derived, the selector is matching deployable rather than base profiles. See status.kind for the full discriminator table.

Step 3: Deploy with an AIMService#

apiVersion: aim.eai.amd.com/v1alpha2
kind: AIMService
metadata:
  name: acme-custom-transformer-service
  namespace: ml-team
  annotations:
    aim.eai.amd.com/reconciler-pipeline: profile
spec:
  model:
    name: acme-custom-transformer

The AIMService resolver treats spec.model.name as a shortcut for “every deployable profile produced by this model” and applies the shared deterministic profile ranking. The annotation selects the profile pipeline during the v1alpha1 → v1alpha2 migration window. See Deploying Services for the ranking order and other resolution shapes (by-profile-name, by-selector, with profile overlay).

Identity stamping#

Target identity for the derived profile lives entirely in overrides. The selector is filter-only.

Field

Behaviour

Required when role=base?

overrides.aimId

Stamped on derived profile. Wins over the source’s spec.aimId if both are set.

Yes (CEL)

overrides.modelId

Stamped on derived profile. Wins over the source’s spec.modelId AND over modelSources[0].modelId auto-derivation.

Yes (CEL)

overrides.profileId

Stamped on derived profile. Wins over the source’s spec.profileId.

No

For role=deployable remixes (where you’re re-derivating an already-deployable profile to e.g. expand across accelerator variants), all three identity overrides are optional — the source profile’s existing identity carries through if you don’t override it. For role=base derivations they’re required (well, aimId and modelId) because the source has no identity to inherit.

This is enforced at admission time by CEL on both AIMProfileSetSpec and AIMModelProfilesSpec:

selector.aimId/modelId/profileId are not allowed when selector.role=base;
set overrides.aimId/modelId/profileId instead (base profiles have no
source identity to filter on)

selector.role=base requires overrides.aimId and overrides.modelId
(base profiles carry no identity of their own; overrides supply it)

So an attempt to set selector.aimId under selector.role=base is rejected by the API server before reaching the controller.

Common patterns#

Filter base profiles by accelerator#

A base image typically ships profiles for several GPUs. Narrow the derivation to just the ones you want:

spec:
  profiles:
    derivedFrom:
      selector:
        role: base
        modelRef:
          name: aim-base-vllm
        acceleratorModel: MI300X
        acceleratorCount: 2
    overrides:
      aimId: acme/custom-transformer
      modelId: acme/custom-transformer
      modelSources:
        - modelId: acme/custom-transformer
          sourceUri: s3://acme-models/custom-transformer

Pin to a specific precision#

spec:
  profiles:
    derivedFrom:
      selector:
        role: base
        modelRef:
          name: aim-base-vllm
        precision: fp8
        metric: latency
    overrides:
      aimId: acme/custom-transformer
      modelId: acme/custom-transformer
      modelSources:
        - modelId: acme/custom-transformer
          sourceUri: s3://acme-models/custom-transformer

Authenticate to private weights#

overrides.modelSources[].env accepts the same credential references as a normal profile:

spec:
  profiles:
    derivedFrom:
      selector:
        role: base
        modelRef:
          name: aim-base-vllm
    overrides:
      aimId: acme/custom-transformer
      modelId: acme/custom-transformer
      modelSources:
        - modelId: acme/custom-transformer
          sourceUri: s3://acme-models/custom-transformer
          env:
            - name: AWS_ACCESS_KEY_ID
              valueFrom:
                secretKeyRef:
                  name: s3-credentials
                  key: access-key
            - name: AWS_SECRET_ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  name: s3-credentials
                  key: secret-key

Cluster-scoped variant#

For platform-managed custom models that should be visible across namespaces, use AIMClusterModel with the same spec.profiles shape. The derivation will produce AIMClusterProfile objects instead of namespace-scoped ones, and modelRef.scope can be set to Cluster to pin to a cluster-scoped base-image model.

Troubleshooting#

Base-image model status shows base: 0#

Discovery hasn’t found any profiles to materialise. Common causes:

  • The base image’s profile YAMLs aren’t under /workspace/aim-runtime/profiles/general/.

  • The image has AIM_ID set, so discovery is looking under $AIM_ID/ instead of general/.

  • The profile YAMLs populate model_id or model_sources — they’re treated as deployable, not base.

Inspect the discovery cache:

kubectl -n ml-team get aimmodel aim-base-vllm \
  -o jsonpath='{.status.discoveryCacheRef.name}'
# aim-base-vllm-discovery-<hash>
kubectl -n ml-team get configmap aim-base-vllm-discovery-<hash> -o yaml

Custom-model AIMModel is NotAvailable with NoMatchingProfiles#

The selector matched zero base profiles. Verify:

# Check that base profiles exist under the modelRef
kubectl -n ml-team get aimprofile \
  -l aim.eai.amd.com/source-model=aim-base-vllm,aim.eai.amd.com/profile-role=base

# Check the derivation's profile set
kubectl -n ml-team get aimmodel acme-custom-transformer \
  -o jsonpath='{.status.profileSetRef.name}'
kubectl -n ml-team get aimprofileset <profileset-name> -o yaml

Common causes:

  • selector.modelRef.name doesn’t match the base-image model’s metadata name.

  • selector.modelRef.scope mismatches the base-image model’s scope (use Auto if unsure).

  • An accelerator/precision filter excluded every candidate.

AIMService stuck in Pending with ProfileNotFound#

Confirm the derivation produced deployable profiles before debugging the service:

kubectl -n ml-team get aimmodel acme-custom-transformer \
  -o jsonpath='{.status.managedProfiles}'

If deployable == 0, fix the derivation first. If deployable > 0, the service-side resolver isn’t finding them — check that the service’s spec.model.name matches the custom-model AIMModel’s name exactly, and that any spec.profile.selector you set doesn’t narrow more than the available profiles.

Next Steps#

  • Fine-Tuned Models — Derive from existing official AIM profiles instead of base-image base profiles

  • Deploying Services — Resolution shapes and overlay patterns for AIMService

  • AIM Models — Full lifecycle of all four model flows

  • AIM Profile Sets — The derivation machinery underneath spec.profiles

  • Profiles — Base vs deployable, provenance labels, status fields

Contents