Quickstart

Quickstart#

Deploy your first inference service in minutes.

Prerequisites#

  • AIM Engine installed on your cluster

  • AMD GPUs available in the cluster (or a CPU-only profile for testing)

  • kubectl configured to access your cluster

Step 1: Apply a model#

Apply an AMD-published AIM model. This is the official flow — spec.image points at the AIM container image and discovery materialises deployable profiles.

apiVersion: aim.eai.amd.com/v1alpha2
kind: AIMClusterModel
metadata:
  name: qwen3-32b
spec:
  image: amdenterpriseai/aim-qwen-qwen3-32b:0.8.5

kubectl apply -f model.yaml

If you set up model discovery during installation, official models are already in the catalog:

kubectl get aimclustermodels

Wait for discovery to complete and at least one deployable profile to land:

kubectl get aimclustermodel qwen3-32b -o jsonpath='{.status.managedProfiles}'
# {"deployable":5,"notAvailable":0,"ready":5,"base":0,"total":5}

Step 2: Deploy an AIMService#

Reference the model and let the controller pick the best deployable profile for your hardware.

apiVersion: aim.eai.amd.com/v1alpha2
kind: AIMService
metadata:
  name: qwen-chat
  namespace: default
  annotations:
    aim.eai.amd.com/reconciler-pipeline: profile
spec:
  model:
    name: qwen3-32b

kubectl apply -f service.yaml

AIM Engine then:

  1. Resolves the model to the highest-ranked deployable AIMProfile for the available hardware.

  2. Pre-warms the cache by creating an AIMProfileCache and downloading model artifacts to a PVC.

  3. Creates a KServe InferenceService mounting the cache and the profile’s container image.

  4. Optionally creates an HTTPRoute if routing is enabled.

The reconciler-pipeline: profile annotation is required for model-shaped services during the v1alpha1 → v1alpha2 migration window. See Spec-shape dispatch.

Step 3: Monitor progress#

kubectl get aimservice qwen-chat -w

The status progresses through Pending → Starting → Running. The service pauses in Starting while the cache fills (this can take several minutes for large models).

For detail:

kubectl get aimservice qwen-chat -o jsonpath='{.status.conditions}' | jq

The key conditions to watch:

  • ProfileReady → ProfileResolved

  • ProfileCacheReady → CacheReady

  • InferenceServiceReady → RuntimeReady

  • Ready → AllComponentsReady

Step 4: Send a request#

The InferenceService name is derived from the service — look it up by label:

kubectl get inferenceservice -n default -l aim.eai.amd.com/service.name=qwen-chat

Port-forward the predictor service:

kubectl port-forward -n default svc/<isvc-name>-predictor 8080:80

Discover the model ID registered by the serving runtime:

MODEL_ID=$(curl --fail --silent --show-error http://localhost:8080/v1/models | jq -er '.data[0].id')
echo "${MODEL_ID}"
# Qwen/Qwen3-32B

Use the returned id as the model value in inference requests. The served model ID is typically the model’s canonical Hugging Face name, not the AIMService resource name, and varies with the deployed AIM image.

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"${MODEL_ID}\",
    \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]
  }"

Variations#

Pick a specific profile#

Bypass automatic selection by referencing a profile directly:

spec:
  profile:
    name: qwen-qwen3-32b-mi300x-fp8-latency

Narrow by precision or accelerator#

spec:
  model:
    name: qwen3-32b
  profile:
    selector:
      precision: fp8
      acceleratorModel: MI300X
      metric: latency

Dedicated cache#

The default Shared caching mode reuses the PVC across services. Use Dedicated for per-service isolation:

spec:
  model:
    name: qwen3-32b
  caching:
    mode: Dedicated

See Model Caching for the full lifecycle.

Next steps#

Contents