Quickstart

Quickstart#

Deploy your first inference service in minutes.

Prerequisites#

  • AIM Engine installed on your cluster

  • AMD GPUs available in the cluster (or a CPU-only profile for testing)

  • kubectl configured to access your cluster

Step 1: Apply a model#

Apply an AMD-published AIM model. This is the official flow — spec.image points at the AIM container image and discovery materialises deployable profiles.

apiVersion: aim.eai.amd.com/v1alpha2
kind: AIMClusterModel
metadata:
  name: qwen3-32b
spec:
  image: amdenterpriseai/aim-qwen-qwen3-32b:0.8.5

kubectl apply -f model.yaml

If you set up model discovery during installation, official models are already in the catalog:

kubectl get aimclustermodels

Wait for discovery to complete and at least one deployable profile to land:

kubectl get aimclustermodel qwen3-32b -o jsonpath='{.status.managedProfiles}'
# {"deployable":5,"notAvailable":0,"ready":5,"base":0,"total":5}

Step 2: Deploy an AIMService#

Reference the model and let the controller pick the best deployable profile for your hardware.

apiVersion: aim.eai.amd.com/v1alpha2
kind: AIMService
metadata:
  name: qwen-chat
  namespace: default
spec:
  model:
    name: qwen3-32b

kubectl apply -f service.yaml

AIM Engine then:

  1. Resolves the model to a deployable AIMProfile (ranking by primary > type > version).

  2. Pre-warms the cache by creating an AIMProfileCache and downloading model artifacts to a PVC.

  3. Creates a KServe InferenceService mounting the cache and the profile’s container image.

  4. Optionally creates an HTTPRoute if routing is enabled.

Step 3: Monitor progress#

kubectl get aimservice qwen-chat -w

The status progresses through Pending → Starting → Running. The service pauses in Starting while the cache fills (this can take several minutes for large models).

For detail:

kubectl get aimservice qwen-chat -o jsonpath='{.status.conditions}' | jq

The key conditions to watch:

  • ProfileReady → ProfileResolved

  • ProfileCacheReady → CacheReady

  • InferenceServiceReady → RuntimeReady

  • Ready → AllComponentsReady

Step 4: Send a request#

The InferenceService name is derived from the service — look it up by label:

kubectl get inferenceservice -n default -l aim.eai.amd.com/service.name=qwen-chat

Port-forward the predictor service:

kubectl port-forward -n default svc/<isvc-name>-predictor 8080:80

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen-chat",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Variations#

Pick a specific profile#

Bypass automatic selection by referencing a profile directly:

spec:
  profile:
    name: qwen-qwen3-32b-mi300x-fp8-latency

Narrow by precision or accelerator#

spec:
  model:
    name: qwen3-32b
  profile:
    selector:
      precision: fp8
      acceleratorModel: MI300X
      metric: latency

Dedicated cache#

The default Shared caching mode reuses the PVC across services. Use Dedicated for per-service isolation:

spec:
  model:
    name: qwen3-32b
  caching:
    mode: Dedicated

See Model Caching for the full lifecycle.

Next steps#

Contents