AIM Engine

AIM Engine#

AIM (AMD Inference Microservice) Engine is a Kubernetes operator that simplifies the deployment and management of AI inference workloads on AMD GPUs. It provides a declarative, cloud-native approach to running ML models at scale.

Quick example#

Deploy an inference service in two resources:

apiVersion: aim.eai.amd.com/v1alpha2
kind: AIMClusterModel
metadata:
  name: qwen3-32b
spec:
  image: amdenterpriseai/aim-qwen-qwen3-32b:0.8.5
---
apiVersion: aim.eai.amd.com/v1alpha2
kind: AIMService
metadata:
  name: qwen-chat
  namespace: ml-team
  annotations:
    aim.eai.amd.com/reconciler-pipeline: profile
spec:
  model:
    name: qwen3-32b

AIM images (like amdenterpriseai/aim-qwen-qwen3-32b) package open-source models optimized for AMD Instinct GPUs. Each image includes the model weights and a serving runtime tuned for specific GPU configurations and precision modes.

The AIMModel runs discovery on the image and publishes one AIMProfile per supported (GPU, precision, metric) combination. The AIMService resolves to the best deployable profile for your hardware, pre-warms the model cache, and creates a KServe InferenceService.

Why the reconciler-pipeline: profile annotation?

AIMService dispatch is decided by spec shape, not by apiVersion. During the v1alpha1 → v1alpha2 migration window, spec.model.name and spec.model.image default to the legacy template pipeline so existing deployments keep working unchanged. The annotation forces this service onto the v1alpha2 profile pipeline, which resolves qwen3-32b to one of the AIMClusterProfiles produced by the AIMClusterModel above. The annotation becomes unnecessary once v1alpha1 is removed — see Migration window for the full dispatch table.

Three model flows#

How you onboard a model depends on its relationship to AMD’s published catalog:

Flow

Use when

Read more

Official

Deploying a published AMD-supported AIM model unmodified

AIM Models

Fine-tuned

Deploying a fine-tune of a published architecture

Fine-Tuned Models

Custom

Deploying a model whose architecture isn’t in the catalog

Custom Models

Where to start#

Cluster administrators

Install AIM Engine, configure KServe, manage GPU resources, and set up cluster-wide defaults.

Installation
Developers & integrators

Deploy inference services, configure scaling, set up routing, integrate with your applications.

Quickstart
Data scientists

Browse the model catalog, deploy fine-tunes or custom models, tune inference parameters.

Model Catalog

Key features#

  • Three-flow model onboarding — official AIM images, fine-tuned models, and custom models that bring their own weights, all expressed through a single AIMModel shape.

  • Profile-driven deployment — AIMService resolves to a self-contained AIMProfile with everything the runtime needs (image, accelerator, engine config, model sources).

  • Smart selection — pick a profile by name, by model, by selector, or by model+selector; the controller ranks candidates by primary > type > version.

  • Profile overlays — spec.profileOverrides rebases a published profile onto custom weights without forking it.

  • Model caching — pre-download artifacts to shared PVCs for faster startup; HuggingFace downloader falls back across XET / HF_TRANSFER / HTTP protocols.

  • HTTP routing — expose services through Gateway API with customizable path templates.

  • Autoscaling — KEDA integration with OpenTelemetry metrics for demand-based scaling.

  • Multi-tenancy — namespace-scoped and cluster-scoped resources for flexible team isolation.

Documentation#

Getting started#

Guides#

Task-oriented walkthroughs for common workflows:

Administration#

Concepts#

Reference#

Legacy (v1alpha1)#

Contents