Deploy and Configure AMD Solution Blueprints

Deploy and Configure AMD Solution Blueprints#

This guide walks you through deploying and configuring the Agentic Translation Solution Blueprint. By the end of this guide, you will be able to:

  • Swap the default AIM LLM (Llama 3.3 70B Instruct) for an alternative model (Qwen3-32B)

  • Adjust hardware configurations to match workload requirements

Prerequisites#

  • Kubernetes access with permission to create resources in a namespace

    • The Kubernetes namespace in this guide is called demo

  • kubectl and Helm installed and configured for your cluster

  • Optional: k9s or similar for inspecting workloads

  • Hardware: The guide was validated on an AMD Instinct GPU, see the documented requirements for more information

Hugging Face Token#

AIM images are hosted publicly on Docker Hub and do not require authentication to pull. However, certain models are gated on Hugging Face and require an access token to download. Store your token as a Kubernetes secret so it can be referenced securely by the deployment.

Create a secret for the demo namespace:

kubectl create secret generic hf-token \
    --from-literal="hf-token=YOUR_HUGGINGFACE_TOKEN" \
    -n demo

secret/hf-token created

Overview#

Depending on your requirements, you may want to configure the Solution Blueprint. For example, which AIM to use. Because each blueprint is packaged as a Helm chart, you can override the defaults in values.yaml to adjust the configuration.

In this section, we change the default AIM using the --set flag method; this tells Helm to merge your values with the default values. This allows you to change settings without editing the original chart. Alternatively, you could clone the public repository and modify the files directly, but for simplicity, we use the flag.

For the upcoming adjustments, we assume you have access to the public repository. For this guide, the following two folders are important:

  • aimcharts/aimchart-llm: Deploys AMD Inference Microservice (AIM) LLMs. This chart includes functionality designed to be used as a subchart (a dependency in a larger application such as a Solution Blueprint). If you wish to understand the AIM deployment, this folder is a good starting point.

  • solution-blueprints: Contains all available Solution Blueprints, with corresponding code, documentation, and reference architectures.

An alias can be assigned to any subchart that is added as a dependency to your Solution Blueprint chart. For the Agentic Translation blueprint, the AIM subchart is aliased as llm, as shown in the Chart.yaml file. This alias then becomes the key name used to override AIM-related settings.

Note

While the upcoming customizations are valid for the Agentic Translation Solution Blueprint, other Solution Blueprints may require different adjustments. Before customizing a Solution Blueprint, review its corresponding documentation.

Deploying Agentic Translation#

The Agentic Translation Solution Blueprint deploys an AIM running Llama 3.3 70B Instruct by default.

Let’s replace the default Llama 3.3 70B Instruct model with Qwen3-32B and adjust the model precision. First, we need to identify the LLM target image from the AIM Catalog, in this case amdenterpriseai/aim-qwen-qwen3-32b:0.11.1. The version number might change over time. Check the AIM Catalog for the latest version.

Next, generate the manifest with --set flags to override the default model. Here, llm.image specifies the new AIM image, llm.nameOverride sets the name for the AIM resources and llm.env_vars.AIM_PRECISION sets the model precision. For the sake of this guide set the model precision to fp16. To see available profiles, visit Qwen3-32B AIM configuration in the AIM GitHub repository.

name="translator-qwen"
namespace="demo"
chart="aimsb-agentic-translation"
helm template $name oci://registry-1.docker.io/amdenterpriseai/$chart \
--set llm.image=amdenterpriseai/aim-qwen-qwen3-32b:0.11.1 \
--set llm.env_vars.AIM_PRECISION=fp16 \
--set llm.env_vars.HF_TOKEN.name=hf-token \
--set llm.env_vars.HF_TOKEN.key=hf-token \
--set llm.nameOverride=qwen3-32b > at-qwen-deployment.yaml

Defaults for each hardware platform are defined under llm.platformDefaults.<platform> in values.yaml (for example llm.platformDefaults.instinct.image). This guide assumes an AMD Instinct deployment, which is the default platform. Radeon is also supported, pass --set global.platform=radeon to select the matching defaults from llm.platformDefaults.radeon (image, GPU memory, CPU, and other resource settings). You can customize platform defaults directly, for example --set llm.platformDefaults.radeon.image=<image>.

Finally, apply the manifest to deploy the Solution Blueprint:

kubectl apply -f at-qwen-deployment.yaml -n $namespace

Verify the deployment of the new AIM with k9s or kubectl:

kubectl get deployments -n $namespace

NAME
aimsb-agentic-translation-translator-qwen
qwen3-32b-translator-qwen

If you need further confirmation, you can inspect the logs or describe the pod in k9s (e.g., press l or d while selecting the pod).

As a final check, feel free to port-forward and connect to the UI to confirm that it is working. Note, the first query may take longer to complete compared to subsequent queries.

kubectl port-forward services/aimsb-agentic-translation-$name 8501:8501 -n $namespace

Once done, delete the resources:

kubectl delete -f at-qwen-deployment.yaml -n $namespace

Customizing Multiple Parameters with an Override File#

In this section, we dive deeper into customizing the Agentic Translation Solution Blueprint by creating and passing an override file during deployment. Using an override file lets you make several changes to your deployment and have your configurations tracked in a .yaml file, without using --set flags. This enables you to use version control, increases readability, and makes it easier to track your changes. Providing an override file to the Helm command is done by using the -f flag; it tells Helm to merge the values from your override file over the default values.

The following table summarizes the changes for the AIM deployment:

AIM Settings

Default (Llama-3.3 70B Instruct)

Customized (Qwen3-32B)

Model

amdenterpriseai/aim-meta-llama-llama-3-3-70b-instruct

amdenterpriseai/aim-qwen-qwen3-32b

Precision

auto

fp16

GPU

1

1

CPU

4

8

Ephemeral storage

512Gi

350Gi

Create the Override File#

To apply the changes from the table above, create a file named blueprint-override.yaml with the following content:

llm:
  image: "amdenterpriseai/aim-qwen-qwen3-32b:0.11.1"
  env_vars:
    AIM_PRECISION: "fp16"
    HF_TOKEN:
      name: hf-token
      key: hf-token
  nameOverride: qwen3-32b
  cpu_per_gpu: 8
  storage:
    ephemeral:
      quantity: 350Gi

Deploying the Customized Solution Blueprint#

Generate the deployment manifest using the default command, but with the override file attached:

name="customized-translator"
namespace="demo"
chart="aimsb-agentic-translation"
helm template $name oci://registry-1.docker.io/amdenterpriseai/$chart \
-f blueprint-override.yaml > at-custom-deployment.yaml

Then, apply the manifest to deploy the Solution Blueprint:

kubectl apply -f at-custom-deployment.yaml -n $namespace

Verifying the Customization#

Confirm the customization by inspecting the deployed pod. First, find the AIM pod name:

kubectl get pods -n $namespace

Look for the pod whose name contains qwen3-32b, then describe it:

kubectl describe pod <YOUR-AIM-POD-NAME> -n $namespace

In the output, you should see the updated image and resource requests matching the values from the table:

Containers:
  aim-llm:
    ...
    Image:   amdenterpriseai/aim-qwen-qwen3-32b:0.11.1
    ...
    Requests:
      amd.com/gpu:  1
      cpu:          8

You can also verify the storage allocation by inspecting the PersistentVolumeClaim (PVC), which should show 350Gi:

kubectl get pvc -n $namespace | grep qwen3-32b

Clean Up#

Once done, delete the resources by running:

kubectl delete -f at-custom-deployment.yaml -n $namespace
Contents