Local AI with ROCm and Radeon: Reading Support Combinations and Runtime Paths

To run a PyTorch⁴ image task on a Radeon, first identify the exact GPU⁵ and operating system, then find a supported combination for both in AMD’s compatibility table. This guide follows the steps for finding an official route and testing a small task on one Radeon you already own. ROCm is the software stack used to run the work; HIP is a path for porting source code. A CUDA⁶ executable cannot simply be copied and run on an AMD GPU. Instinct accelerators are a separate product family, so this guide focuses on consumer Radeon cards.

ROCm is a software stack that brings together drivers, runtimes, compilers, math libraries and framework integrations so computations can run on AMD GPUs. Running PyTorch also requires a compatible PyTorch build, ROCm components and GPU-kernel support. So a document saying “ROCm supported” does not mean that every PyTorch version and every Radeon model is supported. The meaningful unit for an installation decision is the combination of GPU, OS, driver, ROCm release and framework specified in the documentation.

This is where HIP enters the picture; its name can look similar to CUDA. HIP is a C++ runtime API⁷ and GPU-kernel language, while HIPIFY⁸ helps convert some CUDA source API calls and notation into HIP form. Being able to port source code is different from running an already-compiled CUDA program as-is. A CUDA executable may contain code targeting NVIDIA GPUs and links to NVIDIA libraries. Copying it to a Radeon does not turn it into AMD code. Even with an open-source project, you still need a HIP port or ROCm build and AMD support for the required libraries. Automatic conversion can help you get started, but it does not validate missing features or device-specific behavior.

So the first decision is not a performance comparison between GPUs; it is whether your setup falls within the scope of the support documentation. AMD documents Radeon, Instinct and Ryzen APUs along separate paths. For the consumer Radeon GPUs covered here, check the ROCm on Radeon table. Instinct is a data-center accelerator family with its own server distributions, kernels and software combinations. Do not infer support for one product family because a GPU appears in another family’s table. A table defines support for a specified software configuration; it does not compare performance across every workload.

AMD unified its supported-hardware documentation starting with ROCm Core SDK 7.13.0, and the current 10.0 documentation uses that unified compatibility guidance. Do not decide that a particular Radeon is supported from the release overview alone. Check whether the table lists your GPU, OS, driver, ROCm release and framework together as one combination. The key is not to assemble versions from different rows or documentation cycles.

Your choice of operating system matters too. On Linux, you match the distribution, kernel and driver yourself. The native Windows Radeon path may cover a limited set of PyTorch configurations and tools explicitly listed in the documentation. WSL is not the same as native Windows; it has a separate setup such as ROCDXG and its own support matrix. Do not treat the three paths as interchangeable installation recipes. Choose the one where your familiar development environment overlaps with the official deployment path for the app you want to use. For example, if an app’s ROCm support on Windows applies only to a specific PyTorch version, do not simply apply Linux instructions to it.

Suppose you want to keep your current Windows PC and try a PyTorch image workflow. If the app provides an AMD package for Windows, start by checking that path. If it only provides Linux installation instructions, check WSL support separately. Copying Linux commands into WSL does not create the same environment unless the app says it supports that setup. Conversely, someone already using Linux does not need to switch to Windows first. Start by finding a combination that supports both the app and GPU on the operating system you already use.

Can a Docker image let you skip this process? A container packages the app and Python libraries, but it does not create the host’s GPU or driver. You still need to meet the host requirements and GPU pass-through settings in the official container guide. If installation succeeds but the GPU is not visible inside the container, check that connection before downloading the model files again. Running in a container does not make an unsupported GPU officially supported.

If the program you want to use is CUDA-only, ROCm will not automatically turn its executable into an AMD version. Check whether the program provides open source and a ROCm or HIP build. Do not assume a PyTorch program is NVIDIA-only just because its code contains .to("cuda"). For API compatibility, PyTorch on ROCm retains parts of the torch.cuda namespace and the "cuda" device notation, so identify the installed build with torch.version.hip. Conversely, CUDA in a name does not prove ROCm support either. Base your decision on the runtime path and installation instructions officially supported by the app.

This distinction also changes how you investigate an error. If PyTorch cannot find the GPU, first check the installed build and device connection. If the GPU is visible but a particular image app will not run, check whether the app requires a separate operation or extension package. A successful basic PyTorch operation does not mean that CUDA extensions called by the app also work on AMD. That is why the app’s documentation should cover both the ROCm installation path and support for any required extension packages.

Some programs offer Vulkan as an alternative to ROCm. For example, llama.cpp can be built with a Vulkan backend. But Vulkan is not the ROCm runtime: its driver requirements, build options and feature support differ from installing a PyTorch ROCm wheel. An app using a Radeon GPU through Vulkan does not mean ROCm is installed, and PyTorch support in a ROCm table does not apply to the Vulkan path. First check which backends the app you plan to use supports.

The options can therefore differ by app, even on the same Radeon. If your goal is to chat with a GGUF⁹ model, check whether llama.cpp’s Vulkan path supports the features you need. That success would not answer the question for someone trying to run a PyTorch-based image workflow. The model file, app and features you plan to use come first; the backend is the route that performs that work. When comparing speed across different backends, match the model, quantization¹⁰ and input conditions as closely as possible.

If you already own a Radeon, avoid additional spending for now and run one small task all the way through using the app’s officially supported path. Check that the model loads and produces the result you want, and that it still works after you close and reopen the app. If it does, the next questions are memory and runtime—not basic support. Increase the input or resolution to see what becomes a constraint, then adjust the settings. Finding the point where your current machine falls short is more useful than starting by choosing the newest graphics card.

If you plan to buy a new GPU, do not reverse the order of the decision. First choose the app and model, then check candidate cards against the Radeon path documented by that app. Even a card with more VRAM¹¹ cannot do the job if a required extension module will not run. Community success stories may help with combinations without official support, but factor in the work of fixing and maintaining the setup. If you do not want to take that on, choosing another GPU or backend supported by the app is reasonable. Before deciding whether to buy a Radeon, determine how you will run the work you need to do.