I wanted to start experimenting with AI for a while.
At work, I didn't really have enough time to properly explore this new ecosystem and understand how these tools could be used.
But I had another resource available: my personal hardware.
I'm a gamer, so I have a reasonably powerful desktop PC.
I'm also a developer, with a laptop running Ubuntu and a NAS.
That made me wonder: Can I use my gaming PC to run AI models locally?
The answer is yes.
This article is a small walkthrough of my experience setting up a local AI environment using an AMD Radeon GPU, Windows, WSL, ROCm, and Ollama.
My hardware is (yes, I'm Team Red): AMD Ryzen 7 5700X 32 GB RAM AMD Radeon RX 9060 XT with 16 GB VRAM Ubuntu 26.04 running under WSL Installing WSL and Ubuntu The first step is to make sure that WSL is installed and up to date.
The general process is: Check your current WSL version.
Update WSL.
Install Ubuntu.
Create your user account.
Update the Ubuntu installation.
WSL From PowerShell: Ubuntu Once Ubuntu is installed, update the system and install .
We will need it later to extract Ollama.
Installing Ollama with ROCm support Ollama is the foundation of this setup.
It allows us to download and run AI models locally while providing different compute backends, including CPU, CUDA, and ROCm.
Installation One thing that wasn't immediately obvious to me was that Ollama provides a specific ROCm backend.
The installation is split into two parts: Install Ollama.
Install the ROCm backend.
Then create the Ollama user and add your current user to the Ollama group: The official documentation I used: https://docs.ollama.com/linux#manual-install https://docs.ollama.com/linux#amd-gpu-install Running Ollama as a service I chose to run Ollama as a systemd service.
Create or edit the service configuration: Then reload systemd and start the service: More information about running Ollama as a service: https://docs.ollama.com/linux#adding-ollama-as-a-startup-service-recommended Installing DXG support The GPU needs to be exposed to the WSL environment.
For my setup, I used .
One important detail: version 1.2.2 solved an issue I encountered with version 1.2.0.
So, if you're having issues with GPU detection, the version may matter.
More information about the project: https://github.com/ROCm/librocdxg Checking that Ollama is using the GPU At this point, everything should be configured.
Restart Ollama: Then check the Ollama logs: You should see something similar to: The important part is: You don't want Ollama to silently fall back to CPU inference.
In my case, I get: As you can see, Ollama detects: The ROCm backend. .
The AMD Radeon RX 9060 XT.
Approximately 16 GB of VRAM.
AMD provides a compatibility matrix to identify which version corresponds to your GPU: https://rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html?fam=radeon&gpu=amd-radeon-rx-9070-gre&gfx=gfx1201&os=ubuntu I also found references suggesting that it may be possible to enable support for some older versions, but I didn't need to do this for my setup.
Running a model Time for the interesting part.
Download and run a model: Ask the model something and let it generate a response.
You can exit the interactive session with: You can then check where the model is running: The output should indicate that the model is running on the GPU.
You can also open the Windows Task Manager and monitor your GPU usage.
Once the model starts running, you should see: VRAM usage increasing.
GPU compute activity increasing.
The model using the GPU instead of relying entirely on the CPU.
Bonus Exposing Ollama to your local network By default, your Ollama instance is running inside WSL.
If you want to access it from other machines on your local network, you need to expose the port through Windows.
On the Windows host, open PowerShell as Administrator.
First, create a firewall rule: Then forward the port between Windows and WSL: From another computer on your local network, you can test the Ollama API with: If everything is configured correctly, Ollama should respond with the list of installed models.
Keeping WSL alive after closing the terminal By default, WSL may stop after some time when no active terminal is connected.
To keep the WSL environment alive, create a file in your Windows user profile directory.
Add: After modifying the configuration, restart the WSL distribution so the configuration is taken into account Now once you started the image you will need to kill it manually.
Ollama is no longer accessible after restarting the PC I encountered one additional issue.
After restarting my PC, I could start the WSL distribution, but Ollama was no longer accessible from the local network.
In my case, restarting the Windows service solved the problem.
Open PowerShell as Administrator: After that, Ollama became accessible again.
Conclusion This setup allowed me to turn my gaming PC into a local AI machine without replacing Windows or dedicating the entire computer to Linux.
The combination of: Windows WSL Ubuntu AMD Radeon ROCm Ollama provides a relatively convenient environment for experimenting with local AI models.
The most important part of the setup, in my experience, was making sure that Ollama was actually using the ROCm backend and the GPU.
Once that was working, running a model such as was straightforward.
Hopefully, this will save some time for other AMD GPU users who want to experiment with local AI on Windows.