Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install

Qwen3.8-27B on 16-32 GB Nvidia GPUs one-click install for Windows / Linux

by Mia'a AI Lab

A serving kit for Qwen/Qwen3.8-27B in turboderp's EXL3 quants, on one consumer NVIDIA card. It picks a quant that fits the card it finds, installs its own Python environment, downloads the weights, serves an OpenAI-compatible endpoint, and opens a chat UI. Windows and Linux, same behaviour.

It started as a 16 GB recipe — the 2.0 bpw quant is still that floor, and still the one thing here that is not turboderp's own upload (Mia-AiLab/Qwen3.8-27B-EXL3-2.0bpw, SC_2.00bpw_H3_V3). Everything from 2.5 bpw up is pulled from turboderp/Qwen3.8-27B-exl3 by revision. Which one you get is decided by your VRAM, at setup, and you can change it any time.

Jump to your guide: Windows · Linux

Before you start (both systems)

Not needed: CUDA Toolkit, Visual Studio Build Tools, Git. The engine arrives as a prebuilt wheel; compiling is the fallback for platforms no wheel covers.

Everything the kit installs stays inside its own folder — .venv/, models/, logs/, apps/, .dsh/. Nothing goes into the system Python and nothing needs administrator rights.

Windows

Everything is in the windows\ folder. Run the .bat files from Explorer (double-click) or from a cmd window opened in the kit folder.

1. Install — windows\START-HERE.bat

Double-click it once. It opens a page in your browser and does the whole install there: it shows what it found on the card, offers the model sizes that fit it, then installs and downloads with a progress bar and a live log. Nothing is asked in the console.

When the download finishes it loads that model and hands the page over to the chat, so one double-click takes you from nothing to a working chat window.

windows\START-HERE.bat              install, then start what was installed
windows\START-HERE.bat --no-start   install only — for fetching a second size

Notes:

• The download is resumable. Closing the window, losing the connection or rebooting costs you nothing — it picks up from the byte it stopped at. A model left half-downloaded is shown as such in the menus, and running windows\START-HERE.bat again finishes it. Nothing offers to start a model until every weight file is on disk.
• Prefer the old console questions to the web page? Set SETUP=console in .env.

2. Every day after that — windows\start.bat

windows\start.bat          start a model that is already here
windows\start.bat setup    go to setup instead (same as START-HERE.bat)

It never downloads anything. What it does:

1. Asks which model, if more than one size is on disk. Enter takes the one that ran last, and it starts on its own after 45 seconds so an unattended machine still comes up.
2. Checks free VRAM right before the load — it wants the GPU_MEM_GB budget from .env plus a little margin. If that much is not free it lists the programs holding VRAM (browsers, games, Discord, other AI tools) and waits: Enter re-checks, c continues anyway, q quits, and it continues on its own after 120 seconds. Take this seriously on Windows — with too little free VRAM the driver pages the model into system RAM instead of failing, and it then runs many times slower.
3. Loads the model and opens the chat UI at http://127.0.0.1:3080/.

If nothing is installed yet, or nothing finished downloading, it says so and offers to run setup for you — double-clicking the wrong one is never a dead end.

Two files rather than one because they answer two different questions: windows\start.bat never downloads, and windows\START-HERE.bat --no-start never loads.

3. While it runs

• The console window it opened is the server. Closing it stops the model.
• Simplex puts an icon in the notification area — right-click for Open Simplex, Restart the model, Show the Simplex folder, View the log and Quit Simplex. (TRAY=no in .env turns it off.)
• Every launch writes a full transcript to logs\, so a crash that scrolls past is still readable afterwards.
• The first successful launch adds Start-menu and desktop shortcuts (SHORTCUTS=no in .env to skip that).

4. Stopping — windows\stop.bat

windows\stop.bat                 stop both the model and the chat UI
windows\stop.bat --harness-only  leave the model loaded, close the UI
windows\stop.bat --server-only   leave the UI running, unload the model

5. Windows troubleshooting

Linux

Everything is in the linux/ folder. Run the scripts from the kit root; they find their own way regardless of where you call them from.

If the files arrived without their execute bit (a zip, a copy off Windows), run them as bash linux/setup.sh instead of ./linux/setup.sh, or chmod +x linux/*.sh linux/simplex once.

1. Install — ./linux/setup.sh

./linux/setup.sh

It creates .env from .env.example on the first run, asks the profile questions in the terminal (tools/profiles.py) — there is no setup page on Linux — builds .venv, installs PyTorch and the engine, downloads the weights, and stops. It does not load a model.

The download is resumable: interrupt it and run ./linux/setup.sh again to carry on.

2. Every day after that — ./linux/start.sh

./linux/start.sh                 pick a downloaded model and serve it
./linux/start.sh --no-harness    serve /v1 only, no chat UI
./linux/start.sh -b              run in the background, output in logs/
./linux/start.sh --status        is a backgrounded one running?

It lists the models that finished downloading and asks which one (Enter is the one used last; it auto-picks after 45 seconds), then serves:

http://localhost:8888/v1     the OpenAI-compatible API
http://127.0.0.1:3080/       the chat UI

-b is the honest equivalent of the Windows tray: it detaches, writes to logs/simplex-*.log, and tells you where that log is and how to stop it. First-run setup and the model menu still happen — written to the log instead of the screen.

On a box with no desktop session webbrowser has nothing to open, so the chat address is printed for you to copy. Take the whole thing, token and all — see the note on the token.

There is no free-VRAM preflight on Linux (that check is Windows-specific, because Windows silently spills to system RAM instead of failing). If a load fails with Insufficient VRAM in split for model and cache, lower CONTEXT_SIZE or GPU_MEM_GB in .env, or close what is holding the card.

No tray icon and no desktop shortcuts either — those are Windows.

3. Stopping — ./linux/stop.sh

./linux/stop.sh                 stop both
./linux/stop.sh --harness-only  leave the model loaded
./linux/stop.sh --server-only   leave the chat UI running

4. Linux troubleshooting

One command, both systems

The files above are the double-click doors. Every verb, on either system, is simplex — the same program (tools/cli.py), so the two cannot drift apart:

…

simplex with no verb prints the help and then the status. Every verb takes --help. It is not on your PATH — run it from the kit folder.

simplex doctor

The first thing to run when something is wrong. It checks the Python version, the venv and the engine version inside it, the driver and the card, Node, both ports and who holds them, the .env values that have to be valid, whether the weights are all there, and the free disk.

With or without the chat UI

UI= in .env is the standing answer (browser, server or no); --harness / --no-harness overrides it for one run, on simplex start, linux/start.sh and windows\start.bat alike. The UI is also a verb of its own, so it can be attached to a model that is already loaded, or taken away without unloading one.

What the launcher picks for your GPU

Setup (windows\START-HERE.bat / ./linux/setup.sh, or simplex setup, or PROFILE=ask in .env) runs tools/profiles.py. It reads the card's VRAM with nvidia-smi, computes what fits under a budget of VRAM − max(1.3 GB, 8 %), and offers the sizes that fit. Enter takes the recommendation. If a model is already downloaded, "keep current" is the default, so an unattended start never triggers a surprise download.

The choice is written into .env (MODEL_DIR, HF_TARGET_REPO, HF_REVISION, MODEL_ID, CONTEXT_SIZE, CACHE_QUANT, GPU_MEM_GB, VISION) and everything downstream follows it.