[BUG] “Immediately” model unload is skipped when VAD returns zero audio samples
Before You Submit
I searched the existing Handy issues for model unload / VAD / zero-audio cases and did not find one describing this specific early-return path.
Bug Description
When Model Unload Timeout is set to Immediately, Handy correctly unloads the ASR model after a normal transcription, but it does not unload the model when recording finishes with zero audio samples (for example, when VAD filters everything out).
This leaves the model resident in memory even though the user explicitly selected immediate unload.
Reproduction
- Set Model Unload Timeout to Immediately.
- Use a local GGUF model on Metal (reproduced with Qwen3-ASR 1.7B Q5_K_M).
- Start a transcription so Handy begins loading the model.
- Stop after a short/noisy/silent recording that VAD reduces to zero samples.
- Observe that the transcription path exits without unloading the model.
Expected behavior
Immediately should unload the model whenever the transcription session ends, including:
- successful transcription
- VAD/recording returns zero samples
- empty transcription
- cancellation/error/other early-return paths
Actual behavior
Normal transcription correctly unloads:
[2026-09-16][17:20:57][handy_app_lib::managers::transcription][INFO] Immediately unloading model after transcription
[2026-09-16][17:20:57][handy_app_lib::managers::transcription][DEBUG] Starting to unload model
[2026-09-16][17:20:57][transcribe_cpp][INFO] ggml_metal_free: deallocating
[2026-09-16][17:20:57][handy_app_lib::managers::transcription][DEBUG] Model unloaded manually (took 18ms)But the zero-sample path does not:
[2026-09-16][17:21:16][handy_app_lib::managers::transcription][DEBUG] Starting to load model: handy-computer/Qwen3-ASR-1.7B-gguf/Qwen3-ASR-1.7B-Q5_K_M.gguf
[2026-09-16][17:21:16][transcribe_cpp][INFO] ggml_metal_init: allocating
[2026-09-16][17:21:16][handy_app_lib::managers::transcription][INFO] Loaded whisper model 'handy-computer/Qwen3-ASR-1.7B-gguf/Qwen3-ASR-1.7B-Q5_K_M.gguf' (requested Auto, requested device 'Apple M1 Pro', bound backend 'MTL0', bound device 'Apple M1 Pro', supports_streaming=false, supports_translate=false, supports_language_detect=true)
[2026-09-16][17:21:19][handy_app_lib::audio_toolkit::audio::recorder][DEBUG] VAD at stop: withheld tail 30 frames (~480ms, 2 voiced), in_speech=false, onset_counter=0, hangover_counter=0
[2026-09-16][17:21:19][handy_app_lib::managers::audio][DEBUG] Microphone stream stopped
[2026-09-16][17:21:19][handy_app_lib::actions][DEBUG] Recording stopped and samples retrieved in 15.455125ms, sample count: 0
[2026-09-16][17:21:19][handy_app_lib::actions][DEBUG] Recording produced no audio samples; skipping persistenceThere is no subsequent Immediately unloading model, Starting to unload model, or ggml_metal_free entry.
On this machine the Handy process then remains at roughly 2.4 GB physical footprint until the model is manually unloaded or Handy exits.
Likely cause
The immediate-unload logic appears to run on the normal transcription-completion path, while the zero-sample path returns earlier. It may be safer to centralize session cleanup so all terminal paths honor model_unload_timeout = immediately.
System Information
App Version:
0.9.6
Operating System:
macOS 27.0
CPU:
Apple M1 Pro
GPU:
Apple M1 Pro GPU / Metal (MTL0)
Model:
handy-computer/Qwen3-ASR-1.7B-gguf/Qwen3-ASR-1.7B-Q5_K_M.gguf
Relevant settings:
model_unload_timeout = immediately
transcribe_accelerator = gpu
transcribe_gpu_device = ["metal","name","MTL0"]
vad_enabled = true
vad_backend = earshot
always_on_microphone = falseLogs
The relevant excerpts are included above. Full debug log is available if needed.
Source: cjpais/Handy