#3530·candle

Candle leaves ~ model size of memory unused on Spark

Author: sempervictusCreated May 11, 2026Updated Aug 16, 2026

Candle seems unable to reason about availability of memory in a UMA system very well:

bash
$ free -g
               total        used        free      shared  buff/cache   available
Mem:             119          81          38           0           0          37
Swap:             15           0          15

Looks offhand like that's about the size of an 80B NVPF4 model which makes me wonder if its being mapped to memory and then copied to the same memory resulting in counting its size twice for bounds calculation.