torchao backend is slower than torch.ao on ARM64 CPU

Problem

The torchao quantization backend (selected by _get_backend() when torchao is installed) produces worse performance than both the torch.ao backend and no quantization at all on ARM64 CPU.

Benchmarks (ARM64, Cortex-A76, torch 2.8.0+cpu)

Longer text (9s of audio output):

Config	Generation time	RTF (audio/gen)	First chunk
No quantize	8.18s	1.14x	0.27s
torch.ao (QNNPACK)	11.76s	0.80x	0.71s
torchao 0.13.0	19.93s	0.49x	1.43s

torchao takes 2.4x longer than no-quantize and 1.7x longer than torch.ao for the same output. First-chunk latency is 5.3x worse than no-quantize.

The x86 benchmarks from issue #177 also reported torchao performing worse than torch.ao, but the original reporter used different methodology. The key finding is consistent: torchao does not improve CPU performance.

Environment

• pocket-tts 2.1.0
• torch 2.8.0+cpu (aarch64)
• torchao 0.13.0+cpu
• Python 3.10.21