[Feature/Optimization] OvisOCR2 (finetuned Qwen3.5-0.8B) vision encoder inference latency is significantly high on MNN
When deploying OvisOCR2 (an end-to-end document parsing model based on Qwen3.5-0.8B) with MNN, I found that the encoding time of its vision encoder is significantly too long, making it the main bottleneck in the entire inference pipeline.The more pixel and take more exponentially accumulate time .
request an optimization for the vision encoder path used by OvisOCR2 / Qwen3.5-0.8B in MNN. The encoder latency is not only high, but appears to grow exponentially with the input resolution and/or the number of visual tokens. This makes high-resolution, on-device document parsing impractical, because the vision encoder dominates the total inference time and causes very high first-token latency (TTFT).
If the MNN team currently does not have plan to optimize this, could you please advise users on which aspects we should focus on ourselves?
The ovisOCRv2 export config is the same deafult qwen3.5 0.8b config.
Source: alibaba/MNN