[Rust] Implement ONNX session pool for TableProcessor to improve parallelism

Author: lfgranjaCreated Feb 19, 2026Updated Feb 19, 2026

Problem

The current TableProcessor implementation in the Rust engine uses a std::sync::Mutex<Session> to serialize all ONNX inference operations:

pub struct TableProcessor {
    session: Option<std::sync::Mutex<Session>>,
    model_type: TableModelType,
}

This design means that all table extraction operations are serialized, even when processing multiple pages in parallel. This creates a bottleneck when:

  1. Processing multi-page PDFs with many tables
  2. Running on multi-core systems where parallelism could be exploited
  3. Using Rayon's par_iter() for page processing

Proposed Solution

Implement a session pool pattern that allows multiple concurrent inference operations:

Option 1: Fixed-size Session Pool

pub struct TableProcessor {
    sessions: Vec<Arc<Mutex<Session>>>,
    pool_size: usize,
    next_idx: AtomicUsize,
}

Option 2: Thread-local Sessions (simpler)

Use thread_local! to give each Rayon thread its own session, avoiding lock contention entirely.

Option 3: Async-aware Session Pool

If we move to fully async processing, use tokio::sync::Semaphore with multiple sessions.

Benefits

  • Improved throughput for table-heavy documents
  • Better utilization of multi-core systems
  • Reduced latency for batch processing

Considerations

  • ONNX Runtime sessions are not thread-safe, so each session still needs protection
  • Memory usage increases with pool size (each session has its own allocator state)
  • Need to benchmark to find optimal pool size

Current Workaround

The existing implementation works correctly but may be slower for documents with many tables due to serialization.

Related

  • Issue #205 (Table Transformer ONNX output contract)
  • PR #9 (FusionEngine implementation)