#1506·presidio

Add batch analyze API support for recognizer

Author: jimmyxie-figmaCreated Jan 6, 2025Updated Aug 4, 2026
Labelsenhancementanalyzer

Is your feature request related to a problem? Please describe.

Currently, the BatchAnalyzerEngine works by iterating through either a list or dictionary, analyzing and anonymizing the values one by one. while this is not an issue for the predefined recognizers, and there are improvements in the built-in NLP engine to support batch inference, it does pose an efficiency problem for the transformer recognizers, causing idle resources and low inference throughput.

Describe the solution you'd like

We want to build batch inference API support for recognizers. Early testing shows that even with a small batch size of 4, a BERT-like transformer speeds up inferences by 3x without any additional resource or memory usage.

The exact implementation is still up for discussion and one potential solution would be adding a batch recognizer mix-in, where we batch analyze the the batch recognizer first, and pass the results to the regular analyze for extension. similar to the nlp_engine.process_batch

Describe alternatives you've considered N/A

Additional context N/A

Source: data-privacy-stack/presidio