[Enhancement] Local Content Moderation
Author: iwr-redmondCreated Dec 23, 2025Updated Dec 29, 2025
Labelsenhancement
Motivation
Parlant currently supports the Lakera Guard service for content moderation. It may be helpful to offer a local or self-hosted option as well.
Solution Proposal
- Implement #678 to facilitate Parlant-managed LLM inference
- Create inference code for Llama 3.1 base GGUF with the NemoGuard LoRA adapter
- Test the Llama 3.1 base model as a possible default for xllamacpp, or alternatively document the VRAM requirements for running NemoGuard alongside a primary model
Discussion
The method by which LoRA adapters can be applied in xllamacpp is documented here.
Source: emcie-co/parlant