#690·parlant

[Enhancement] Local Content Moderation

Author: iwr-redmondCreated Dec 23, 2025Updated Dec 29, 2025
Labelsenhancement

Motivation

Parlant currently supports the Lakera Guard service for content moderation. It may be helpful to offer a local or self-hosted option as well.

Solution Proposal

  1. Implement #678 to facilitate Parlant-managed LLM inference
  2. Create inference code for Llama 3.1 base GGUF with the NemoGuard LoRA adapter
  3. Test the Llama 3.1 base model as a possible default for xllamacpp, or alternatively document the VRAM requirements for running NemoGuard alongside a primary model

Discussion

The method by which LoRA adapters can be applied in xllamacpp is documented here.