[Bug]: When using pipeline, Auto Metadata extraction runs successfully, but extracted values are not stored in document metadata fields

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (Language Policy).
• Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• Please do not modify this template :) and fill in all the required fields.

RAGFlow workspace code commit ID

a024bea

RAGFlow image version

v0.27.2

Other environment information

Deployment: Docker / Unraid
LLM: Qwen 3.5 9B via Ollama / OpenAI-compatible API
Parser: custom Dataflow pipeline

Actual behavior

have configured a RAGFlow Dataflow pipeline with an Auto Metadata / Extractor component intended to extract structured metadata from a document. The extractor works if not using pipeline.

The extractor is definitely being executed and the LLM is returning successfully, but the extracted metadata values are not appearing in the document's configured metadata fields. The fields are created in Elasticsearch but remain null.

Pipeline

The relevant pipeline is:

File ↓ Parser ↓ TokenChunker ↓ Extractor (Auto Metadata) ↓ Tokenizer

The Auto Metadata extractor has:

{ "field_name": "metadata", "max_tokens": 4096, "temperature": 0, "thinking": "disabled" }

The system prompt is:

The document contains a metadata section between "--- DOCUMENT METADATA ---" and "--- END DOCUMENT METADATA ---".

Extract the values from that section into a JSON object with exactly these keys:

• "doc_date"
• "project_name"
• "project_address"
• "document_client"
• "document_type"
• "document_title"
• "project_no"

Return only the JSON object. Use null for any field not found.

The document itself contains a metadata block such as:

--- DOCUMENT METADATA --- SOURCE FILE: ... DOCUMENT TYPE: Proposal PROJECT NAME: ... PROJECT NUMBER: ... PROJECT ADDRESS: ... DATE: ... CLIENT: ... --- END DOCUMENT METADATA ---

Evidence that the LLM extraction is running

RAGFlow logs show the extractor prompt being executed and LLMBundle.async_chat completing successfully:

2026-09-29 10:02:54,593 INFO 105 LLMBundle.async_chat used_tokens: 2083 2026-09-29 10:04:08,342 INFO 105 LLMBundle.async_chat used_tokens: 6919 2026-09-29 10:04:41,963 INFO 105 LLMBundle.async_chat used_tokens: 3239

I also tested the same metadata extraction prompt directly against the Ollama model, and it returned valid JSON with the expected metadata values.

For example:

{ "doc_date": "28 September 2026", "project_name": "Lilyfield Road Acoustic Assessment", "project_address": "160 Lilyfield Road, Lilyfield NSW", "document_client": "Example Client", "document_type": "Proposal", "document_title": null, "project_no": "13356P" }

Therefore the LLM itself does not appear to be the problem.

Result in Elasticsearch

After reprocessing a document, the corresponding RAGFlow metadata document exists and is updated, but all configured fields remain null.

Example:

{ "_index": "ragflow_doc_meta_a8b543b89ba311f1aafec9698c6cd662", "_id": "46bf2b132accfa359b825f06a446301d", "_version": 3, "_source": { "id": "46bf2b132accfa359b825f06a446301d", "kb_id": "e67e4bdab5b711f1bc43c751fe22ca7a", "meta_fields": { "document_title": null, "project_no": null, "project_address": null, "doc_date": null, "document_client": null, "project_name": null, "document_type": null } } }

(above formatting is mangled during copy)

The _version changing confirms that the metadata document is being written/updated during processing.

Additional observation

The Dataflow task configuration logged by RAGFlow contains six configured metadata fields:

doc_date project_name document_client document_type document_title project_no

However, the custom Auto Metadata extractor requests seven fields:

doc_date project_name project_address document_client document_type document_title project_no

project_address is therefore not present in the parser's configured metadata field list, although the extractor requests it.

However, this does not explain why the other six fields also remain null.

Expected behavior

If the Auto Metadata extractor returns:

{

"doc_date": "28 September 2026", "project_name": " Lilyfield Road Acoustic Assessment", "project_address": "160 Lilyfield Road, Lilyfield NSW", "document_client": "Example Client", "document_type": "Proposal", "document_title": "example.pdf", "project_no": "13356P" }

I would expect the document metadata record to contain those values:

"meta_fields": { "doc_date": "28 September 2026", "project_name": "Lilyfield Road Acoustic Assessment", "project_address": "160 Lilyfield Road, Lilyfield NSW", "document_client": "Example Client", "document_type": "Proposal", "document_title": "example.pdf", "project_no": "13356P" }

Instead, the metadata fields remain null.

Steps to reproduce

1. Create a RAGFlow dataset with document metadata fields, for example:
doc_date — string
project_name — string
project_address — string
document_client — string
document_type — string
document_title — string
project_no — string

2. Create a Dataflow pipeline containing:

File
  ↓
Parser
  ↓
TokenChunker
  ↓
Extractor (Auto Metadata)
  ↓
Tokenizer

3. Configure the Extractor with:
Destination / field_name: metadata
Temperature: 0
Max tokens: 4096

4. Configure the Extractor system prompt to return JSON containing the metadata fields:

The document contains a metadata section between "--- DOCUMENT METADATA ---" and "--- END DOCUMENT METADATA ---".

Extract the values from that section into a JSON object with exactly these keys:
- "doc_date"
- "project_name"
- "project_address"
- "document_client"
- "document_type"
- "document_title"
- "project_no"

Return only the JSON object. Use null for any field not found.

5. Upload a PDF containing a clearly identifiable metadata block, for example:

--- DOCUMENT METADATA ---
SOURCE FILE: example.pdf
DOCUMENT TYPE: Proposal
PROJECT NAME: Lilyfield Road Acoustic Assessment
PROJECT NUMBER: 13356P
PROJECT ADDRESS: 160 Lilyfield Road, Lilyfield NSW
DATE: 28 September 2026
CLIENT: Example Client
--- END DOCUMENT METADATA ---

6. Parse/re-parse the document using the Dataflow pipeline.

7. Confirm in the RAGFlow logs that the Auto Metadata LLM call executes successfully. For example:

LLMBundle.async_chat used_tokens: 2083

The LLM returns successfully.

8. Inspect the document metadata record in Elasticsearch.

9. The metadata document is created/updated, but the configured metadata fields contain null instead of the values returned by the extractor:

{
  "meta_fields": {
    "doc_date": null,
    "project_name": null,
    "project_address": null,
    "document_client": null,
    "document_type": null,
    "document_title": null,
    "project_no": null
  }
}

Additional information

No response