#23078·llama_index

[Bug]: RichPromptTemplate crashes on embedded audio/video files and corrupts embedded documents

Author: Harsh23KashyapCreated Sep 16, 2026Updated Sep 16, 2026

Bug Description

RichPromptTemplate.format_messages converts banks media blocks by stuffing input_*.data into the llama_index block's url field:

elif bank_block.type == BanksContentBlockType.audio:
    llama_blocks.append(AudioBlock(url=bank_block.input_audio.data))
elif bank_block.type == BanksContentBlockType.video:
    llama_blocks.append(VideoBlock(url=bank_block.input_video.data))
elif bank_block.type == BanksContentBlockType.document:
    llama_blocks.append(DocumentBlock(url=bank_block.input_document.data))

But banks' | audio, | video, and | document filters store embedded files as raw base64 in data (only URL inputs are kept as URLs). Two failure modes:

  • Audio and video files crash. AudioBlock.url / VideoBlock.url are AnyUrl, so raw base64 fails pydantic validation: ValidationError: ... Input should be a valid URL, relative URL without a base. Any chat template embedding an audio or video file is unusable.
  • Document files are silently corrupted. DocumentBlock.url is a plain string, so the base64 blob "succeeds" and every downstream consumer receives a document block whose url is a few kilobytes of base64 instead of a URL or the document bytes.

(Images are unaffected: banks embeds local images as data: URLs, which are valid URLs.)

Proposed fix: convert by payload shape - payloads that are URLs keep going to url, base64 payloads are decoded into the block's bytes field (audio / video / data) with the format banks detected carried over (format, video_mimetype, document_mimetype). I have the patch and regression tests ready - happy to send the PR.

Version

llama-index-core 0.14.24 (also verified on current main, fd4a517); banks 2.1.1

Steps to Reproduce

import os
from llama_index.core.prompts import RichPromptTemplate

os.environ["BANKS_MEDIA_ROOT"] = "/tmp"  # let banks read the test file
open("/tmp/clip.mp3", "wb").write(b"\xff\xf3" + b"\x00" * 100)

t = RichPromptTemplate("{% chat role='user' %}Describe this: {{ p | audio }}{% endchat %}")
t.format_messages(p="/tmp/clip.mp3")

Output:

pydantic_core._pydantic_core.ValidationError: 1 validation error for AudioBlock
url
  Input should be a valid URL, relative URL without a base [type=url_parsing, input_value='//MAAAAAAAAAAAAAAAAAAAA...', input_type=str]

Same crash for {{ p | video }} with an mp4. For {{ p | document }} with a pdf there is no crash, but the resulting block is DocumentBlock(url='JVBERi0xLjQK...') - base64 in the url field.