Qwen3-VL-Embedding

Qwen3-VL-Embedding & Qwen3-VL-Reranker

State-of-the-art multimodal embedding and reranking models built on Qwen3-VL, supporting text, images, screenshots, videos, and mixed-modal inputs for advanced information retrieval and cross-modal understanding.

Table of Contents

• Overview
• Features
• Model Architecture
• Installation
• Usage
• Examples
• Model Performance
• Citation

Overview

The Qwen3-VL-Embedding and Qwen3-VL-Reranker model series are the latest additions to the Qwen family, built upon the recently open-sourced and powerful Qwen3-VL foundation model. Specifically designed for multimodal information retrieval and cross-modal understanding, this suite accepts diverse inputs including text, images, screenshots, and videos, as well as inputs containing a mixture of these modalities.

Building on the success of our text-oriented Qwen3-Embedding and Qwen3-Reranker series, these multimodal models extend best-in-class performance to visual and video understanding tasks. The models work in tandem: the Embedding model handles the initial recall stage by generating semantically rich vectors, while the Reranking model manages the re-ranking stage with precise relevance scoring, significantly enhancing final retrieval accuracy.

Features

• ** Multimodal Versatility**: Seamlessly process inputs containing text, images, screenshots, and video within a unified framework. Achieve state-of-the-art performance across diverse tasks including image-text retrieval, video-text matching, visual question answering (VQA), and multimodal content clustering.

• ** Unified Representation Space**: Leverage the Qwen3-VL architecture to generate semantically rich vectors that capture both visual and textual information in a shared space, facilitating efficient similarity estimation and retrieval across different modalities.

• ** High-Precision Reranking**: The reranking model accepts input pairs (Query, Document)—where both can consist of arbitrary single or mixed modalities—and outputs precise relevance scores for superior retrieval accuracy.

• ** Exceptional Practicality**:

  • Support for over 30 languages, ideal for global applications
  • Customizable instructions for task-specific optimization
  • Flexible vector dimensions with Matryoshka Representation Learning (MRL)
  • Strong performance with quantized embeddings for efficient deployment
  • Easy integration into existing retrieval pipelines

Supported Languages

Qwen3-VL-Embedding and Qwen3-VL-Reranker supports 33 languages:

• English
• Chinese
• Spanish
• French
• Arabic
• Portuguese
• Russian
• Urdu
• Indonesian
• German
• Japanese
• Vietnamese
• Turkish
• Korean
• Persian
• Italian
• Thai
• Polish
• Ukrainian
• Uzbek
• Romanian
• Dutch
• Kazakh
• Greek
• Czech
• Swedish
• Serbian
• Hebrew
• Danish
• Finnish
• Norwegian
• Slovenian
• Irish

Model Architecture

Model Specifications

LoRA Configs

Architecture Design

Qwen3-VL-Embedding: Dual-Tower Architecture

• Receives single-modal or mixed-modal input and maps it into a high-dimensional semantic vector
• Extracts the hidden state vector corresponding to the [EOS] token from the base model's last layer as the final semantic representation
• Enables efficient, independent encoding necessary for large-scale retrieval

Qwen3-VL-Reranker: Single-Tower Architecture

• Receives an input pair (Query, Document) and performs pointwise reranking
• Utilizes Cross-Attention mechanism for deeper, finer-grained inter-modal interaction and information fusion
• Expresses relevance score by predicting the generation probability of special tokens (yes and no)

Feature Comparison

Both models are built through a multi-stage training paradigm that fully leverages the powerful general multimodal semantic understanding capabilities of Qwen3-VL, providing high-quality semantic representations and precise re-ranking mechanisms for complex, large-scale multimodal retrieval tasks.

Installation

Setup Environment

# Clone the repository
git clone https://github.com/QwenLM/Qwen3-VL-Embedding.git
cd Qwen3-VL-Embedding

# Run the script to setup the environment
bash scripts/setup_environment.sh

The setup script will automatically:

• Install uv if not already installed
• Install all project dependencies

After setup completes, activate the environment:

source .venv/bin/activate

Download Models

Our models are available on both Hugging Face and ModelScope.

Install download dependencies:

Download from Hugging Face:

uv pip install huggingface-hub

huggingface-cli download Qwen/Qwen3-VL-Embedding-2B --local-dir ./models/Qwen3-VL-Embedding-2B

Download from ModelScope:

uv pip install modelscope

modelscope download --model qwen/Qwen3-VL-Embedding-2B --local_dir ./models/Qwen3-VL-Embedding-2B

Usage

Quick Start

Embedding Model

Transformers usage

…

vLLM usage

Note: Requires vLLM >= 0.14.0

For vLLM usage examples with the embedding model, please refer to examples/embedding_vllm.ipynb.

Reranking Model

Transformers usage

…

vLLM usage

Note: Requires vLLM >= 0.14.0

For vLLM usage examples with the reranker model, please refer to examples/reranker_vllm.ipynb.

Model Input Specification

Multimodal Object

A dictionary that can contain the following keys:

• text: Text input as a string or a list of strings
• image: Image input, supports:
  • Local file path
  • URL (network path)
  • PIL.Image.Image instance
  • List of any combination of the above (multiple images)
• video: Video input, supports:
  • Local file path
  • URL (network path)
  • Sequence of video frames (list of image paths or PIL.Image.Image instances)
  • List of any combination of the above (multiple videos)

Note: All input types (text, image, video) now support both single objects and lists of objects, allowing you to provide multiple inputs of the same type in a single request. For example, you can pass multiple images as a list, multiple text strings as a list, or multiple videos as a list.

Instruction

Task description for relevance evaluation (default: "Represent the user's input")

Video Sampling Settings

Only effective when video input is a video file:

• fps: Frame sampling rate per second (frames per second)
• max_frames: Maximum number of frames to sample

Input Format

Embedding Model: A list of dictionaries, where each dictionary contains:

• Instruction (optional)
• Video sampling settings (optional)
• Multimodal object keys (text, image, and/or video)

Reranking Model: A dictionary containing:

• query: A multimodal object
• documents: A list of multimodal objects
• instruction: Task description (optional)
• fps: Video sampling rate (optional)
• max_frames: Maximum frames (optional)

Embedding Model

Model Initialization Parameters

…

Examples

Embedding Model

We provide comprehensive examples here demonstrating various tasks across different modalities:

Text Tasks:

• Text Classification (AG News)
• Text Question Answering (SQuAD)
• Text Retrieval (MS MARCO)

Image Tasks:

• Image Classification (CIFAR-10)
• Image Question Answering (VQAv2)
• Image Retrieval (MS COCO)

Examples for video and visual document tasks are presented in the appendix of technical report

We also provide an end-to-end multimodal RAG example using Qwen3-VL-Embedding, Qwen3-VL-Reranker and Qwen3-VL here.

Reranking Model

We provide comprehensive examples here demonstrating various tasks across different modalities:

Text Tasks:

• Text Retrieval (MS MARCO)

Image Tasks:

• Image Retrieval (MS COCO)

Model Performance

Embedding Model

Evaluation Results on MMEB-V2

Results on the MMEB-V2 benchmark. All models except IFM-TTE have been re-evaluated on the updated VisDoc OOD split. CLS: classification, QA: question answering, RET: retrieval, GD: grounding, MRET: moment retrieval, VDR: ViDoRe, VR: VisRAG, OOD: out-of-distribution.