Rust 库用于生成向量嵌入,在本地重新排序!
Click to list models
nomic-embed-vision-v1.5 for image-to-text searchclip-ViT-B-32-vision for image-to-text searchnomic-v2-moe feature (candle backend)qwen3 feature (candle backend)qwen3 feature (candle backend)qwen3 feature (candle backend)qwen3 feature (candle backend, multimodal via Qwen3VLEmbedding)Quantized versions are also available for several models above (append Q to the model enum variant, e.g., EmbeddingModel::BGESmallENV15Q). EmbeddingGemma additionally ships a 4-bit build as EmbeddingModel::EmbeddingGemma300MQ4.
Click to list models
Click to list models
Click to list models
To support the library, please donate to our primary upstream dependency, ort - The Rust wrapper for the ONNX runtime.
Run the following in your project directory:
cargo add fastembedOr add the following line to your Cargo.toml:
[dependencies]
fastembed = "7"…………Alternatively, local model files can be used for inference via the try_new_from_user_defined(...) methods of the respective structs.
Helpers in the similarity module score and rank the vectors embed returns, so a quick in-memory search needs no extra crate:
use fastembed::similarity::{cosine_similarity, top_k};
// `embeddings` is the Vec from model.embed(...)
let query = &embeddings[0];
// Score two vectors directly ([-1.0, 1.0], higher = closer)
let score = cosine_similarity(query, &embeddings[1]);
// Or rank the corpus: (index, score) pairs, best first
let hits = top_k(query, &embeddings, 5);
println!("Closest: {:?}", hits);For larger corpora or persistence, push the vectors to a vector search engine (e.g. Qdrant) and query there.
use fastembed::{SparseInitOptions, SparseModel, SparseTextEmbedding};
let mut model = SparseTextEmbedding::try_new(
SparseInitOptions::new(SparseModel::OpenSearchNeuralSparseDocV3Gte).with_max_length(8192),
)?;
// This model emits one score per vocabulary entry per token.
// So keep the batch size small.
let documents = model.embed(vec!["Hello World"], Some(4))?;
// Queries are from the tokenizer and a precomputed IDF table alone, without any inference.
let queries = model.query_embed(vec!["Hello World"])?;Qwen3 embedding models are available behind the qwen3 feature flag (candle backend).
[dependencies]
fastembed = { version = "7", features = ["qwen3"] }use candle_core::{DType, Device};
use fastembed::Qwen3TextEmbedding;
let device = Device::Cpu;
let model = Qwen3TextEmbedding::from_hf(
"Qwen/Qwen3-Embedding-0.6B",
&device,
DType::F32,
512,
)?;
// Text-only usage with the Qwen3-VL embedding checkpoint is also supported:
// let model = Qwen3TextEmbedding::from_hf("Qwen/Qwen3-VL-Embedding-2B", &device, DType::F32, 512)?;
let embeddings = model.embed(&["query: ...", "passage: ..."])?;
println!("Embeddings length: {}", embeddings.len());For multimodal text/image usage with Qwen/Qwen3-VL-Embedding-2B:
use candle_core::{DType, Device};
use fastembed::Qwen3VLEmbedding;
let device = Device::Cpu;
let model = Qwen3VLEmbedding::from_hf(
"Qwen/Qwen3-VL-Embedding-2B",
&device,
DType::F32,
2048,
)?;
let image_embeddings = model.embed_images(&["tests/assets/image_0.png", "tests/assets/image_1.png"])?;
let text_embeddings = model.embed_texts(&["query: blue cat", "query: red cat"])?;
println!("Image embeddings: {}", image_embeddings.len());
println!("Text embeddings: {}", text_embeddings.len());The nomic-embed-text-v2-moe model is available behind the nomic-v2-moe feature flag (candle backend). First general-purpose MoE embedding model with 100+ language support.
[dependencies]
fastembed = { version = "7", features = ["nomic-v2-moe"] }use candle_core::{DType, Device};
use fastembed::NomicV2MoeTextEmbedding;
let device = Device::Cpu;
let model = NomicV2MoeTextEmbedding::from_hf(
"nomic-ai/nomic-embed-text-v2-moe",
&device,
DType::F32,
512,
)?;
let embeddings = model.embed(&["search_query: ...", "search_document: ..."])?;
println!("Embeddings length: {}", embeddings.len());The BGE-M3 model produces dense, sparse, and ColBERT embeddings simultaneously in a single forward pass.
…Models download on first use and load from cache afterwards (no network needed at runtime once cached).
FASTEMBED_CACHE_DIR — cache location (default: .fastembed_cache). Equivalent to TextInitOptions::with_cache_dir.HF_HOME — if set, takes precedence over the above.HF_ENDPOINT — Hugging Face mirror base URL, for restricted networks.HF_TOKEN — Hugging Face access token, for gated repositories and higher rate limits. Without it, the token saved by huggingface-cli login is used when present.To run models on a GPU via DirectML on Windows, enable the directml feature:
[dependencies]
fastembed = { version = "7", features = ["directml"] }Then pass a DirectML execution provider when initializing a model:
use fastembed::{TextEmbedding, TextInitOptions, EmbeddingModel};
use ort::ep::DirectML;
let model = TextEmbedding::try_new(
TextInitOptions::new(EmbeddingModel::AllMiniLML6V2)
.with_execution_providers(vec![DirectML::default().into()]),
)?;When DirectML is detected, fastembed automatically disables memory pattern optimization and parallel execution on the ONNX Runtime session, as required by the DirectML execution provider.
Fastembed returns a typed fastembed::Error. The type is re-exported from the crate root. The enum is #[non_exhaustive]. New variants can be added in minor releases without breaking match arms.
use fastembed::{Error, Result, TextEmbedding};
fn load() -> Result {
let model = TextEmbedding::try_new(Default::default())?;
Ok(model)
// ...
}To handle an error, match on the variant that applies:
use fastembed::{Error, TextEmbedding, TextInitOptions, EmbeddingModel};
match TextEmbedding::try_new(TextInitOptions::new(EmbeddingModel::AllMiniLML6V2)) {
Ok(model) => { /* ... */ }
Err(Error::ModelRetrieval { file, source }) => {
eprintln!("could not fetch {file}: {source}");
}
/*
...
*/
Err(Error::Ort(err)) => {
eprint暂无开放 Issues，或尚未同步最近议题。