Baike.dev
All toolsOnline toolsAI codingTrendingOpen sourceNewsSubmit
Log in
< Back to tools
C

chunk

> 编程语言
Open source

The Fastest Chunker in the West Upto 1TB/s "semantic" chunking, quick and easy!

355 stars0 likes0 views
WebsiteGitHub

About

The Fastest Chunker in the West Upto 1TB/s "semantic" chunking, quick and easy!

chunk

the fastest text chunking library — up to 1 TB/s throughput


you know how every chunking library claims to be fast? yeah, we actually meant it.

chunk splits text at semantic boundaries (periods, newlines, the usual suspects) and does it stupid fast. we're talking "chunk the entire english wikipedia in 120ms" fast.

want to know how? read the blog post where we nerd out about SIMD instructions and lookup tables.

[!NOTE] chunk was previously known as memchunk. It still contains the chunk method but now also includes other basic operations that power chonkie core.

See benches/ for detailed benchmarks.

Installation

bash
cargo add chunk

looking for python or javascript?

Usage

…

Citation

If you use chunk in your research, please cite it as follows:

bibtex
@software{chunk2025,
  author = {Minhas, Bhavnick},
  title = {chunk: The fastest text chunking library},
  year = {2025},
  publisher = {GitHub},
  howpublished = {\\url{https://github.com/chonkie-inc/chunk}},
}

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.

Issues· 0 open

View all issuesOpen on GitHub

No open issues yet, or sync has not completed.

> Tags

Rust

No comments yet. Be the first to share.

> Details

PublishedAug 1, 2026
UpdatedSep 17, 2026
Category编程语言
PricingOpen source

> Related tools

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言