ratchet

We are on a mission to bring fast, cross platform GPU accelerated inference on native + browser.

[!NOTE]
Ratchet is currently in active development. We are working on the engine, adding more models and improving compatibility. Please, reach out if you'd like to help!

Getting Started

The easiest way to experience Ratchet is to check out our Hugging Face spaces:

• Whisper
• Phi

To dig deeper, check out the examples

We welcome contributions from the community. If you have any ideas or suggestions, please feel free to open an issue or pull request.

Javascript

// Asynchronous loading & caching with IndexedDB
let model = await Model.load(AvailableModels.WHISPER_TINY, Quantization.Q8, (p: number) => setProgress(p))
let result = await model.run({ input });

Rust

Rust crate & CLI coming soon...

Philosophy

We want a toolkit for developers to make integrating performant AI functionality into existing production applications easy. The following principles will help us accomplish this:

1. Inference only
2. WebGPU/CPU only
3. First class quantization support
4. Lazy computation
5. Inplace by default

Supported Models

• Whisper
• Phi 2 & 3
• Moondream

Upcoming Models

• Gemini 2 2B