Baike.dev
All toolsOnline toolsAI codingTrendingOpen sourceNewsSubmit
Log in
< Back to tools
O

onnxruntime-rs

> 编程语言
Open source

Rust wrapper for Microsoft's ONNX Runtime (version 1.8)

329 stars0 likes2 views
WebsiteGitHub

About

Rust wrapper for Microsoft's ONNX Runtime (version 1.8)

ONNX Runtime

This is an attempt at a Rust wrapper for Microsoft's ONNX Runtime (version 1.8).

This project consist on two crates:

  • onnxruntime-sys: Low-level binding to the C API;
  • onnxruntime: High-level and safe API.

Changelog

The build.rs script supports downloading pre-built versions of the Microsoft ONNX Runtime, which provides the following targets:

CPU:

  • Linux x86_64
  • macOS x86_64
  • macOS aarch64 (no pre-built binaries, no CI testing, see #74)
  • Windows i686
  • Windows x86_64

GPU:

  • Linux x86_64
  • Windows x86_64

WARNING:

  • This is an experiment and work in progress; it is not complete/working/safe. Help welcome!
  • Basic inference works, see onnxruntime/examples/sample.rs or onnxruntime/tests/integration_tests.rs
  • ONNX Runtime has many options to control the inference process but those options are not yet exposed.
  • This was developed and tested on macOS Catalina. Other platforms should work but have not been tested.

Setup

Three different strategy to obtain the ONNX Runtime are supported by the build.rs script:

  1. Download a pre-built binary from upstream;
  2. Point to a local version already installed;
  3. Compile from source (not yet implemented).

To select which strategy to use, set the ORT_STRATEGY environment variable to:

  1. download: This is the default if ORT_STRATEGY is not set;
  2. system: To use a locally installed version (use ORT_LIB_LOCATION environment variable to point to the install path)
  3. compile: To compile the library

The download strategy supports downloading a version of ONNX that supports CUDA. To use this, set the environment variable ORT_USE_CUDA=1 (only supports Linux or Windows).

Until the build script allow compilation of the runtime, see the compilation notes for some details on the process.

Note on 'ORT_STRATEGY=system'

When using ORT_STRATEGY=system, executing a built crate binary (for example the tests) might fail, at least on macOS, if the library is not installed in a system path. An error similar to the following happens:

dyld: Library not loaded: @rpath/libonnxruntime.1.7.1.dylib
  Referenced from: onnxruntime-rs.git/target/debug/deps/onnxruntime_sys-22eb0e3e89a0278c
  Reason: image not found

To fix, one can either:

  • Set the LD_LIBRARY_PATH environment variable to point to the path where the library can be found.

  • Adapt the .cargo/config file to contain a linker flag to provide the full path:

    toml
    [target.aarch64-apple-darwin]
    rustflags = ["-C", "link-args=-Wl,-rpath,/full/path/to/onnxruntime/lib"]

See rust-lang/cargo #5077 for more information.

Example

The C++ example that uses the C API (C_Api_Sample.cpp) was ported to both the low level crate (onnxruntime-sys) and the high level on (onnxruntime).

onnxruntime-sys

To run this example (onnxruntime-sys/examples/c_api_sample.rs):

…

onnxruntime

To run this example (onnxruntime/examples/sample.rs):

…

See also the integration tests (onnxruntime/tests/integration_tests.rs) that performs simple model download and inference, validating the results.

Bindings Generation

Bindings (the basis of onnxruntime-sys) are committed to the git repository. This means bindgen is not a dependency anymore on every build (it was made optional) and thus build times are better.

To generate new bindings (for example if they don't exists for your platform or if a version bump occurred), build the crate with the generate-bindings feature.

NOTE: Make sure to have the rustfmt rustup component present so that bindings are formatted:

bash
rustup component add rustfmt

Then on each platform build with the proper feature flag:

bash
❯ cd onnxruntime-sys
❯ cargo build --features generate-bindings

Generating Bindings for Linux With Docker

Prepare the container:

bash
❯ docker run -it --rm --name rustbuilder -v "$PWD":/usr/src/myapp -w /usr/src/myapp rust:1.50.0 /bin/bash
❯ apt-get update
❯ apt-get install clang
❯ rustup component add rustfmt

Generate the bindings:

bash
❯ docker exec -it --user "$(id -u)":"$(id -g)" rustbuilder /bin/bash
❯ cd onnxruntime-sys
❯ cargo build --features generate-bindings

Generating Bindings for Windows With Vagrant

You can use nbigaouette/windows_vagrant_rust to provision a Windows VM that can build the project and generate the bindings.

Windows can build both x86 and x86_64 bindings:

bash
❯ rustup target add i686-pc-windows-msvc x86_64-pc-windows-msvc
❯ cd onnxruntime-sys
❯ cargo build --features generate-bindings --target i686-pc-windows-msvc
❯ cargo build --features generate-bindings --target x86_64-pc-windows-msvc

Conduct

The Rust Code of Conduct shall be respected. For escalation or moderation issues please contact Nicolas ([email protected]) instead of the Rust moderation team.

License

This project is licensed under either of

  • Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
  • MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Issues· 0 open

View all issuesOpen on GitHub

No open issues yet, or sync has not completed.

> Tags

Rustinferenceonnxonnx-runtimeonnxruntime

No comments yet. Be the first to share.

> Details

PublishedAug 1, 2026
UpdatedSep 18, 2026
Category编程语言
PricingOpen source

> Related tools

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言