< 返回工具列表
O

onnxruntime-rs

> 编程语言
开源

Rust 为微软的 ONNX 运行时间 (1.8 版) 包装器

329 stars0 点赞2 次浏览
访问官网GitHub

工具介绍

Rust 为微软的 ONNX 运行时间 (1.8 版) 包装器

ONNX Runtime

This is an attempt at a Rust wrapper for Microsoft's ONNX Runtime (version 1.8).

This project consist on two crates:

Changelog

The build.rs script supports downloading pre-built versions of the Microsoft ONNX Runtime, which provides the following targets:

CPU:

  • Linux x86_64
  • macOS x86_64
  • macOS aarch64 (no pre-built binaries, no CI testing, see #74)
  • Windows i686
  • Windows x86_64

GPU:

  • Linux x86_64
  • Windows x86_64

WARNING:

  • This is an experiment and work in progress; it is not complete/working/safe. Help welcome!
  • Basic inference works, see onnxruntime/examples/sample.rs or onnxruntime/tests/integration_tests.rs
  • ONNX Runtime has many options to control the inference process but those options are not yet exposed.
  • This was developed and tested on macOS Catalina. Other platforms should work but have not been tested.

Setup

Three different strategy to obtain the ONNX Runtime are supported by the build.rs script:

  1. Download a pre-built binary from upstream;
  2. Point to a local version already installed;
  3. Compile from source (not yet implemented).

To select which strategy to use, set the ORT_STRATEGY environment variable to:

  1. download: This is the default if ORT_STRATEGY is not set;
  2. system: To use a locally installed version (use ORT_LIB_LOCATION environment variable to point to the install path)
  3. compile: To compile the library

The download strategy supports downloading a version of ONNX that supports CUDA. To use this, set the environment variable ORT_USE_CUDA=1 (only supports Linux or Windows).

Until the build script allow compilation of the runtime, see the compilation notes for some details on the process.

Note on 'ORT_STRATEGY=system'

When using ORT_STRATEGY=system, executing a built crate binary (for example the tests) might fail, at least on macOS, if the library is not installed in a system path. An error similar to the following happens:

dyld: Library not loaded: @rpath/libonnxruntime.1.7.1.dylib
  Referenced from: onnxruntime-rs.git/target/debug/deps/onnxruntime_sys-22eb0e3e89a0278c
  Reason: image not found

To fix, one can either:

  • Set the LD_LIBRARY_PATH environment variable to point to the path where the library can be found.

  • Adapt the .cargo/config file to contain a linker flag to provide the full path:

    toml
    [target.aarch64-apple-darwin]
    rustflags = ["-C", "link-args=-Wl,-rpath,/full/path/to/onnxruntime/lib"]

See rust-lang/cargo #5077 for more information.

Example

The C++ example that uses the C API (C_Api_Sample.cpp) was ported to both the low level crate (onnxruntime-sys) and the high level on (onnxruntime).

onnxruntime-sys

To run this example (onnxruntime-sys/examples/c_api_sample.rs):

…

onnxruntime

To run this example (onnxruntime/examples/sample.rs):

…

See also the integration tests (onnxruntime/tests/integration_tests.rs) that performs simple model download and inference, validating the results.

Bindings Generation

Bindings (the basis of onnxruntime-sys) are committed to the git repository. This means bindgen is not a dependency anymore on every build (it was made optional) and thus build times are better.

To generate new bindings (for example if they don't exists for your platform or if a version bump occurred), build the crate with the generate-bindings feature.

NOTE: Make sure to have the rustfmt rustup component present so that bindings are formatted:

bash
rustup component add rustfmt

Then on each platform build with the proper feature flag:

bash
❯ cd onnxruntime-sys
❯ cargo build --features generate-bindings

Generating Bindings for Linux With Docker

Prepare the container:

bash
❯ docker run -it --rm --name rustbuilder -v "$PWD":/usr/src/myapp -w /usr/src/myapp rust:1.50.0 /bin/bash
❯ apt-get update
❯ apt-get install clang
❯ rustup component add rustfmt

Generate the bindings:

bash
❯ docker exec -it --user "$(id -u)":"$(id -g)" rustbuilder /bin/bash
❯ cd onnxruntime-sys
❯ cargo build --features generate-bindings

Generating Bindings for Windows With Vagrant

You can use nbigaouette/windows_vagrant_rust to provision a Windows VM that can build the project and generate the bindings.

Windows can build both x86 and x86_64 bindings:

bash
❯ rustup target add i686-pc-windows-msvc x86_64-pc-windows-msvc
❯ cd onnxruntime-sys
❯ cargo build --features generate-bindings --target i686-pc-windows-msvc
❯ cargo build --features generate-bindings --target x86_64-pc-windows-msvc

Conduct

The Rust Code of Conduct shall be respected. For escalation or moderation issues please contact Nicolas ([email protected]) instead of the Rust moderation team.

License

This project is licensed under either of

at your option.

暂无开放 Issues,或尚未同步最近议题。

> 标签

Rustinferenceonnxonnx-runtimeonnxruntime

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月18日
分类编程语言
定价开源