Rust wrapper for Microsoft's ONNX Runtime (version 1.8)
Rust wrapper for Microsoft's ONNX Runtime (version 1.8)
This is an attempt at a Rust wrapper for Microsoft's ONNX Runtime (version 1.8).
This project consist on two crates:
onnxruntime-sys: Low-level binding to the C API;onnxruntime: High-level and safe API.The build.rs script supports downloading pre-built versions of the Microsoft ONNX Runtime,
which provides the following targets:
CPU:
GPU:
WARNING:
onnxruntime/examples/sample.rs or onnxruntime/tests/integration_tests.rsThree different strategy to obtain the ONNX Runtime are supported by the build.rs script:
To select which strategy to use, set the ORT_STRATEGY environment variable to:
download: This is the default if ORT_STRATEGY is not set;system: To use a locally installed version (use ORT_LIB_LOCATION environment variable to point to the install path)compile: To compile the libraryThe download strategy supports downloading a version of ONNX that supports CUDA. To use this, set the
environment variable ORT_USE_CUDA=1 (only supports Linux or Windows).
Until the build script allow compilation of the runtime, see the compilation notes for some details on the process.
When using ORT_STRATEGY=system, executing a built crate binary (for example the tests) might fail, at least on macOS,
if the library is not installed in a system path. An error similar to the following happens:
dyld: Library not loaded: @rpath/libonnxruntime.1.7.1.dylib
Referenced from: onnxruntime-rs.git/target/debug/deps/onnxruntime_sys-22eb0e3e89a0278c
Reason: image not foundTo fix, one can either:
Set the LD_LIBRARY_PATH environment variable to point to the path where the library can be found.
Adapt the .cargo/config file to contain a linker flag to provide the full path:
[target.aarch64-apple-darwin]
rustflags = ["-C", "link-args=-Wl,-rpath,/full/path/to/onnxruntime/lib"]See rust-lang/cargo #5077 for more information.
The C++ example that uses the C API
(C_Api_Sample.cpp)
was ported to both the low level crate (onnxruntime-sys) and the high level on (onnxruntime).
To run this example (onnxruntime-sys/examples/c_api_sample.rs):
…To run this example (onnxruntime/examples/sample.rs):
…See also the integration tests (onnxruntime/tests/integration_tests.rs)
that performs simple model download and inference, validating the results.
Bindings (the basis of onnxruntime-sys) are committed to the git repository. This means bindgen is not
a dependency anymore on every build (it was made optional) and thus build times are better.
To generate new bindings (for example if they don't exists for your platform or if a version bump occurred), build the crate with the generate-bindings feature.
NOTE: Make sure to have the rustfmt rustup component present so that bindings are formatted:
rustup component add rustfmtThen on each platform build with the proper feature flag:
❯ cd onnxruntime-sys
❯ cargo build --features generate-bindingsPrepare the container:
❯ docker run -it --rm --name rustbuilder -v "$PWD":/usr/src/myapp -w /usr/src/myapp rust:1.50.0 /bin/bash
❯ apt-get update
❯ apt-get install clang
❯ rustup component add rustfmtGenerate the bindings:
❯ docker exec -it --user "$(id -u)":"$(id -g)" rustbuilder /bin/bash
❯ cd onnxruntime-sys
❯ cargo build --features generate-bindingsYou can use nbigaouette/windows_vagrant_rust to provision a Windows VM that can build the project and generate the bindings.
Windows can build both x86 and x86_64 bindings:
❯ rustup target add i686-pc-windows-msvc x86_64-pc-windows-msvc
❯ cd onnxruntime-sys
❯ cargo build --features generate-bindings --target i686-pc-windows-msvc
❯ cargo build --features generate-bindings --target x86_64-pc-windows-msvcThe Rust Code of Conduct shall be respected. For escalation or moderation issues please contact Nicolas ([email protected]) instead of the Rust moderation team.
This project is licensed under either of
at your option.
No open issues yet, or sync has not completed.