Skip to content

Welcome to OAAX

OAAX OAAX

OAAX (Open AI Accelerator eXchange) is an open standard that lets one ONNX model run on many AI accelerators through one conversion step and one runtime API. Export your model from any framework to ONNX, convert it with the accelerator's toolchain, and load it from your application through the same C interface on every chip. No framework or vendor specific integration.

OAAX is an open-source, community-driven project of the LF AI & Data Foundation.

Pick your accelerator

Every page covers the same things: requirements, where to get the runtime and the toolchain, every option, and a complete example you can copy.

  • CPU

    x86_64 and aarch64, Linux and Windows. The reference implementation: nothing to install but Docker.

    CPU

  • NVIDIA (GPU)

    CUDA GPUs, Jetson and DGX Spark. ONNX Runtime or native TensorRT backend.

    NVIDIA

  • Intel (CPU, GPU, NPU)

    OpenVINO based: Intel CPUs, GPUs and the NPU.

    Intel

  • Hailo (NPU)

    Hailo-8, one runtime build per HailoRT version.

    Hailo

  • DEEPX (NPU)

    DX-M1 and DX-H1 on Ubuntu, x86_64 and aarch64.

    DEEPX

  • Run your first model

    Convert the YOLOX object detector, run it on a CPU or an Intel accelerator and see a dog and a bicycle detected.

    Hello World

See the full list of compliant hardware.

OAAX Manager: get a runtime or toolchain in one command

What it is for. Every accelerator ships a runtime library and a conversion toolchain, in several builds (CPU architecture, CUDA or HailoRT version, Ubuntu version). oaax-manager finds the right one for your machine, downloads it, checks its checksum and unpacks it, so you never hunt for download links.

How to use it. It needs uv and nothing else; run it without installing anything:

uvx oaax-manager list                       # everything available; * marks builds that fit this OS and CPU
uvx oaax-manager fetch runtime cpu          # downloads and unpacks the runtime for this machine
uvx oaax-manager fetch toolchain cpu        # saves the toolchain image and prints the `docker load` command

Replace cpu with intel, nvidia-ort, nvidia-trt, hailo or deepx. When a chip has several builds (several CUDA versions, HailoRT versions or Ubuntu versions can all be marked *), it never guesses: it lists them and you choose with --variant.

Command or option What it does
list [--kind runtime\|toolchain] [--vendor V] Show the available runtimes and toolchains
fetch runtime <vendor> Download, verify and extract the runtime for this OS and CPU
fetch toolchain <vendor> Download and verify the toolchain Docker image (.tar)
--variant V Pick a build when several exist (for example cuda12, hailort4.20.0)
--os, --arch Fetch a runtime for another platform (for example --arch aarch64)
--artifact-version X.Y.Z Pin an OAAX version (default: the newest)
-o DIR Where to write

Safe by default

Every download is checked against a pinned SHA-256, interrupted downloads resume, archives are extracted all-or-nothing with strict path checks, and an existing folder is never overwritten.

Install it permanently with uv tool install oaax-manager, or download the single file oaax-manager.pyz from the releases page and run it with any Python 3.9+ (python3 oaax-manager.pyz list). Details: PyPI, GitHub.

How OAAX works

  1. Convert the ONNX model with the accelerator's toolchain. The result is an OAAX bundle for that chip.
  2. Load the accelerator's runtime library in your application, and call runtime_init(config).
  3. Load the model with runtime_load_models(num_models, model_configs). A model is referred to by its index in model_configs (its model_id).
  4. Exchange data asynchronously.
    1. Enqueue inputs with runtime_enqueue_input(model_id, input_tensors), setting a unique id on each Tensors.
    2. Retrieve outputs with runtime_retrieve_output(&model_id, &output_tensors, timeout_ms). The output carries the id of the input it answers.
  5. Clean up with runtime_cleanup().

The application code is the same for every chip: only the runtime library and the model file change. See the specification and the interface header for the full API.

Terminology
  • OAAX: Open AI Accelerator eXchange, a standard (specification, recipe) for deploying AI models across different hardware accelerators.
  • XPU, AI accelerator or AI hardware: Any processing unit that can execute AI models, such as CPUs, GPUs, NPUs or specialized AI accelerators.
  • Compile or Convert: Turning an ONNX model into a format optimized and supported for a specific XPU.
  • Runtime: The shared library (.so or .dll) that implements the OAAX runtime interface to interact with an XPU.
  • Conversion toolchain: The software that compiles ONNX models.
  • Input/Output tensors: Data structures (Tensors) that hold the input and output data for the model.
  • Host or runtime host: The software that uses a runtime to offload model computation to the AI hardware.
  • OAAX-compliant XPU: Any AI accelerator with an implementation of the OAAX standard, both the conversion toolchain and the runtime library.

Learn more