Deployment configurations

IREE provides a flexible set of tools for various deployment scenarios. Fully featured environments can use IREE to load programs on demand and to take advantage of multi-threaded hardware, while embedded systems can bypass IREE's runtime entirely or interface with custom accelerators.

Stable configurations

• CPU for general purpose CPU deployment
• CPU - Bare-Metal with minimal platform dependencies
• GPU - Vulkan
• GPU - CUDA/ROCm

These are just the most stable configurations IREE supports. Feel free to reach out on any of IREE's communication channels if you have questions about a specific platform, hardware accelerator, or set of system features.

Compiler target backends

Compiler target backends are used to generate executable code for hardware APIs and device architectures. Compiler targets may implement special optimizations or generate distinct code for certain device/architecture/performance profiles.

When compiling programs, a list of target backends must be specified via

• --iree-hal-target-backends= (command-line)
• target_backends=[...] (Python)

Target backend	Description	Compatible HAL devices
llvm-cpu	Code generation for CPU-like devices supported by LLVM	local-sync, local-task
vmvx	Portable interpreter powered by a microkernel library	local-sync, local-task
vulkan-spirv	Portable GPU support via SPIR-V for Vulkan	vulkan
cuda	NVIDIA GPU support via PTX for CUDA	cuda
rocm	Experimental AMD GPU support via HSACO for ROCm	rocm
webgpu-wgsl	Experimental GPU support on the Web via WGSL for WebGPU	webgpu
metal-spirv	Stale - see Issue#4370 GPU support on Apple platforms via MSL for Metal	metal

Tip

The list of available compiler target backends can be queried with

• iree-compile --iree-hal-list-target-backends (command-line)
• iree.compiler.query_available_targets() (Python)

Runtime HAL drivers/devices

Runtime HAL devices call into hardware APIs to load and run executable code. Devices may use multithreading or other system resources, depending on their focus and the build configuration.

HAL device	Description
local-sync	Synchronous local CPU device with inline execution
local-task	Multithreaded local CPU device using a 'task' executor
vulkan	Portable GPU execution using the Vulkan API
cuda	NVIDIA GPU execution using CUDA
rocm	Experimental AMD GPU execution using ROCm
webgpu	Experimental GPU execution on the web using WebGPU
metal	Stale - see Issue#4370 GPU execution on Apple platforms using Metal

Tip

The list of available runtime HAL devices can be queried with

• iree-run-module --list_devices (command-line)
• iree.runtime.query_available_drivers() (Python)

Additional HAL drivers can also be defined external to the core project via IREE_EXTERNAL_HAL_DRIVERS.