ExecuTorch: PyTorch's On-Device AI Runtime

What is ExecuTorch?

ExecuTorch is PyTorch's runtime and toolchain for running trained models on-device, across phones, embedded boards, and microcontrollers, without porting them to another framework first. It captures a model graph with torch.export(), compiles it ahead of time into a .pte file, and runs that file through a lightweight C++ runtime with backends for chips from Apple, Qualcomm, Samsung, MediaTek, and ARM.

Core Capabilities for Deployment

• ✓Native export from torch.export() straight to a .pte file, with no intermediate ONNX or TFLite conversion step, so the model graph's structure survives the trip, per the README.
• ✓A 50KB base runtime footprint, small enough for microcontrollers as well as flagship phones.
• ✓12+ hardware backends listed in the README, including XNNPACK, Apple CoreML, Qualcomm QNN, Samsung Exynos, MediaTek, Vulkan, and ARM Ethos-U for microcontrollers.
• ✓Switch hardware targets by swapping one partitioner line, such as XnnpackPartitioner, CoreMLPartitioner, or QnnPartitioner, instead of re-exporting per chip.
• ✓Quantization support through torchao covering 8-bit and 4-bit precision, plus a dynamic mode, per the README.
• ✓Developer tooling: an ETDump profiler and an ETRecord inspector for looking inside a running .pte model.
• ✓Selective build strips unused operators to cut binary size, and custom operators let you add domain-specific kernels.
• ✓Dynamic shapes are supported for models with variable input sizes within bounded ranges.

Real-World Applications and Supported Models

• •Shipping an LLM chat feature inside a mobile app: the README lists Llama 3.2, 3.1, and 3, Qwen 3, Phi-4-mini, and LiquidAI LFM2 as examples with working export scripts.
• •Running vision or speech models on-device, with MobileNetV2, DeepLabV3, YOLO26, and Whisper listed as example models in the repo.
• •Multimodal apps that combine text with images or audio, using the Llava and Voxtral examples plus the MultiModal runner API.
• •Deploying one trained PyTorch model across Android, iOS, and embedded Linux from a single export, switching only the partitioner.
• •Building for microcontrollers or DSPs, where ARM Ethos-U, NXP, and Cadence DSP backends cover the embedded/MCU row in the platform table.

Getting Started with ExecuTorch

Install the Python package with `pip install executorch`. That's the entire command for the core package. Platform-specific setup for Android, iOS, or embedded targets isn't spelled out step by step in the README itself; it points to a separate Quick Start doc for that. Exporting a model needs `torch`, `executorch.exir`, and a backend-specific partitioner import, such as XnnpackPartitioner for CPU or CoreMLPartitioner for iOS, both shown in the README's own export snippet.

Deploying PyTorch Models On-Device

The flow the README shows has three steps. First, capture the model graph with torch.export.export() on an eval()'d model and example inputs. Second, call to_edge_transform_and_lower() with a chosen partitioner, then .to_executorch(), which quantizes, optimizes, and partitions the graph into a .pte file you write to disk. Third, load that .pte file on-device: the C++ Module API, Swift's Module class on iOS, or Kotlin's Module.load() on Android all call forward() the same way. For an LLM specifically, the README points to the export_llm script or Optimum-ExecuTorch's optimum-cli export executorch command, then running the result through a matching LLM runner API, such as create_llama_runner in C++ or TextRunner in Swift.

Comparing On-Device ML Frameworks

Common Questions about ExecuTorch

The problem it solves

Getting a trained PyTorch model onto a phone or a microcontroller usually means converting it to another format like ONNX or TFLite, hand-porting inference code to C++, and picking a hardware vendor's SDK you're then stuck with. ExecuTorch's README frames its own reason for existing around exactly that friction: keep the same PyTorch APIs from training through deployment, skip the conversion step, and avoid getting locked into one chip vendor's toolchain.

Who should try it — and who should skip

Try ExecuTorch if you already train models in PyTorch and need to ship them on phones, wearables, or microcontrollers without maintaining a second conversion pipeline — the one-export, multiple-backend flow and the C++/Swift/Kotlin runtime APIs are built for that handoff. Skip it if your models come from TensorFlow or another framework, since you'd be adding back the conversion step ExecuTorch exists to avoid for PyTorch users. Skip it too if you need a runtime with a long production track record outside Meta's own apps, since the README doesn't document independent adopters.

Related repositories