[RFC] ExecuTorch Build Presets

[jathu]

This is a public version of an internal doc proposing “build presets” for ExecuTorch.

Problem

• There are currently an overwhelming number of build options for ExecuTorch. These options differ based on the targeting platform/architecture, have intertwined dependencies, and scattered throughout the codebase
  • This makes it difficult to maintain the build system, and must rely on the full GitHub CI workflow to validate correctness of any changes
  • This makes it difficult for developers to understand the full scope of the each option. Do most developers even care?
• Building for a specific platform is difficult. Each platform has its own bespoke inconsistent script i.e. Apple, Android, Vulkan, pip, one-off libs (i.e. quantized_ops_aot). This makes it difficult to discover how to build for a specific platform. For example, the Apple build script has its own set of options and defaults, links frameworks independent of the build system, and relies on buck2 for generating headers
  • Building with just cmake requires multiple options to be turned on, which are opaque and not documented
  • Due to the heavy usage of shell scripts, including ExecuTorch as a cmake library will require reconfiguring all the options to match the desired platform
• CI scripts add another layer of configurations on-top of the shell scripts. This implies the CI is running a workflow that might differ from a common developer’s workflow

Goal

• Reduce building for common platforms to a single consistent command
  • Eliminate the need for bespoke build scripts
  • CI uses the same build command as a developer without additional wrappers
• Make it easier to configure builds by creating an API and guardrails
  • Including ExecuTorch as a cmake dependency should be trivially configurable
  • Option inter-dependencies and conflicts should be gracefully handled

Proposal

ExecuTorch builds will be configured with two options:

• CMAKE_TOOLCHAIN_FILE — the idiomatic way to configure toolchains to dictate targeting platform and architecture
  • If this option is not provided, cmake falls back to building for the host
• EXECUTORCH_BUILD_PRESET_FILE — a preset file that configures the build
  • If this option is not provided, fallback to a default platform agnostic build

The build preset file will be a cmake file that conforms to a known API to set build configurations. Example for targeting macos-arm64:

# File: tool/cmake/preset/macos-arm64.cmake

# Default prestets
include(${PROJECT_SOURCE_DIR}/tool/cmake/preset/default.cmake)

# macos arm64 specific overrides
set_executorch_config(EXECUTORCH_BUILD_COREML ON)
set_executorch_config(EXECUTORCH_BUILD_PYBIND ON)

The default preset options will be centrally configured, thus they are validated and ensure valid inter-dependencies:

# File: tool/cmake/preset/default.cmake

define_executorch_config(EXECUTORCH_BUILD_COREML "Enable CoreML backend" OFF)
define_executorch_config(EXECUTORCH_BUILD_PYBIND "Enable pybinding" OFF)

# Ensure platform constraints
define_executorch_config(EXECUTORCH_BUILD_QNN "Enable QNN backend" OFF)
if(EXECUTORCH_BUILD_QNN)
  if(NOT ANDROID)
	message(FATAL_ERROR "EXECUTORCH_BUILD_QNN can only be on for Android")
  endif()
endif()

# Ensure dependencies
if(EXECUTORCH_BUILD_PYBIND)
  set(_default_executorch_build_extension_tensor ON)
else()
  set(_default_executorch_build_extension_tensor OFF)
endif()
define_executorch_config(EXECUTORCH_BUILD_EXTENSION_TENSOR "Build the Tensor extension"  ${_default_executorch_build_extension_tensor})

Types of presets and available targets

Given build presets are just a set of configurations, they can be scoped and packaged arbitrarily. For example, these scopes can be based on:

• Platform
  • The build presets are scoped to a combination of operating system and architecture
    • i.e. macos-arm64, linux-x86_64, android-arm64-v8
    • This implies that, all the buildable targets will be made available and it will up to the users to determine what targets they want to build
  • Positives
    • Once a platform is chosen, it should be trivial to build the executorch library, pybinding, example projects, and any other targets available on the platform without additional configuration
  • Negatives
    • We currently expose a large amount of intermediate targets. This might make it difficult for developers to understand what targets are the “main” targets without documentation
• Application
  • Instead of exposing all the targets per platform, the build presets are instead scoped based on “applications”. Example:
    • libexecutorch-macos-arm64 — builds the main library for macOS arm64
    • pybinding-windows-x86_64 — builds the pybindings library for Windows on x86_64
    • llama-runner-linux-x86_64 — builds the llama runner (and implicitly builds the require executorch library dependency) for Linux on x86_64
  • Positives
    • It is immediately obvious what targets are available to be built
    • We can still achieve platform specific defaults, by composing several of these presets. For example “macos-arm64”, can contain libexecutorch-macos-arm64, pybinding-macos-arm64, llama-runner-macos-arm64, etc.
  • Negatives
    • There will be a large number of presets, which might make it difficult to maintain

We propose to go with application presets, and platform presets that compose the application presets. This will provide us the most flexibility, and ability to consolidate in the future if needed.

Building with CLI

By leveraging cmake-presets, we can achieve a minimal consistent build step:

// File: CMakePresets.cmake
{
  "version": 10,
  "cmakeMinimumRequired": {
    "major": 3,
    "minor": 31,
    "patch": 0
  },
  "$comment": "On-device AI across mobile, embedded and edge for PyTorch.",
  "configurePresets": [
    {
      "name": "common",
      "hidden": true,
      "binaryDir": "${sourceDir}/cmake-out",
      "generator": "Unix Makefiles"
    },
    {
      "name": "macos-arm64",
      "inherits": ["common"],
      "generator": "Xcode",
      "cacheVariables": {
        "CMAKE_TOOLCHAIN_FILE": "${sourceDir}/third-party/ios-cmake/ios.toolchain.cmake",
        "EXECUTORCH_BUILD_PRESET_FILE": "${sourceDir}/tools/cmake/preset/macos-arm64.cmake",
        "PLATFORM": "MAC_ARM64",
        "DEPLOYMENT_TARGET": "10.15"
      },
      "condition": {
        "lhs": "${hostSystemName}",
        "type": "equals",
        "rhs": "Darwin"
      }
    },
    {
      "name": "ios-arm64",
      "inherits": ["common"],
      "generator": "Xcode",
      "cacheVariables": {
        "CMAKE_TOOLCHAIN_FILE": "${sourceDir}/third-party/ios-cmake/ios.toolchain.cmake",
        "EXECUTORCH_BUILD_PRESET_FILE": "${sourceDir}/tools/cmake/preset/ios-arm64.cmake",
        "PLATFORM": "OS64",
        "DEPLOYMENT_TARGET": "17.0"
      },
      "condition": {
        "lhs": "${hostSystemName}",
        "type": "equals",
        "rhs": "Darwin"
      }
    },
    {
      "name": "linux-x86_64",
      "inherits": ["common"],
      "cacheVariables": {
        "CMAKE_SYSTEM_NAME": "Linux",
        "CMAKE_SYSTEM_PROCESSOR": "x86_64",
        "EXECUTORCH_BUILD_PRESET_FILE": "${sourceDir}/tools/cmake/preset/linux-x86_64.cmake"
      },
      "condition": {
        "lhs": "${hostSystemName}",
        "type": "equals",
        "rhs": "Linux"
      }
    },
    {
      "name": "android-arm64-v8a",
      "inherits": ["common"],
      "cacheVariables": {
        "CMAKE_TOOLCHAIN_FILE": "$env{ANDROID_NDK}/build/cmake/android.toolchain.cmake",
        "EXECUTORCH_BUILD_PRESET_FILE": "${sourceDir}/tools/cmake/preset/android-arm64-v8a.cmake",
        "ANDROID_ABI": "arm64-v8a",
        "ANDROID_PLATFORM": "android-26"
      }
    }
  ]
}

# Use presets to build for specific platforms
$ cmake --preset macos-arm64                    && cmake --build cmake-out --target pybinding
$ cmake --preset ios-arm64                      && cmake --build cmake-out --target executorch
$ cmake --preset android-arm64-v8a              && cmake --build cmake-out --target executorch
$ cmake --preset linux-x86_64                   && cmake --build cmake-out --target executorch pybinding
$ cmake --preset llama-runner-linux-x86_64      && cmake --build cmake-out --target all

# Override specific preset options
$ cmake -DEXECUTORCH_BUILD_MPS=ON --preset macos-arm64 && cmake --build cmake-out

# Automatically determine presets based on the host
$ cmake -S . -B cmake-out && cmake --build cmake-out

# Provide a custom preset
$ cmake -DEXECUTORCH_BUILD_PRESET_FILE=my_preset.cmake -S . -B cmake-out && cmake --build cmake-out

Building as a 3rd party library

Including ExecuTorch should automatically determine the preset for well known platforms, otherwise fallback to the default:

add_subdirectory(executorch)

Preset options can be overridden:

# If we are building for macOS, always turn off CoreML
set(EXECUTORCH_BUILD_COREML OFF)
add_subdirectory(executorch)

Presets can be explicitly set:

# Always build for iOS
set(EXECUTORCH_BUILD_PRESET_FILE ${PROJECT_SOURCE_DIR}/third-party/executorch/tools/cmake/preset/ios-arm64.cmake)
add_subdirectory(executorch)

Presets can be extended for custom presets:

# File: my_preset.cmake

include(${PROJECT_SOURCE_DIR}/third-party/executorch/tools/cmake/preset/ios-arm64.cmake)

set_executorch_config(EXECUTORCH_BUILD_MPS ON)

set(EXECUTORCH_BUILD_PRESET_FILE my_preset.cmake)
add_subdirectory(executorch)

Building in CI

By eliminating the bespoke build scripts, CI builds should only use the presets.

Extendability

The presets will be designed with an API that will allow for composability, and encourage developers to create their own bespoke presets. Let’s walk through some examples:

• Startup wants to use ET lib to include it in an iOS and Android app
  • Given these are supported platforms, the developers just need to include the library and it should work out of the box
  • Add the “executorch” target as a lib dependency
• Researcher wants to build ET lib and llama runner on Windows and Linux
  • Given these are supported platforms, the developers just need to include the library and it should work out of the box
  • Add the “executorch” target as a lib dependency
  • Build and run the llama-runner target on the host machine
• Hobbyist wants to build ET lib for a novel platform (i.e. MIPS)
  • The developer will provide a toolchain for that platform using CMAKE_TOOLCHAIN_FILE
  • Without a build preset specified, the default targets will be available. Which includes the executorch library
  • The developer can turn on other features by creating a new preset or composing a preset from the built-in presets

FAQ

• Why not inline the options within the CMakePresets.cmake?
  • Extracting the configuration to its own file allows it to be more flexible, and reusable via composition
• How does this fix the overwhelming number of options?
  • In the process of creating these presets, we will comb through the list of options and eliminate redundant options, and consolidate dependent options
  • By using presets for the most common platforms/workflows, developers shouldn’t need to set options. For one-off option configuration, this can be done idiomatically with presets. If the developer requires many option overrides, they are encouraged to create their own preset
• How does this work with IDEs?
  • Given these are just option configurations, this should not change IDE support
  • cmake-presets are supported in Visual Studio: https://learn.microsoft.com/en-us/cpp/build/cmake-presets-vs

[shoumikhin]

It's probably beneficial for us, core developers, to optimize the building process. And we should keep in mind that ideally most of the customers rarely need to build anything, if at all.

E.g. most of the Apple devs never used or even heard of Cmake in their career, so no matter how easy it is for them to build ET runtime locally, it's still an extra effort. That why on Apple platforms we provide Swift PM binary packages built nightly that devs can just add to their project in a few clicks in Xcode and never care how exactly those are built - with Cmake, Buck or any sort of custom scripts.