Validate Core ML Models in an iOS App Bundle on a Cloud Mac

An image classification model works correctly during local debugging, and the archive succeeds, yet the delivered app cannot find the resource when it performs its first inference. The problem is usually not the model algorithm. More often, the model was not added to the target, a script copied it to the wrong directory, or an older version with the same name remains in the app bundle. A cloud Mac pipeline therefore needs to verify more than whether the project builds: it must also confirm that the model source can be compiled independently and determine exactly what was included in the final deliverable.

Split model validation into two gates

The first gate processes the .mlmodel or .mlpackage directly, isolating problems with the model format, toolchain, or output directory. The second gate inspects the .app after Xcode finishes archiving and confirms that the compiled .mlmodelc is actually present in the deliverable target. These gates must remain separate. If the pipeline only runs xcodebuild, a failure is much harder to trace quickly to model compilation, target membership, or the resource-copying phase.

Start by pinning the developer directory and recording the tool versions:

set -euo pipefail

export DEVELOPER_DIR="/Applications/Xcode.app/Contents/Developer"
xcodebuild -version
xcrun --find coremlcompiler
swift --version

If multiple Xcode versions are installed on ArmVMS, the job should set DEVELOPER_DIR explicitly instead of relying on the xcode-select state of an interactive shell. Environment variables can differ between pipeline jobs and manual login sessions, so an explicit path makes builds easier to reproduce.

The validation target should be a specific app bundle produced by a specified Xcode toolchain, not a build directory that happened to succeed once on a particular machine.

Compile the Core ML model independently first

Create a separate output directory for each model, and never let multiple jobs share the same temporary directory. The following script accepts both .mlmodel and .mlpackage inputs:

MODEL_PATH="${1:?model path required}"
OUTPUT_ROOT="${2:-$PWD/build/coreml}"
MODEL_NAME="$(basename "$MODEL_PATH")"
MODEL_NAME="${MODEL_NAME%.*}"
MODEL_OUTPUT="$OUTPUT_ROOT/$MODEL_NAME"

rm -rf "$MODEL_OUTPUT"
mkdir -p "$MODEL_OUTPUT"

xcrun coremlcompiler compile "$MODEL_PATH" "$MODEL_OUTPUT"

find "$MODEL_OUTPUT" -type f -print0 |
  sort -z |
  xargs -0 shasum -a 256 > "$MODEL_OUTPUT.files.sha256"

du -sk "$MODEL_OUTPUT"

After the command succeeds, verify at least three conditions: the output directory is not empty, a file manifest can be generated, and the size is not zero. Do not use the compilation directory directly as a long-lived cache because its contents depend on the Xcode version, model contents, and compiler behavior. If reuse is necessary, build the cache key from the model source digest, the output of xcodebuild -version, and the build script version.

Add input validation for generated models

Some projects generate models with Python or conversion tools before the build. In that case, validate the input files before invoking coremlcompiler. The generation job must write to a new directory and atomically move the completed output to the agreed path, preventing Xcode from reading a partially generated model.

Consider recording the following fields in the manifest:

Confirm the final artifacts in the archive

After the independent compilation passes, create a clean archive. To avoid reading leftovers from a previous job, assign dedicated directories for this run’s DerivedData and archive:

RUN_ROOT="$PWD/build/model-validation"
DERIVED_DATA="$RUN_ROOT/DerivedData"
ARCHIVE_PATH="$RUN_ROOT/App.xcarchive"

rm -rf "$RUN_ROOT"
mkdir -p "$RUN_ROOT"

xcodebuild \
  -workspace Example.xcworkspace \
  -scheme Example \
  -configuration Release \
  -destination "generic/platform=iOS" \
  -derivedDataPath "$DERIVED_DATA" \
  -archivePath "$ARCHIVE_PATH" \
  archive

Next, locate the app bundle and export the model inventory:

APP_PATH="$(find "$ARCHIVE_PATH/Products/Applications" -maxdepth 1 -name '*.app' -print -quit)"
test -n "$APP_PATH"

find "$APP_PATH" -type d -name '*.mlmodelc' -print |
  LC_ALL=C sort > "$RUN_ROOT/bundled-models.txt"

test -s "$RUN_ROOT/bundled-models.txt"

Do not limit the search to the main app’s root directory. A model may belong to an extension or be carried inside a resource bundle. The validation script should record complete relative paths and then compare them against the locations permitted by the project. If one model is expected but two directories with the same name are found, immediately check whether the main app, an extension, and a dependency bundle each copied their own instance.

Establish a baseline without guesswork

Model size depends on its structure and the compilation tools, so a single percentage-growth threshold should not be used for every model. A more reliable approach is to store a reviewed baseline for each model and define both absolute and proportional growth limits. Absolute changes are more useful for small models, while proportional changes matter more for large ones.

The baseline file can use a simple tab-separated format:

Classifier.mlmodelc 8421376
Embedding.mlmodelc  27156480

Comparisons must also account for additions, deletions, and renames. A newly added model should not automatically be treated as an error, but the change record must explain its purpose. A deleted model should block the build unless the manifest update has also been reviewed. Treat a rename as deleting the old entry and adding a new one so that historical data is not inherited incorrectly.

Save the manifest as a build artifact, but do not upload sensitive configuration unrelated to the model source. If the model is retrieved through a controlled download step, record only its digest, logical name, and final path inside the bundle. Never print token-bearing download URLs in the logs.

Common failures and delivery checklist

If a model compiles independently but does not appear in the app bundle, check Target Membership, Copy Bundle Resources, resource bundle dependencies, and conditional build settings first. If the model is present in Debug but missing from Release, compare the resource phases and custom script input and output lists for both configurations.

If duplicate models with the same name appear, inventory their complete relative paths before deleting anything. The main app and an extension may legitimately require separate copies; the actual problem is duplication that was not intentionally designed. If the archive size increases suddenly, also check whether the entire generation directory was added to the resource phase, packaging the source model, temporary files, and compiled artifacts together.

Validate the deliverable in this order:

1. Pin DEVELOPER_DIR and record the Xcode version.
2. Run coremlcompiler independently in an isolated directory.
3. Generate digest manifests for the model source and compiled output.
4. Create a Release archive with a completely new DerivedData directory.
5. Enumerate every .mlmodelc inside the .xcarchive.
6. Compare the results with the allowed paths, expected count, and reviewed size baseline.
7. Save the manifests, command output, and archive identifier for later comparison.

This process does not evaluate model accuracy, nor does it replace end-to-end inference testing. It addresses earlier issues that are better suited to automation: whether the model can be compiled, whether it is packaged correctly, and whether the delivered contents changed without explanation. Once these results are captured in a machine-readable manifest, missing models can be detected during archiving instead of at runtime.

Frequently asked questions