· 7 min read
One AI feature, several models: composing on-device and cloud steps in Swift
Small steps, a model per step, one explicit fallback, and privacy you can check. What I learned building AI features as workflows in Swift.
Most AI features I've built didn't fail because of the model. They failed in the code around it.
The demo is always one prompt: paste some text, get a nice answer. Then the real feature arrives. Read a receipt. Pull out the merchant and the total. Check the currency. Look something up. Draft a reply. Let the user review it before anything is sent. Suddenly you have five model calls, three of which could run on the device, one that really needs a bigger model, some plain Swift in between, and a lot of questions nobody asked during the demo. Which model runs which step? What happens when one isn't available? Can this data leave the phone? How many calls is this allowed to make?
I kept writing that coordination code by hand, so I turned it into a library: ModelRelay, an open-source Swift package for composing small model steps into one workflow. This post is about the ideas behind it more than the API. They apply whether you use the library or not.
1. A feature is a process, not a prompt
The first shift is to stop thinking of an AI feature as "the prompt" and start thinking of it as a short pipeline of small, named steps, each with a narrow job.
Here's a receipt turned into typed fields:
import ModelRelay
let workflow = Workflow(
name: "Receipt to expense"
) {
Step.extract("expense", instruction: "Extract the expense from this receipt.", model: .appleOnDevice) {
Field("merchant", .text)
Field("total", .number)
Field("currency", .choice(["EUR", "USD", "GBP"]))
}
}
let report = try await WorkflowExecutor().run(
workflow,
input: .text("BLUE BOTTLE COFFEE · TOTAL EUR 8.30")
)
Small steps are easier to reason about. Each one has an input, an output and a name you can see in logs. They're easier to test, because you can swap a model for a scripted one. And they make the next idea possible.
2. Choose the model per step, not per app
Once the feature is a sequence of steps, there's no reason for every step to use the same model.
Classifying urgency or extracting three fields is a job for a small model, and Apple's on-device model is free, private and already on the device. Drafting a careful reply to an angry customer might deserve a larger cloud model. Reading text from a photo might not need a language model at all: Vision's text recognition is fast and runs locally.
let workflow = Workflow(
name: "Support email",
output: "reply"
) {
Step.extract("request", instruction: "Extract the customer's request.", model: .appleOnDevice) {
Field("orderID", .text)
Field("topic", .choice(["refund", "shipping", "other"]))
}
Step.classify("urgency", "How urgent is this email?",
labels: ["low", "high"], input: "input", model: .appleOnDevice)
Step.prompt("reply", "Draft a short, friendly reply to this email: {{input}}",
model: "claude", fallback: .appleOnDevice)
}
"claude" here is just a name you registered once in a ModelCatalog. It's your workflow's name for a model, not the provider's model ID. The workflow doesn't know or care which provider sits behind it. Under the hood, ModelRelay uses Hugging Face's AnyLanguageModel for the actual inference: Apple's models, OpenAI, Anthropic, Ollama, any OpenAI-compatible server, or MLX running inside your process.
The payoff is economic as much as technical. Most steps in a real feature are boring: extract, classify, route. If those run on-device, you only pay for the one step that needs a bigger model.
3. Fallbacks should be explicit, and there should be one
Apple's on-device model isn't always there. It needs compatible hardware, a supported OS, Apple Intelligence turned on and the model downloaded. Cloud models fail too: no network, rate limits, an expired key.
The tempting answer is a magic chain: try A, then B, then C, then whatever is left. I deliberately didn't build that. Each step in ModelRelay gets one explicit fallback:
Step.readImage("receipt", instruction: "Read the total.",
model: "claude", fallback: .appleTextRecognition)
One fallback is something you can reason about. You know the worst case for cost, latency and where the data goes. A chain of five is a decision you've stopped making.
4. Privacy is a property you should be able to check
"This feature runs on-device" is a sentence that tends to become false quietly. Someone adds a fallback to a cloud model, or a tool that calls an API, and nobody updates the privacy copy.
So in ModelRelay, every registered model has a location, and you can make the workflow prove it:
let executor = WorkflowExecutor(policy: .localOnly)
With .localOnly, every step has to use a model registered as .local, and HTTP and MCP calls are blocked, including calls to localhost. The check runs before the model is even created or the request is sent, so a misconfigured step fails with a policy error instead of quietly sending data out. Apple's on-device model, Vision text recognition and MLX presets count as local. Private Cloud Compute is remote. Anything you register yourself is "unspecified" until you say otherwise, because a URL that says localhost is not proof of anything.
There are smaller versions of the same idea: .readOnly filters out tools that write, .noTools blocks tools entirely, and writes are denied by default unless you supply a WritePolicy that approves them.
5. Give it a budget before you give it tools
The moment a model can call tools, "how many times will this run?" stops being a theoretical question. An agent step that loops, retries and calls an API on every turn can turn a two-cent feature into a two-euro one. On a phone, it can drain a battery.
So the executor takes explicit limits:
let executor = WorkflowExecutor(limits: .init(
maxDuration: .seconds(60),
maxModelCalls: 8,
maxToolCalls: 12,
maxConcurrentSteps: 2
))
When a run finishes, the report tells you how much it actually used. If a workflow blows its budget in testing, that's a design smell you'd much rather find at your desk than in your crash reports.
6. Some steps are just Swift
Not everything needs a model. Trimming text, validating that the line items add up to the total, applying your app's business rules: that's code, and it should stay code.
Step.custom("normalized") { context in
.text(context.input.textValue.trimmingCharacters(in: .whitespacesAndNewlines))
}
Custom steps are trusted, compiled code from your app. They're not a sandbox for running code a model wrote. That distinction matters, and I think it's worth being explicit about in any system that mixes models and code.
7. Be honest about what it doesn't do
This is the part I find most important to write down, because AI libraries tend to overpromise.
- It doesn't make Apple's model faster or remove its hardware, OS or entitlement requirements.
- Workflow events are not token streaming. You get progress per step, not text as it's generated.
- Checkpoints need your storage. You can pause a workflow for human review and resume it later, but you provide durable storage that supports atomic compare-and-swap. Neither checkpoints nor timeouts guarantee that a write happens exactly once.
- Timeouts and cancellation are cooperative, like the rest of Swift concurrency.
Why not just use Foundation Models?
You should, for a lot of things. Foundation Models already gives you sessions, streaming, guided generation and tool calling, and on OS 27 it supports custom model providers. If your feature is a single prompt or a conversation with Apple's model, use it directly. That's the simplest option.
ModelRelay is for the layer above: when one feature becomes several steps across several models, and you want the routing, fallbacks, privacy rules and budgets written down in one place instead of spread across your view models. It also runs on iOS 18 and macOS 15, so you can ship workflows on remote or local models to users who don't have Apple Intelligence yet. Apple's model steps simply light up where they're available.
Using it for real: Runline
The best test of a library is building something with it. Runline is my app for iPhone, iPad and Mac, built on ModelRelay. You build a workflow visually, choose a model and an optional fallback for each step, and run it from Shortcuts. It's available on the App Store.
Try it
ModelRelay is MIT-licensed and on GitHub: github.com/Juanpe/ModelRelay. If you just want to look around without a model or an API key, the standalone example runs a deterministic workflow:
swift run --package-path Examples/StandaloneClient StandaloneClient
It's still early (0.2), so I'd love feedback, especially on the API shape and the privacy and approval model. Open an issue, or find me on X.