I did not run these models for this article. Everything below comes from Apple's and Google's documentation, with the places they disagree or go silent marked. The last section gives you a test to run on your own devices.

The window: one number, and one number you cannot find

Apple states the limit. The on-device model's context window is 4,096 tokens, and the count "includes instructions, prompts, and outputs for a session instance." Apple also gives you two calls to read it at runtime: contextSize and tokenCount(for:). Use them, because Apple's WWDC26 sample prints a different number from a newer model.

One detail that surprises people: your type definitions count too. Apple tells you to keep @Generable property descriptions "as short as possible" because "long descriptions take up additional context size and can introduce latency." The same goes for tool descriptions.

Google does not state a number on any page I fetched. Its pages say Gemini Nano has "token limits" and "device hardware limitations," and give advice instead: keep the prompt concise, keep the output short, and keep system instructions under about 150 words. Output length matters because, in Google's words, "LLM inference speeds are heavily dependent on the output length."

So the planning rule is the same on both: budget by measuring the prompt, the schema, the tool descriptions and the expected reply together, then leave room.

Typed output: both platforms have it, with different catches

Free text is the model's default. For anything your code has to read, ask for a typed value.

Apple calls it guided generation. You mark a Swift type @Generable and the framework uses constrained sampling, which "prevents the model from producing malformed output." It works on structures, enumerations and nested types, and on basic types like Bool, Int, Float, Double and Array. Properties are generated in the order you declare them. For a fixed label set, an enum is the natural fit:

@Generable
enum Sentiment {
    case positive, neutral, negative
}

let response = try await session.respond(to: review, generating: Sentiment.self)
let label: Sentiment = response.content

If you do not know the shape until runtime, Apple offers DynamicGenerationSchema, for example to limit the model to today's menu.

Google has the same idea in Kotlin, with the same macro names but a heavier setup. You annotate a data class with @Generable and @Guide, add the KSP plugin and a separate schema-compiler dependency, and wrap your request with generateTypedContentRequest:

@Generable
data class ReviewResult(
    @Guide(
        description = "Overall sentiment of the review",
        enumValues = ["positive", "neutral", "negative"]
    )
    val sentiment: String
)

val base = GenerateContentRequest.Builder(TextPart(prompt)).build()
val typed = generateTypedContentRequest(
    generateContentRequest = base,
    outputClass = ReviewResult::class
)
val result: ReviewResult? =
    generativeModel.generateContent(typed).candidates.firstOrNull()?.response

Two lessons from the table. First, check feature availability separately from model availability on Android, because the typed API is its own feature. Second, the schema compiler is alpha. Pin its version.

Tool calling: the model can ask your code for help

A tool is a function the model may call to fetch data or take an action. It is how a 4,096-token model answers questions about data it cannot hold.

Apple's Tool protocol takes a name, a description, a @Generable arguments type and a call method. The model decides whether a tool would help, and "can call a tool multiple times in parallel," so your tool code must handle concurrent calls. Tool output can be a string or any @Generable type. Apple also gives you a lever most teams miss: toolCallingMode, with three settings, allowed (the default), required, and disallowed. Required forces a lookup before an answer. If you set required, Apple warns you must define an exit, by throwing from the tool or changing the mode after the first call, "otherwise, the model continues to call the tool." I could not confirm from the pages I read which OS version added toolCallingMode, so check your deployment target.

Google's version is the genai-agent library and its AgentRunner. It is alpha: Google says it "is not subject to any SLA or deprecation policy," needs Android 14 and enrollment in the AICore Developer Preview, and can break compatibility. Tools can be defined as Android AppFunctions. The runner stops after maxTurns, which defaults to 10, "to prevent runaway execution loops, conserve battery, and protect device responsiveness." Google's caution applies to both platforms: model predictions "might not always suggest the correct tool or arguments," and you are responsible for the actions taken.

One privacy point from Google's page is worth repeating. The reasoning loop runs on the device, but "individual tools implemented in your app can still make network calls." A tool that calls your server sends data off the phone, whatever the model's location.

There is also an open disagreement. Google's ML Kit page documents tool execution through AgentRunner. A July 2026 third-party post claims the separate ADK for Android GenaiPrompt path has no tool execution. I have not verified that claim, and it may be out of date. If you use the ADK path, test tool calls on a real device before you plan around them.

Images: both platforms say yes, with conditions

Apple's on-device model accepts images alongside text, through an Attachment in the prompt. The framework scales and converts images for you. For classification, Apple's own sample combines an image with a @Generable enum and greedy sampling, so you get one of your labels and nothing else. Apple's advice is to try the on-device model first, and move to Private Cloud Compute only if the task needs more reasoning or context.

Google's Prompt API accepts "text or multimodal (image and text) inputs." Its device lists are the catch: which phones support the Prompt API depends on the Gemini Nano version, so check the list before you promise an image feature in a market. For text pulled from a photo, Google suggests running its text-recognition API first and then passing the text to the model.

What neither model should be asked to do

Apple publishes the list. It says the on-device model is a fit for summarizing, extracting entities, understanding and refining text, classifying, tagging, and creative writing. It says the model "may not be suitable" for basic math, generating code, or logical reasoning, and gives examples of each.

Google's advice reads the same way in different words: prefer simple logic and a focused task. If a prompt needs "do X first, if the result is A, do M, otherwise N," split it into separate calls and chain them in your code. Google's reference mentions a thinking mode for harder tasks, which I did not fetch.

Google's prompt guidance has a few specifics worth copying:

• Use ## delimiters between prompt sections. Google calls this "particularly critical for Gemini Nano."
• Start with a temperature of 0.2 for extraction and translation.
• Add examples, and use prefix caching when they make the prompt long.

A test you can run on your own phones

The documentation tells you what is allowed. It does not tell you how often the model gets it right on your data. Run this before you choose:

1. Collect 30 real inputs for one feature, and write the correct answer for each by hand. Include the awkward ones: very short, very long, mixed language, profanity.
2. Run each input through Apple's model, Gemini Nano and your external model, with the same instructions.
3. For each run, record whether the output was valid and typed, whether it was correct, how long it took, how many tokens it used, and whether the call errored (guardrail, refusal, busy, unavailable).
4. Count the failures by kind. A wrong answer, a refusal and an error need different fixes.
5. Repeat on the oldest phone you support, not only your own.

You will end up with a table of about 90 rows. It will tell you more than any vendor page, and it is the input Part 6 of this series uses to compare cost and latency.

What this series does next

Part 4 turns the table into a decision: which features go on the phone, which go to the cloud, and which need both.

#AI #iOS #Android #OnDeviceAI #MobileEngineering

Work with me: I take 1:1 calls on mobile architecture reviews, mobile and AI questions, and startup tech consults. Book a slot on Topmate: topmate.io/sairamkotha

Sources

Apple (fetched as Apple's documentation JSON)

• Generating Swift data structures with guided generation. Constrained sampling, supported types, declaration order, short descriptions, dynamic schemas.
• Expanding generation with tool calling. The Tool protocol, parallel calls, toolCallingMode, the exit condition.
• Analyzing images with multimodal prompting. Attachment, image types, the label-enum sample, "on-device first."
• Generating content and performing tasks with Foundation Models. The capability table and the "may not be suitable" list.
• SystemLanguageModel.contextSize and GenerationError.exceededContextWindowSize. The 4,096 figure and what counts toward it.
• Improving the safety of generative model output. Refusal behaviour for typed requests.

Google (fetched)

• Structured output guide. @Generable, @Guide, supported types, KSP, schema compiler version, ProGuard, isStructuredOutputFeatureAvailable().
• Prompt design guide. Delimiters, temperature, short output, focused tasks.
• System instructions guide. The 150-word advice.
• Build agentic workflows with Gemini Nano and Get started with agentic workflows. Alpha status, AgentRunner, AppFunctions, maxTurns, Android 14 and Developer Preview, tools making network calls.
• Send custom requests with Gemini Nano direct model access. Text and multimodal inputs, beta status.
• Overview of the ML Kit GenAI APIs. Device lists by Nano version.

What I could not verify

• Any measured accuracy, latency or token figure. I ran nothing for this article. The test above is how you get them.
• Which OS version added Apple's toolCallingMode and the dynamic profile API. The pages I read do not say.
• A token limit for Gemini Nano. Google's pages mention limits and give no number.
• Which Android phones support image input. The device list covers the Prompt API as a whole, and I did not find an image-specific list.
• Whether the ADK for Android GenaiPrompt path can execute tools. A third-party post says it cannot. Google's AgentRunner page says tool calling works through ML Kit.
• Google's thinking mode. A Google page mentions it, and I did not fetch it.
• Whether Apple's model refuses on a typed request in every case. Apple's page says it throws a refusal error. I did not test it.

The task: take a customer review and return a sentiment, one word, positive, neutral or negative. It is small on purpose. The point is the plumbing around the call, not the answer.

Apple: check first, then ask

Apple's on-device model is SystemLanguageModel. Apple's documentation says to check availability before you create a session, and to "plan for a fallback experience in case the model is unavailable." It can also take time: Apple says the model may still be downloading after someone turns on Apple Intelligence.

import FoundationModels

func sentiment(for review: String) async throws -> String? {
    let model = SystemLanguageModel.default
    switch model.availability {
    case .available:
        break
    case .unavailable(.deviceNotEligible):
        return nil          // Permanent. Show the non-AI version.
    case .unavailable(.appleIntelligenceNotEnabled):
        return nil          // The user can turn it on in Settings.
    case .unavailable(.modelNotReady):
        return nil          // Downloading, or another system reason. Retry later.
    case .unavailable(let other):
        return nil          // A reason Apple has not listed yet.
    }

    // Single-turn task: a new session each time.
    let session = LanguageModelSession(
        instructions: "Classify the review as positive, neutral or negative. Reply with one word."
    )
    let response = try await session.respond(to: review)
    return response.content
}

Four details that matter more than the code:

• One session per review. Apple says to "create a new session each time you call the model" for a single-turn task, and to reuse a session only for multi-turn chat. A session keeps a transcript, and the transcript counts toward the context window. Reusing one session across a hundred reviews fills it.
• Instructions are yours, the review is the prompt. Apple trains the model to obey instructions over prompts, so keep the fixed text in the instructions and the user's text in the prompt.
• Check Apple's own list of what the model is for. Apple lists classify and judge text, summarize, extract entities and tag as good fits. It lists basic math, code generation and logical reasoning as things to avoid. A sentiment label is on the good list. A feature that adds up an order total is not.
• The content tagging model is not a classifier. Apple ships a second use case, SystemLanguageModel(useCase: .contentTagging), which "always responds with tags": short lowercase topic, action, object and emotion words. It will give you an emotion tag like "delight," not one of three fixed labels. For a fixed label set, use the general model and a typed result, which Part 3 covers.

Reviews also carry a trap. Apple's guardrails check both the prompt and the output. A review with profanity or a sensitive topic can throw a guardrail error instead of returning a label. Apple's permissiveContentTransformations setting lets a String response go through without throwing, though the model can still refuse. When you ask for a typed value instead of a string, the permissive mode behaves like the default and still throws. Plan for both: catch the error and show "could not classify," not a crash.

If you know the user is about to ask, Apple's prewarm(promptPrefix:) loads the model early. Apple says to use it only when you have at least one second before the request, and that the system may not honour it if the app is in the background or the device is under load.

Google: a status check with four answers

On Android, Gemini Nano runs through ML Kit's direct model access. Google's integration guide gives the status check four values, not two: UNAVAILABLE, DOWNLOADABLE, DOWNLOADING and AVAILABLE. The model is only usable when it is AVAILABLE. When it is DOWNLOADABLE, your app is the one that starts the download.

import com.google.mlkit.genai.prompt.Generation

class SentimentModel {
    private val model = Generation.getClient()

    suspend fun classify(review: String): String? {
        when (model.checkStatus()) {
            FeatureStatus.UNAVAILABLE -> return null      // Unsupported, or config not fetched yet.
            FeatureStatus.DOWNLOADING -> return null      // Try again later.
            FeatureStatus.DOWNLOADABLE -> {
                model.download().collect { status ->
                    when (status) {
                        is DownloadStatus.DownloadStarted -> { /* show progress */ }
                        is DownloadStatus.DownloadProgress -> { /* status.totalBytesDownloaded */ }
                        DownloadStatus.DownloadCompleted -> { /* ready */ }
                        is DownloadStatus.DownloadFailed -> { /* status.e.message */ }
                    }
                }
                return null                                // Answer on the next call.
            }
            FeatureStatus.AVAILABLE -> Unit
        }

        val request = generateContentRequest(
            SystemInstruction("Classify the review as positive, neutral or negative. Reply with one word."),
            TextPart(review)
        )
        return try {
            model.generateContent(request).candidates.firstOrNull()?.text
        } catch (e: GenAiException) {
            null
        }
    }

    fun close() = model.close()   // Call from onCleared() or onDestroy().
}

The structure comes from Google's sample code. The names Generation.getClient(), checkStatus(), download(), FeatureStatus, DownloadStatus, generateContentRequest, SystemInstruction, TextPart, candidates and GenAiException all appear on Google's Prompt API pages. I have not compiled this exact class.

Six details that matter:

• System instructions need a newer model. Google's page says they "work on devices running Gemini Nano V3 and higher." On an older device, put the instruction text in the prompt instead. Google also recommends keeping system instructions under 150 words, and counting them in your token estimate.
• Close the client. Google says to call close() when the Activity, Fragment or ViewModel is destroyed. That is why the model lives in a class with a close(), not in a one-off function.
• Warm up for the first call. Google offers an optional warmup() that loads Gemini Nano into memory before the first inference. Apple's equivalent is prewarm.
• Foreground only. Google's overview says inference is allowed only when your app is the top foreground application. A background job gets BACKGROUND_USE_BLOCKED.
• Streaming is a separate call. generateContentStream returns chunks. Google says to use streaming for long replies and the non-streaming call for short ones. A one-word label is short.
• The version numbers disagree. Google's get-started page shows com.google.mlkit:genai-prompt:1.0.0-beta2. Google's integration skill says to use "at least 1.0.0-beta4." The API is in beta and Google says it can "break backward compatibility." Pin the version and read its release notes before you update.

Typed output exists here too, with the same names Apple uses. Google's structured output page uses @Generable and @Guide annotations, but it is Kotlin only, needs the KSP plugin at version 2.3.6 or higher and a separate schema-compiler dependency that Google lists as 1.0.0-alpha1, and can break under ProGuard unless you add keep rules. Part 3 compares the two.

External model: never from the phone

The third call goes to an external model, such as Claude. The rule from Part 1 applies: the API key does not go in the app. Your app calls your own backend, and the backend calls the provider.

On the phone, this is an ordinary network call:

struct SentimentRequest: Encodable { let review: String }
struct SentimentResponse: Decodable { let sentiment: String }

func sentimentFromServer(review: String) async throws -> String {
    var request = URLRequest(url: URL(string: "https://api.example.com/sentiment")!)
    request.httpMethod = "POST"
    request.setValue("application/json", forHTTPHeaderField: "Content-Type")
    request.httpBody = try JSONEncoder().encode(SentimentRequest(review: review))

    let (data, _) = try await URLSession.shared.data(for: request)
    return try JSONDecoder().decode(SentimentResponse.self, from: data).sentiment
}

On your backend, the key stays in an environment variable. This is the request shape from Anthropic's quickstart:

curl https://api.anthropic.com/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-haiku-5-5",
    "max_tokens": 5,
    "system": "Classify the review as positive, neutral or negative. Reply with one word.",
    "messages": [{"role": "user", "content": "The battery lasts all day. Love it."}]
  }'

Four details:

• The reply is a list of blocks. The response has a content array, and the quickstart's own code loops over it and keeps the blocks whose type is text. Do not read content[0] blindly.
• The usage comes with the reply. The response includes usage with input_tokens and output_tokens. That is the external side of your receipt.
• The fee is per request. The backend is where you put the spend limit and the rate limit, so a bug in one app version cannot run up the bill.
• The model name is a setting. Keep it in configuration, so you can move to a smaller model without an app release. The quickstart I fetched shows claude-opus-5-5 as its example, and claude-haiku-5-5 is the smaller model I used here. Check the current model list when you build.

The same receipt from all three

Whichever path answers, log one record: which backend answered, how long it took, and the token counts. Apple reports tokens on each response's usage, with input and output counts and a total, and a running total on the session. Anthropic returns usage with input_tokens and output_tokens. I did not find a documented token count in Google's direct-access response, so for Gemini Nano log the time and your own estimate. Part 6 builds the comparison on this record.

What this series does next

Part 3 covers what the default models can and cannot do: the context limits, typed output on each platform, and tool calling. You will run the same review set through each path and see where the answers differ.

#AI #iOS #Android #OnDeviceAI #MobileEngineering

Work with me: I take 1:1 calls on mobile architecture reviews, mobile and AI questions, and startup tech consults. Book a slot on Topmate: topmate.io/sairamkotha

Sources

Apple (fetched as Apple's documentation JSON)

• Generating content and performing tasks with Foundation Models. Availability sample, "plan for a fallback," one session per single-turn call, the capability and "avoid" lists.
• Categorizing and organizing data with content tags and SystemLanguageModel.UseCase.contentTagging. "Always responds with tags."
• SystemLanguageModel.Guardrails.permissiveContentTransformations. String versus typed behaviour.
• LanguageModelSession.prewarm(promptPrefix:) and LanguageModelSession.Usage.
• SystemLanguageModel.Availability.UnavailableReason.
• WWDC25: Meet the Foundation Models framework. Instructions versus prompts.

Google (fetched)

• Prompt API get-started guide. Generation.getClient(), checkStatus(), the four FeatureStatus values, download(), generateContent, generateContentStream, warmup(), API level 26, 1.0.0-beta2.
• System instructions guide. Nano V3 and higher, the 150-word advice, SystemInstruction.
• Structured output guide. @Generable, @Guide, KSP 2.3.6, schema compiler 1.0.0-alpha1, Kotlin only, ProGuard.
• Integration skill. "At least 1.0.0-beta4," close().
• Send custom requests with Gemini Nano direct model access. Beta notice.
• Overview of the ML Kit GenAI APIs. Foreground-only rule, BACKGROUND_USE_BLOCKED.

Anthropic (fetched)

• Get started with Claude. The curl request, headers, response shape and usage fields.
• Messages API reference. The page I fetched was truncated, so I used only the fields it showed.

What I could not verify

• That the Kotlin class compiles. The API names are from Google's pages, and I did not build the code. The generateContentRequest import path and the exact GenAiException package are not shown on the pages I read.
• Whether the foreground-only rule still applies on every Nano version. The overview page states it without a version.
• A token count in Google's direct-access response. I found none documented.
• That claude-haiku-5-5 is current on the day you publish. The quickstart I fetched uses claude-opus-5-5 as its example.
• Whether Anthropic requires max_tokens on every request. The Messages reference I fetched lists it as a body parameter but did not state that it is required.
• Latency, cost and accuracy for each path on a real review set. Parts 3 and 6 measure them.

This is Part 1 of 10 in a series on using the AI built into the phone, and comparing it with the models you can rent from outside. This part covers the three places, what each one costs, and the details in Apple's and Google's documentation that catch teams after launch.

The three places

On the phone. The model runs on the user's device. On iPhone it is Apple's Foundation Models framework, which Apple calls the SystemLanguageModel. On Android it is Gemini Nano, which runs inside a system service called AICore, and your app reaches it through ML Kit.

A private cloud run by the platform. Apple also offers a larger model on its own servers, the Private Cloud Compute model, or PCC. Your app calls it through the same framework. Apple's documentation says switching to it is a one-line change.

An external cloud. Claude, the Gemini API, OpenAI and others. Your app, or your backend, sends the prompt to the provider's servers and gets the reply back.

What each one means for you

Apple's own comparison table for its two models says the same thing in five rows: both preserve privacy, only the on-device model works offline, the on-device model has unlimited usage while PCC has a limit per day, only PCC supports reasoning, and the context sizes are 4K and 32K.

Read the whole table as a trade. The on-device path keeps data local and works offline, but it is the smallest model. The external path has the strongest model, but each reply carries a fee and a data-handling agreement.

Hidden detail 1: "available" is not a yes or no

Apple's framework does not have an on/off switch. The availability value is an enum with two cases. When it is unavailable, it carries a reason. Apple's documentation lists three:

• deviceNotEligible: "The device does not support Apple Intelligence." Permanent. Hide the feature or show the non-AI version.
• appleIntelligenceNotEnabled: "Apple Intelligence is not enabled on the system." The user can turn it on in Settings. Prompt once. Do not nag.
• modelNotReady: "The models aren't available on the user's device." Apple's sample comments that this covers downloading "or other system reasons." Temporary. Retry later and do not show an error.

Apple's sample also keeps a catch-all branch for reasons that do not exist yet:

switch model.availability {
case .available:
    // Show your intelligence UI.
case .unavailable(.deviceNotEligible):
    // Show an alternative UI.
case .unavailable(.modelNotReady):
    // The model isn't ready because it's downloading or because
    // of other system reasons.
case .unavailable(let other):
    // The model is unavailable for an unknown reason.
}

One more case hides behind the check. Apple's assetsUnavailable error says the model's files can disappear while your app is running, for example "if the user disables Apple Intelligence while your app is running," and that you "may be able to recover from this error by retrying later after the device has freed up enough space to redownload model assets." So the availability check is not a one-time gate. The model can leave mid-session.

Gemini Nano has the same shape. Google's own guidance says the model "must be fully downloaded and available before calling the first inference," and that you check its FeatureStatus is AVAILABLE first.

Hidden detail 2: the on-device model has a context of 4,096 tokens, and it counts everything

Apple's error documentation for exceededContextWindowSize is the clearest statement I found: the error "occurs when you use the available tokens for the context window of 4,096 tokens. The token count includes instructions, prompts, and outputs for a session instance."

It also gives a rough token rule: "approximately three to four characters in languages like English, Spanish, or German, and one token per character in languages like Japanese, Chinese, and Korean." A Japanese prompt spends the window about four times faster than an English one.

Do not hardcode 4,096. Apple's contextSize property returns "the total number of tokens that can be used in a single session, including both input prompts and generated responses," and Apple's WWDC26 session prints 8,192 from a newer model. Article 21 in this series covers why three numbers exist. Read the value from the device, and when the session overflows, Apple's advice is to "start a new session" and "try again using a shorter prompt or shorter output length."

Apple also gives you the receipt. Each response has a Usage value with input and output token counts, and the session keeps an accumulated usage total. That is the number to log.

Hidden detail 3: the model changes under you

Apple's documentation says it "periodically updates SystemLanguageModel in routine OS updates," and that there are currently three model versions, aligned to OS 26.0 to 26.3, to 26.4, and to 27.0. Apple's guide on updating prompts says to test your prompts against the old and new model, record what the old model produced, and version your prompts, for example by OS availability checks or by fetching them from a server.

Google says the same about Gemini Nano. Its overview page lists devices by model version, nano-v2, nano-v3 and nano-v4, and warns that "different versions of Gemini Nano may return different output from the same prompt." There is a getBaseModelName() call to find out which version a device runs.

For a product team, this means an on-device prompt is not written once. It is retested every OS release, on every model version your users have.

Hidden detail 4: Android has quotas and a foreground rule

Apple says on-device usage is unlimited, with one exception: the rateLimited error "will only happen if your app is running in the background and exceeds the system defined rate limit."

Google is stricter. Its ML Kit GenAI overview says "AICore enforces an inference quota per app," that too many requests in a short period return ErrorCode.BUSY, and that a long-duration quota such as a daily quota can return ErrorCode.PER_APP_BATTERY_USE_QUOTA_EXCEEDED. Google suggests exponential backoff for BUSY.

The foreground rule is the one that breaks designs. Google's page says inference "is permitted only when the app is the top foreground application," and that running it from the background, "including using a foreground service," returns ErrorCode.BACKGROUND_USE_BLOCKED. A background summarizer, an indexing job, or a notification pipeline cannot use Gemini Nano through this API. Plan those features for the foreground, or for the cloud.

Hidden detail 5: the Android device list is long, specific and versioned

Google's overview page now lists the devices that support direct model access, grouped by Nano version. The nano-v3 group includes the Pixel 9 and Pixel 10 families, Samsung's Galaxy S26 family, OnePlus 15, OPPO Find X8 and X9, Sony Xperia 1 VIII, and several vivo, Honor and Lenovo devices. The nano-v2 group covers older flagships from Xiaomi, OnePlus, Motorola, Samsung foldables and others. A nano-v4 group lists the Pixel 11 family and Samsung's Galaxy Z Flip8 and Fold8.

Two things to notice. The Pixel 8 family is not in the direct-access list. And Samsung's Galaxy S25 appears only in the list for the task-specific APIs (summarization, proofreading, rewriting, image description), not in the list for custom prompts. Which API you use decides which phones you reach.

Google also notes that language support "may vary depending on the particular device's configuration and the models that have been downloaded to the device." Check the device list and the language before you promise the feature in a market.

Hidden detail 6: Apple's cloud has a daily budget, and a button to buy more

Apple's PCC model is for when the on-device model is not enough. Apple's guidance is to "start with the on-device model and evaluate it," then move to PCC if your feature needs more reasoning or context. PCC needs iOS 27 or later, a network connection, a device that supports Apple Intelligence, and an entitlement that Apple grants after an eligibility review.

The hidden part is the quota. Apple's documentation says people get "a daily request limit" and "can upgrade their iCloud+ subscription to get more access." The framework exposes this as quotaUsage, with isLimitReached, a status that reports when a person is approaching the limit, a resetDate, and a limitIncreaseSuggestion that shows Apple's upgrade UI. Apple asks you to show the person their usage status in your UI, not just an alert, and to treat quotaLimitReached differently from a rate limit, because the person has to wait for the reset or upgrade.

So the "free" tier has a meter on it, and the meter belongs to the user, not to you. The WWDC26 cost note still holds: Apple says there are no cloud API costs for developers with fewer than 2 million first-time downloads. Both statements are true at once. You do not pay, and the user has a daily cap.

PCC has its own unavailable reasons, deviceNotEligible and systemNotReady. Apple's advice when the network fails is to "retry the request using the on-device model."

Hidden detail 7: safety is a layer you partly control on iPhone and not at all on PCC

Apple's safety guide says the framework has two base layers: models trained to handle sensitive topics, and guardrails that check both the input prompt and the output. On the on-device model you can choose a guardrail level through SystemLanguageModel.Guardrails, which has a default and a permissiveContentTransformations option. On PCC the guardrails "have different policies that you can't directly configure."

Two behaviors matter for your UI. A guardrail violation throws an error. A refusal may not. Apple says that for a plain string response, a refusal "might begin with" text like "Sorry, I can't help with that," and that you "might not be able to programmatically determine whether a string response is a normal response or a refusal." If you use guided generation to get a typed Swift value instead, a refusal throws a refusal error you can catch and explain.

External models have their own policies and their own refusal styles. Whichever path you pick, design the screen for the refusal, not only the answer.

Hidden detail 8: the fallback is the product, and Google's SDK will tell you which model answered

Google's hybrid inference page says cloud models are the fallback when on-device models "are unavailable due to device hardware or OS constraints," and that routine tasks on the device "reduce cloud inference costs." The Firebase AI Logic Hybrid API has four modes:

• PREFER_ON_DEVICE: try the device, fall back to cloud if unavailable or unsupported.
• PREFER_IN_CLOUD: try the cloud when online, fall back to the device only when offline.
• ONLY_ON_DEVICE and ONLY_IN_CLOUD: no fallback, throw instead.

val model = Firebase.ai(backend = GenerativeBackend.googleAI())
    .generativeModel(
        modelName = "gemini-3.5-flash",
        onDeviceConfig = OnDeviceConfig(mode = InferenceMode.PREFER_ON_DEVICE)
    )

Three details hide in the configuration page. The SDK "does not have a default cloud-hosted model," so you must name one for every mode that can reach the cloud. The response carries an inferenceSource property that says whether the device or the cloud answered, which is the field to log. And the feature is Experimental, "not subject to any SLA or deprecation policy."

Google's page also lists the inputs that apps like Gboard use for custom routing: network latency, battery level, processor load and query complexity. Articles 31 and 32 cover how to read the battery and heat signals.

Apple's version of the fallback is the LanguageModel protocol. WWDC26 says SystemLanguageModel and PrivateCloudComputeLanguageModel both conform to it, that Apple is open-sourcing CoreAILanguageModel and MLXLanguageModel for other local models, and that Anthropic and Google publish Swift packages for their server models. The session code stays the same; the backend changes. Apple's warning for third-party models: you handle authentication and per-token billing, and "never store private keys in your app binary."

Hidden detail 9: it is built in, but it is still beta on Android

Apple says the on-device model is "built into the operating system, so it won't increase your app size." Google says AICore manages the model download and updates, and that task APIs share the base model, which "conserves storage space." Neither adds weight to your binary.

The release risk is elsewhere. Google's Prompt API page says the API "is offered in beta, and is not subject to any SLA or deprecation policy. Changes may be made to this API that break backward compatibility." Google's own integration guide sets the minimum as Android API level 26 and version 1.0.0-beta4 of com.google.mlkit:genai-prompt. The ML Kit GenAI terms add that you must tell users about "Google's processing of metrics data." A feature built on this needs a fallback path and a plan for an API change.

Where the cost actually lands

The on-device call has no usage fee from Apple or Google. The phone still pays. Apple's prewarm method, which loads the model before the first prompt, says it should only be called with at least a one-second window before the request, and that the system may not load the assets "if your app is running in the background or the system is under load." That is a model competing for the same chip as the rest of the phone. A long session warms it, drains the battery and, as article 31 shows, can slow the reply.

The external call costs you on every reply, and grows with the conversation. Article 12 covers how that bill grows. The cost is visible, which makes it easier to plan.

A quick way to choose

Ask three questions about one feature:

1. Must the text stay on the device? If yes, start on-device, and stop there unless quality fails.
2. Must it work with no network, or in the background? Offline means on-device. Background on Android means cloud, because Gemini Nano refuses background calls.
3. Does it need the strongest model, or more than 4,096 tokens? Long reasoning, coding and long documents usually do. Summaries, rewrites and short suggestions usually do not.

Most features answer these three questions differently. Part 4 covers that decision in detail.

What this series will do next

Part 2 writes the same small task three ways: Apple's LanguageModelSession, Gemini Nano through ML Kit's direct model access, and one external call. You will see the availability check, the token receipt, and the error cases above in working code.

#AI #iOS #Android #OnDeviceAI #MobileEngineering

Work with me: I take 1:1 calls on mobile architecture reviews, mobile and AI questions, and startup tech consults. Book a slot on Topmate: topmate.io/sairamkotha

Sources

Apple developer documentation (fetched as Apple's documentation JSON)

• SystemLanguageModel. Model versions by OS, availability sample.
• SystemLanguageModel.Availability.UnavailableReason. The three reasons and their one-line descriptions.
• contextSize. What the number includes.
• GenerationError.exceededContextWindowSize. The 4,096 figure and the characters-per-token rule.
• GenerationError.assetsUnavailable, rateLimited, concurrentRequests.
• LanguageModelSession.Usage, prewarm(promptPrefix:).
• Adding server-side intelligence with Private Cloud Compute. The comparison table, daily limit, iCloud+ note, quota API, network fallback advice.
• PrivateCloudComputeLanguageModel.QuotaUsage.
• Updating prompts for new model versions.
• Improving the safety of generative model output.
• Supporting languages and locales with Foundation Models.
• WWDC25: Meet the Foundation Models framework. Built-in, no app size increase, instructions versus prompts.
• WWDC26: What's new in the Foundation Models framework. The 8,192 contextSize output, the 2-million-download note, the LanguageModel protocol, third-party packages, billing and key advice.

Google (fetched)

• Overview of the ML Kit GenAI APIs. Device lists by Nano version, getBaseModelName(), per-app quota, error codes, foreground rule, shared base model, language note.
• Send custom requests with Gemini Nano direct model access. Beta status, inputs, multi-turn, structured output, task API trade-offs.
• Gemini Nano on Android. AICore, offline, privacy, memory budget statements.
• Hybrid inference. The four modes, cost and reach rationale, custom routing inputs, Gboard and Kakao Mobility examples.
• Configuration options for hybrid experiences in Android apps. No default cloud model, inferenceSource, Experimental status, on-device model categories.
• ML Kit GenAI API additional terms. Metrics data disclosure, experimental technology.
• Google's ML Kit GenAI Prompt API integration skill. API level 26, 1.0.0-beta4, FeatureStatus check, close(), prefix caching above 200 words.

What I could not verify

• The on-device context size on the device you ship to. Apple's error doc says 4,096, Apple's PCC table says 4K, and the WWDC26 sample prints 8,192. The number depends on the OS and model version, so read contextSize at runtime.
• The 2-million-download threshold and what applies above it. The WWDC26 session states it briefly and points to a separate video on PCC.
• The exact daily PCC request limit per user, and what iCloud+ tiers change. Apple's documentation describes the mechanism, not the numbers.
• The Gemini Nano Prompt API token limits. Google's pages say "token limits" exist and do not give a number.
• Whether the Android device list is complete on the day you publish. Google's page says support will expand, and it was last updated 2026-10-27.
• Latency and cost figures for each path. This article makes no speed claims. Part 6 measures them.
• Whether Apple charges anything for the on-device model. No Apple page I read states a price either way. The table's "None" rests on the absence of any fee in the documentation.

Phones work the same way. When an AI model runs on a phone, the operating system knows when the phone is getting hot, when memory is running low, and why an app was shut down. It will tell your app, but only if the app signs up to listen. Most apps never do. They find out about the problem from a one-star review.

This article shows what iOS and Android can tell an app, how an app signs up, and what to do with the answers. Each section starts in plain words. Where engineers need more, a "For engineers" part follows with the exact calls and code.

It is the follow-up to three earlier articles: why on-device AI crashes after an hour, why it slows down when the phone heats up, and how to cool it down without losing the conversation. Those explained what goes wrong. This one explains how to see it happening.

The big idea: two kinds of messages

The phone sends two kinds of messages, and they need different handling.

• Alarms, right now. "The phone is getting hot." "Memory is running low." These arrive while the app is running, so the app can react: shorten an answer, free some memory, pause a background job.
• Reports, later. "Yesterday, your app was shut down twice for using too much memory." These arrive after the fact. The app can't change what happened, but it learns why.

You need both. Alarms protect the session in front of you. Reports tell you what keeps going wrong across all your users.

What to watch

The first three come from the phone's own signals, which this article covers. The last three you measure yourself. No phone reports them for you.

iPhone: five ways to listen

1. Heat

In plain words: iOS does not tell an app the temperature. It gives four labels instead: normal, a bit warm, hot, and too hot. Apple's own advice is to cut back as the labels climb. At "a bit warm", put off work that can wait. At "hot", do less. At "too hot", stop what you can.

For engineers: ProcessInfo.thermalState has four values: nominal, fair, serious and critical. Apple's power guide says to register for the change notification instead of polling. Read the current value once at start, because the notification only fires on a change.

final class ThermalWatcher {
    private var token: NSObjectProtocol?

    func start(onChange: @escaping (ProcessInfo.ThermalState) -> Void) {
        onChange(ProcessInfo.processInfo.thermalState) // initial value
        token = NotificationCenter.default.addObserver(
            forName: ProcessInfo.thermalStateDidChangeNotification,
            object: nil, queue: .main
        ) { _ in onChange(ProcessInfo.processInfo.thermalState) }
    }

    deinit { token.map(NotificationCenter.default.removeObserver) }
}

There is no temperature API on iOS. Log the label, not a number you can't read.

2. Running low on memory

In plain words: When the phone is short on memory, iOS first asks apps to give some back. If that doesn't work, it shuts an app down. Apple says these warnings are best-effort, so a sudden big demand can skip straight to the shutdown. A quiet log does not prove a safe session.

For engineers: Apple lists four routes: the app delegate callback, the view controller callback, didReceiveMemoryWarningNotification, and a dispatch source of type DISPATCH_SOURCE_TYPE_MEMORYPRESSURE. For a model, the notification and the dispatch source are handy because you can register them from the inference layer, without a view controller.

NotificationCenter.default.addObserver(
    forName: UIApplication.didReceiveMemoryWarningNotification,
    object: nil, queue: .main
) { _ in
    metrics.memoryWarnings += 1
    cache.dropColdEntries()
}

Pair the warning counter with a footprint reading on each reply. os_proc_available_memory() returns the bytes the process can still allocate. I could confirm it only through search results and a reference on Apple's entitlement page, so check the wording in your SDK. Call it once per reply, never per token.

3. Daily reports from Apple (MetricKit)

In plain words: Apple will mail your app a daily summary from real users' phones: how much memory it used at its peak, how it was shut down, and more. If something serious happens, a separate report arrives right away. The app has to sign up first, and the earlier the better. The reports come once a day, so this is a history book, not a live feed.

For engineers: Adopt MXMetricManagerSubscriber, then add the object to MXMetricManager.shared. This follows Apple's own example.

final class AppMetrics: NSObject, MXMetricManagerSubscriber {
    func start() { MXMetricManager.shared.add(self) }
    func pause() { MXMetricManager.shared.remove(self) }

    // Daily metrics.
    func didReceive(_ payloads: [MXMetricPayload]) {
        payloads.forEach { upload($0.jsonRepresentation()) }
    }

    // Diagnostics, delivered as soon as they are available.
    func didReceive(_ payloads: [MXDiagnosticPayload]) {
        payloads.forEach { upload($0.jsonRepresentation()) }
    }
}

What Apple's documentation says:

• Metric reports arrive at most once per day per metric source, and may come as more than one payload a day.
• A report covers the past 24 hours plus any earlier undelivered ones.
• Diagnostic reports are delivered immediately on iOS 15 and later.
• Adding a subscriber and receiving reports are safe in performance-sensitive code, such as app launch.
• To test, use a physical device.
• pastPayloads and pastDiagnosticPayloads give access to reports you haven't processed.

Register at launch, before the model loads. The useful parts for on-device AI are peak memory, termination counts by category, and the memory-exception diagnostic, which says the system ended the app for exceeding its limit.

4. Your own stopwatch inside Apple's reports

In plain words: You can mark the stages of an AI reply, such as "reading the question" and "writing the answer", and Apple's daily report will include how often each ran and how long it took.

For engineers: Create a log handle through the manager, then wrap each stage in a signpost.

let log = MXMetricManager.makeLogHandle(category: "llm")

mxSignpost(.begin, log: log, name: "prefill")
// ... run prefill ...
mxSignpost(.end, log: log, name: "prefill")

makeLogHandle(category:) is on Apple's MXMetricManager page. I did not confirm the exact mxSignpost signature, so check it in your SDK. The system limits how many custom metrics it keeps, so name only the main stages: model load, prefill, first token, decode. Never put a prompt or a user ID in a name.

Android: five ways to listen

1. Heat, as a label

In plain words: Like iOS, Android reports a heat level. It runs from "not throttled" up to "shutdown". The levels in the middle mean the phone is slowing itself down to cool off.

For engineers: PowerManager.addThermalStatusListener(executor, listener) adds a listener for changes. Callbacks run on the executor you pass, so the main executor puts them on the main thread. Read getCurrentThermalStatus() once at start.

val pm = getSystemService(PowerManager::class.java)
val listener = PowerManager.OnThermalStatusChangedListener { status ->
    state.thermalStatus = status
    log("thermal_change", mapOf("status" to status))
}
pm.addThermalStatusListener(mainExecutor, listener)
// on teardown:
pm.removeThermalStatusListener(listener)

2. Heat, as a number

In plain words: Android also offers a score for how close the phone is to being slowed down. Zero is cool. One means the phone is at the point where it starts throttling hard. You can even ask for a forecast a few seconds ahead. The catch: if you ask too often, it answers "not a number".

For engineers: getThermalHeadroom(forecastSeconds) returns a float where 1.0 means the device is, or will be, throttled at THERMAL_STATUS_SEVERE. Values above 1.0 have no specific mapping to thermal levels. It returns NaN if the device doesn't support it, and the forecast range is 0 to 60 seconds.

On call rate my two sources differ. The PowerManager reference says calling more than about once per second may return NaN. The Android games thermal guide says not to call more than every 10 seconds. I used the slower one, and poll only while the status is elevated.

private var lastHeadroomAt = 0L

fun headroomOrNull(): Float? {
    val now = SystemClock.elapsedRealtime()
    if (now - lastHeadroomAt < 10_000) return null
    lastHeadroomAt = now
    return pm.getThermalHeadroom(0).takeUnless { it.isNaN() }
}

3. Running low on memory

In plain words: Android sends the app a "please trim" message when the phone is short on memory. Only two of those messages matter now: "your screen just went away" and "you're in the background". Older, finer-grained messages stopped being sent in Android 14.

For engineers: The Android memory guide says to act on onTrimMemory and to focus on TRIM_MEMORY_UI_HIDDEN and TRIM_MEMORY_BACKGROUND. The other legacy levels are no longer sent since Android 14 and were deprecated in Android 15.

class App : Application() {
    override fun onTrimMemory(level: Int) {
        super.onTrimMemory(level)
        metrics.trimLevels += level
        if (level >= TRIM_MEMORY_BACKGROUND) model.releaseColdState()
    }
}

The ActivityManager.getMemoryInfo reference says polling is not recommended and onTrimMemory is the preferred route. Take one getMemoryInfo reading per reply for the record, never in a loop.

4. Why did the app die? (read on the next launch)

In plain words: Android keeps a short log of why your app's recent processes ended. When the app starts again, it can read the log and learn: was it shut down for low memory, or something else? There is one trap. Some phones can't report "low memory" and use a generic label instead, so a plain count would undercount. The app should check whether the phone can report it and save that answer with each record.

For engineers: ActivityManager.getHistoricalProcessExitReasons(packageName, pid, maxNum) returns recent deaths, newest first, from a ring buffer, so only recent records survive. null for the package matches your own; 0 for pid and 0 for maxNum means no filter.

fun collectExits(am: ActivityManager, lastSeen: Long) {
    val supported = ActivityManager.isLowMemoryKillReportSupported()
    am.getHistoricalProcessExitReasons(null, 0, 10)
        .filter { it.timestamp > lastSeen }
        .forEach {
            log("exit", mapOf(
                "reason" to it.reason,          // e.g. REASON_LOW_MEMORY
                "importance" to it.importance,
                "rss_kb" to it.rss,
                "lmk_reported" to supported,
                "state" to it.processStateSummary?.decodeToString()
            ))
        }
}

From the reference:

• REASON_LOW_MEMORY means the system's low-memory killer ended the process (API 30 and later).
• Where unsupported, such a kill shows as REASON_SIGNALED.
• REASON_MEMORY_LIMITER is separate: the process exceeded the system memory limits.
• getRss() and getPss() are the last sampled values, not the exact memory before death. getRss() can be zero if the process died before a sample was taken.

Save the timestamp of the newest record you processed, so each launch handles only new ones.

5. A note left for the next launch

In plain words: Before it dies, the app can leave a tiny note, for example "model B, long chat, phone was warm". After the death, the note comes back inside the exit record. It is small, it must contain nothing private, and the app shouldn't write it too often.

For engineers: ActivityManager.setProcessStateSummary(byte[]) stores custom data in the ApplicationExitInfo record. The reference says it is for analysing app health, not restoring UI state, with these limits: the maximum is 128 bytes, no PII or other sensitive data, and the system may throttle calls, so excessive calls can throw a RuntimeException. Write on a state change, not on each reply.

fun note(modelId: Int, ctxBucket: Int, thermal: Int) {
    am.setProcessStateSummary(byteArrayOf(modelId.toByte(), ctxBucket.toByte(), thermal.toByte()))
}

Android has no daily report. Here is what fills the gap.

Apple sends a daily report into the app. Android doesn't have one single equivalent, so you combine four tools.

Android vitals, in the Play Console

In plain words: Google collects quality numbers from users who allow it, and shows them in the Play Console: crash rate, "app not responding" rate, battery wake-ups and memory use. It's the nearest thing to a free report card. It is aggregated across users, not per chat.

For engineers: The core vitals are user-perceived crash rate, user-perceived ANR rate, excessive partial wake locks, memory usage and bitmap memory usage. You can read them in the dashboard or through the Google Play Developer Reporting API. Memory is measured as anonymous RSS plus swap, with limits per RAM tier and app state. For apps on an 8 GB tier (usable memory 6,800 to 9,216 MB) the page lists 2.25 GB foreground and 1.5 GB for background services. On a 4 GB tier (3,200 to 4,800 MB) it lists 2.00 GB and 1.00 GB. The page says apps over the memory, bitmap or code-optimization thresholds may see store visibility affected from February 2027, and it asks you to log ActivityManager.MemoryInfo.totalMem against the device profile.

The page does not say whether memory-mapped model weights count as anonymous RSS. Measure your app's number on a real device and compare it with the tier limit before you assume you're under it.

Perfetto battery counters (lab only)

In plain words: In the lab you can record how much battery the whole phone uses while the model runs. It measures the phone, not just your app, so you compare two runs: one with the model idle, one with it running.

For engineers: Perfetto can read remaining charge (µAh), capacity (%) and instantaneous current (µA) from the battery. It needs power-management hardware, which Perfetto's page says most Pixel phones have, and resolution depends on the manufacturer. The counters cover the whole device. On USB the phone is charging and current reads positive, so use a USB hub that electrically disconnects the phone or run unplugged. The on-device power rails monitor, which measures per hardware block, needs dedicated hardware the page says most production phones lack. The sample config polls the battery every 250 ms.

Trace sections (lab only)

In plain words: The app can drop labels into a recording, like chapter markers: "reading started", "reading finished". Open the recording beside the heat and battery tracks and you can see which chapter got slower when the phone warmed up.

For engineers: androidx.tracing's Trace writes async sections and counters into the system trace.

Trace.beginAsyncSection("llm.prefill", turnIndex)
// ... prefill ...
Trace.endAsyncSection("llm.prefill", turnIndex)
Trace.setCounter("llm.cache_hit_tokens", cacheHit)

The name and cookie must match on begin and end. Overlapping sections need unique cookies. The trace stays on the device, so it doesn't feed a field dashboard.

ProfilingManager, a recording from a real user

In plain words: On newer Android versions, the app can ask the system to record what was happening, and the file is saved to the app's own storage. It's rate limited and not guaranteed.

For engineers: ProfilingManager (Android 15, API 35 and later) supports system traces, Java heap dumps, heap profiles and stack samples. Apps can register triggers so the system captures profiling on their behalf, and results arrive through a listener. Results are redacted and contain only information about the requesting process. I did not check which trigger types fit a "reply was slow" case, so read the trigger list before you design around it.

Gemini Nano is a black box

In plain words: If your app uses Google's built-in Gemini Nano model, the model runs inside a protected system service. That is good for privacy, and it means the app can't see inside. All it can do is time the calls.

For engineers: AICore is isolated, has no direct internet access, and per Google's page "doesn't store any record of the input data or the resulting outputs" after processing. I found no documented way to read its memory or cache use from the app. Log what your code sees: time to first output, time to the last, output length, and any error or unavailable state.

Put it together: one receipt per reply

In plain words: After each AI reply, the app writes a one-line receipt: how long it took, how warm the phone was, how much memory was left. No conversation text, only numbers and labels. Put the receipts from all users in one place and the patterns show up.

For engineers: One record per reply, counts, durations and states only.

{
  "platform": "android", "turn_index": 27,
  "prompt_tokens": 1840, "cache_hit_tokens": 1790, "output_tokens": 212,
  "ttft_ms": 410, "decode_tok_s": 18.4,
  "thermal_state": 2, "thermal_headroom": 0.71,
  "memory_available_mb": 840, "trim_levels_in_turn": 0,
  "foreground": true
}

Keep it small, batch it, and send it on a schedule. The model fields (prompt_tokens, cache_hit_tokens, ttft_ms, decode_tok_s) come from your inference engine and timers. No platform API reports them. TTFT is time to first token: the wait between sending a question and seeing the first word of the answer.

Which message is for what

What to alert on

Set the thresholds from your own lab data. These are the shapes that pay off:

• Shutdowns per thousand sessions, by platform and app version. A jump after a release points at a memory change.
• Time to first token, split by heat label. If it rises only when the phone is warm, the OS is slowing the work. If it rises when cool, look at memory reuse.
• Memory reuse rate, by version. A drop usually means someone changed how the chat history is built.
• Sessions with a memory warning or trim, as a share of all sessions.

Mistakes to avoid

• Signing up late. Register at launch, before the model loads.
• Trusting silence. Memory warnings are best-effort. Add a memory reading to each receipt.
• Asking for the Android heat number in a loop. Use the listener for changes and a throttled read for the number.
• Logging what the user said. Prompts and replies are user data. Receipts hold states and counts.
• Trusting the Android low-memory count blindly. Some phones can't report it. Store the support flag.
• Treating Apple's report as live. It comes once a day. Use your own receipts for anything urgent.

What to do this week

1. Sign up for the heat and memory messages on both platforms, and write the latest values into each reply's receipt.
2. On iOS, subscribe to MetricKit at launch and upload both payload types.
3. On Android, read the exit records at each launch, and store the low-memory support flag.
4. Leave a small state note on Android when the heat level or model changes.
5. On Android, open Play Console vitals and compare your memory number with your RAM tier's limit. Add trace sections around prefill and decode, and record one Perfetto trace with the battery counters.
6. Chart time to first token by heat label. That one chart tells you whether you have a heat problem or a memory-reuse problem.

The phone already knows when it is hot, short on memory, or about to end your app. It only tells the apps that are listening.

#AI #iOS #Android #OnDeviceAI #MobileEngineering

Work with me: I take 1:1 calls on mobile architecture reviews, mobile and AI questions, and startup tech consults. Book a slot on Topmate: topmate.io/sairamkotha

Sources

Apple (fetched)

• MXMetricManager, through Apple's documentation JSON
• MXMetricManagerSubscriber
• Responding to power notifications
• Responding to low-memory warnings
• ProcessInfo.ThermalState

Android (fetched)

• ApplicationExitInfo
• ActivityManager
• PowerManager
• Android vitals
• ProfilingManager
• androidx.tracing Trace
• Gemini Nano
• Perfetto: power data sources
• Thermal API, Android Games ADPF
• Manage your app's memory

What I could not verify

• The exact mxSignpost call signature. I confirmed makeLogHandle(category:) on Apple's page, not the signpost function. Check your SDK.
• os_proc_available_memory() wording. Search results and a reference only.
• The Swift and Kotlin snippets are sketches built from the documented signatures. I did not compile or run them.
• The DISPATCH_SOURCE_TYPE_MEMORYPRESSURE setup. Apple's page names it; I did not include code for it.
• The headroom call rate. The PowerManager reference says about once per second; the games guide says 10 seconds. I used the slower one.
• Whether getRss() and getPss() in exit records are enough to see a gradual memory climb. They are single last-sample values.
• Whether memory-mapped model weights count toward Android vitals' anonymous RSS plus swap. The page does not say.
• Which ProfilingManager trigger types fit a slow-reply case. I read the class summary, not the trigger list.
• Any way to read AICore memory or cache use from an app. I found none documented.
• Perfetto power rails on specific phones. The page says most production phones lack the hardware.
• The Android vitals thresholds and the February 2027 date are as shown on Google's page today. Check the page before you quote them.
• Any numbers on how many users hit each signal. None of this is measured on a live app.

This article follows that failure from the mechanism to the fix. It uses Apple's and Google's own documentation where they publish it, marks where they don't, and gives you a way to read the report the system leaves behind.

Why it takes an hour

Model weights are a fixed cost. You pay them once, at load. What grows is the conversation.

Each token adds entries to the key-value (KV) cache, and the cache keeps growing for the life of the session. A KVSwap paper gives a concrete case: for Qwen3-4B at 16K context and batch size 4, the weights take 7.5 GiB and the KV cache reaches 9 GiB, already larger than the weights. That is a server-class setup, but the shape of the curve is the same on a phone.

A mobile study makes the same point. mzCache (arXiv 2609.01338) reports that a multitasking phone forces the OS to evict model weights and KV cache when the user switches apps, and that restoring them is slow. On a Galaxy S25+ and a OnePlus 12, storage reads account for 59 to 61% of time-to-first-token after eviction.

How iOS decides to stop you

The system sends warnings first. Apps free memory after a low-memory notification. If all apps together release enough memory, nothing happens. If pressure continues, the system terminates applications to reclaim memory. Apple calls that a jetsam event.

Apple says warnings reach your app four ways: applicationDidReceiveMemoryWarning on your app delegate, didReceiveMemoryWarning on active view controllers, the didReceiveMemoryWarningNotification, and a dispatch source of type DISPATCH_SOURCE_TYPE_MEMORYPRESSURE.

Warnings are best-effort. If memory demand rises faster than the warnings relieve pressure, the system may skip them and go straight to jetsam. Apple's rule: allocate large memory slowly, so the system has time to free memory.

Read the report the system leaves behind

When your app is jettisoned, iOS writes a jetsam event report. It contains the memory use of all apps and system processes, it is JSON, and it has no backtraces for your threads. If the system jettisons your app while it is visible, it looks like a crash. The report is the evidence that it wasn't one.

• pageSize in the header converts page counts to bytes. Apple's example uses 16,384 bytes.
• largestProcess names the process using the most memory pages.
• reason appears only on the killed process. per-process-limit means the process crossed the resident memory limit. vm-pageshortage means the system ran out of memory and needed background memory for the foreground app.

To turn pages into bytes, multiply rpages by pageSize. Apple's example: 92,802 pages × 16,384 bytes = 1,520,467,968 bytes, about 1.52 GB.

If your 1-hour crash shows per-process-limit with a rising lifetimeMax, you have a growing allocation in your process. If it shows vm-pageshortage with your app in the background, the phone was short on memory and you were the process it chose.

What you can do

• Unbounded collections. A conversation history or an embedding cache that grows without a limit grows the same way Apple's scrolling message dictionary does. Discard old content, or write it to disk.
• Views and large resources in the background. Release them when the app enters the background, and rebuild on return.
• Leaks. Use Instruments' Leaks template. A retain cycle around a model session is a common cause of a process that never gets smaller.
• Image size. Transform images with Image I/O before they enter the model.

For the warnings, Apple gives three points. Don't search your whole object graph for memory to release, because iOS compresses pages the app hasn't touched recently. If your data is easy to recreate, use NSPurgeableData. Apple advises against combining NSCache with NSPurgeableData.

For context, Apple's WWDC26 session says developers should query the model rather than assume a size. It introduced APIs in iOS 26.4 for reading contextSize and counting tokens. The sample prints 8192. Other coverage gave 4K, so measure on the device you run on. The session does not say what happens when a session exceeds the context limit.

Android: the killer goes after the background first

Android's low-memory killer terminates processes by oom_adj_score. Cached processes and processes with a service and no UI go first. Focus on onTrimMemory levels TRIM_MEMORY_UI_HIDDEN and TRIM_MEMORY_BACKGROUND. Since Android 14 the system no longer sends the other legacy levels.

Google's Gemini Nano page says AICore manages the model and that it doesn't affect your app's disk and runtime memory budget. It does not say where a long session's KV cache lives. Measure that on a device.

A plan for the next release

1. Reproduce it. Run a scripted conversation for an hour on the oldest device you support. Log memory and context length each turn.
2. Read the jetsam report after each crash. Record reason, lifetimeMax and largestProcess.
3. Bound the state. Set a context budget from the measured curve, and summarize or drop older turns past it.
4. Respond to warnings. Drop caches and offload cold context. Release the model when backgrounded, and measure the reload cost.
5. Consider offload. mzCache and KVSwap show that moving KV entries out of RAM can work on mobile-class hardware, with a latency cost.
6. Test again. Keep the one-hour soak test in CI for release candidates.

The crash at minute sixty is almost never one bad allocation. It is state that grows with the conversation, meeting a limit the operating system sets, and the operating system writes you a file that says why.

#AI #iOS #Android #OnDeviceAI #MobileEngineering

Work with me: I take 1:1 calls on mobile architecture reviews, mobile and AI questions, and startup tech consults. Book a slot on Topmate: topmate.io/sairamkotha

Sources

• Identifying high-memory use with jetsam event reports (Apple)
• Responding to low-memory warnings (Apple)
• Making changes to reduce memory use (Apple)
• WWDC26: What's new in the Foundation Models framework (Apple)
• Manage your app's memory (Android)
• Gemini Nano on Android (Android)
• mzCache: On-Device LLM Memory Management under Multitasking
• KVSwap: Disk-aware KV Cache Offloading for Long-Context On-device Inference

What I could not verify

• A fixed per-app memory limit per iPhone model. Single developer reports cite about 6 GB on App Store builds and about 15 GB on development builds.
• Whether the Foundation Models KV cache counts against the app's process.
• What happens when a Foundation Models session exceeds its context window.
• Where Gemini Nano's per-session state lives.
• The mzCache results on other devices. The paper tests two phones and two small models.
• The KVSwap numbers on a phone. They come from a Jetson Orin AGX.
• The one-hour timing in general. I found no source that measures it on phones.

This article follows one notification from the server to the screen, marks where an on-device model can sit in that path, and lists the limits that decide whether the summary is ready when you look.

The path a notification takes

1. Your server sends a push. The payload goes to Apple Push Notification service (APNs) or Firebase Cloud Messaging (FCM). Apple's documentation caps a standard remote notification at 4 KB (4,096 bytes) for the whole JSON payload. Title, body, custom data and deep links all count. Non-Latin text takes more bytes per character, so the same message can fit in English and fail in Hindi.
2. The push service delivers it. Delivery is not instant or guaranteed. The device may be asleep, in low-power mode, or offline. The notification waits in the service until the phone can receive it.
3. The phone may run your code first. On iOS, a Notification Service Extension runs only when the payload has an alert and sets mutable-content to 1. Silent pushes and badge-only pushes skip it. The extension has a short time budget. Apple's sample code handles serviceExtensionTimeWillExpire(), which is the last chance to return content before the system stops the extension. I could not confirm the exact seconds from Apple's pages, so treat any specific number as unverified until you test it.
4. The system shows the notification. If there is no extension, or the extension runs out of time, the original content is shown.

Only the first three steps are about your app. The summary arrives at step 3 or 4, and only if the phone decides to make one.

Where the model sits

Apple Intelligence notification summaries need an iPhone 16 family model, iPhone 15 Pro, or iPhone 15 Pro Max on iOS 18.1 or later, with Apple Intelligence turned on, according to Apple's support page. Press coverage says the summaries work offline, which implies on-device processing. The Apple page I could reach did not show that sentence, so I am not quoting Apple on it.

I also could not confirm which step runs the summary: the extension, the system after delivery, or a separate pass. Apple's public docs for third-party developers do not say that a third-party extension can call the summarizer. If you are building an app that needs its own on-device model in this path, assume you cannot rely on Apple's summaries and run your own model under your own budget.

Press coverage in January 2025 reported that summaries were paused for News and Entertainment apps after accuracy complaints. The behavior may have changed since, so check the current iOS release notes before you design around it.

The budget that matters

For a model call inside a notification, three budgets compete:

• Payload size. 4 KB for the whole push, before the device does anything. A summary cannot be sent in the push if the source text is already near the cap. Send an ID and fetch the body in the extension if you need more text, and budget the fetch against the time limit.
• Time. The extension is short-lived. A slow network call followed by a model call can use up the budget before the summary is ready. The safe pattern is to return the original text first and update only if the model finishes in time.
• Context window. The on-device model has a fixed token window. My earlier article on Apple's 4,096-token window shows how instructions, tools and output all spend the same room. A notification body is small, but a thread history or a long bundle of messages is not.

What to do with this

• Keep the push payload small, and put the full content behind an ID.
• Return the best content you have before the time limit. Do not wait for a model to finish.
• Test on a real device with Apple Intelligence on and off, and on a low-power setting. Simulator timing is not the same.
• Do not assume a summary exists. The user may have it turned off, the device may not qualify, or the app may be excluded.

The notification looks like the simplest part of a phone. It is a small, time-limited pipeline with a model at one optional step, and the limits in that pipeline decide what you read first.

Work with me: I take 1:1 calls on mobile architecture reviews, mobile and AI questions, and startup tech consults. Book a slot on Topmate: topmate.io/sairamkotha

What I could not verify

• The exact time budget for a Notification Service Extension. Third-party guides say 30 seconds. I could not fetch Apple's page with the number.
• Which step in the pipeline runs the summary, and whether a third-party extension can call it.
• The exact Apple sentence on on-device processing for notification summaries.
• Current status of the News and Entertainment exclusion after iOS 18.3.

#AI #iOS #Android #OnDeviceAI #MobileEngineering

Two different sign-ins

"Sign in with Google" and a passkey are not the same thing, and most people mix them up.

• Sign in with Google is a redirect. The app sends you to Google, Google checks who you are, and Google gives the app a signed ID token. The token is a signed JWT whose sub claim is a stable ID for you at Google, scoped to that app. It says who signed in only as that opaque ID. Name and email appear only if the app asked for them through scopes. The app never sees your Google password. Your Google account may itself use a passkey, but the basic ID token doesn't say which sign-in method you used.
• A passkey is a key pair made for one site. The private key stays on your device. The site stores only the public key. To sign in, the site sends a random challenge, your phone signs it after you unlock it, and the site checks the signature with the public key.

What the device stores

For each passkey, the device holds a private key. The app never gets a copy. The key is meant to sit in hardware-backed storage, but I could not confirm which hardware Apple or Android uses for passkey keys. See the last section.

The signature is bound to the site's domain. A fake site at a look-alike domain gets a signature that doesn't match, so the sign-in fails. That is why passkeys resist phishing in a way passwords don't.

How it reaches your other devices

A passkey on one phone is useless on the next unless it syncs. Synced passkeys are stored by:

• iCloud Keychain on Apple devices
• Google Password Manager on Android and Chrome
• third-party managers such as 1Password or Bitwarden

Apple's and Google's stores don't copy into each other. A passkey saved in iCloud Keychain won't appear in Google Password Manager, and the reverse holds too. Cross-ecosystem sign-in still works in one case: a site or a new computer can show a QR code, and your phone's passkey signs the challenge over Bluetooth. The key is never copied. Hardware security keys stay bound to the physical key and don't sync.

Why sign-in fails after a phone change

Usually it's one of these:

1. The passkey was never synced. It lived only on the old phone.
2. The new phone is in a different ecosystem, so the iCloud and Google stores don't share.
3. Cloud sync was off on the old phone, or the account was signed out.
4. The site only registered a passkey on the old device.

What to do before you switch phones

• Check that passkeys are synced. On iPhone: Settings → Passwords. On Android: Google Password Manager.
• Sign in to important accounts on the new phone before you hand back the old one.
• Keep a second sign-in method, such as an authenticator app or recovery codes, stored outside the phone.
• After a move, register a new passkey on the new device if the site offers the option.

Passkeys are safer than passwords, but they are tied to where the key lives. Knowing where that is decides whether a new phone is a five-minute setup or a locked account.

What I could not verify

Confirmed against primary sources (checked 2026-10-22):

• iCloud Keychain sync. Apple's Platform Security guide ("iCloud Keychain security overview") says passkeys sync across iPad, iPhone, Mac, Apple Watch, and Apple Vision Pro, and that they are "encrypted end-to-end so that Apple and other devices can't read their contents."
• Apple new-device join. Apple's "Use passkeys in iPhone and iPad" support page says passkeys sync through iCloud Keychain and are stored in the Passwords app. A new device joins the syncing circle by pairing with an existing iCloud Keychain device, or through keychain recovery. Recovery needs the Apple Account password and a verification code.
• Google Password Manager PIN. Google's Chrome help page (support.google.com/chrome/answer/16608973) says a Google Password Manager PIN is created when you first create a passkey, and that it helps you "use your passkeys on a new device." Its stated purpose includes: "Make sure that no one, not even Google, can access your encrypted data." The page does not say "end-to-end."
• Google sync scope. Passkey sync across Windows, macOS, Linux, and Android is from 2024 press coverage (The Hacker News, BleepingComputer). I did not retrieve Google's own page for it.
• OIDC ID token (OpenID Connect Core 1.0, section 2). iss, sub, aud, exp, and iat are REQUIRED. aud MUST contain the client's client_id. The ID token MUST be a JWS-signed JWT. nonce must be verified when the request sent one. The spec defines no password claim. Name and email come from scope-based standard claims in section 5.1, which I did not read.
• Passkey challenge-response. The site-stores-public-key, phone-signs-challenge model matches the WebAuthn flow described above. I did not fetch the WebAuthn spec itself.

Still unverified:

• Secure Enclave for passkey keys. Apple's Secure Enclave page says key material is protected in hardware. It does not say passkey private keys live there. The passkey pages I read don't say it either. Treat the Secure Enclave claim as unconfirmed.
• Android secure element or TPM. Not checked.
• Hardware security keys don't sync. Not checked against a FIDO source.
• Settings paths. Apple's support page gives no menu path. The "Settings → Passwords" path is from third-party guides. The Google Chrome path (More → Passwords and autofill → Google Password Manager) is from Google's help page.
• Google PIN as the only new-device factor. Google's help page names the PIN. Android screen lock as an alternative comes from 2024 press coverage. Google's Security Blog (2022) was not readable, and Google's "Passkeys" support page returned 404.
• Passkeys in Google Password Manager on iPhone. Press coverage (heise) says iOS support is there. I did not confirm it on Google's own page.
• New-device phishing risk. How each platform's new-device step can be phished has not been tested on current phones.

#Passkeys #Security #Android #iOS

Work with me: I take 1:1 calls on mobile architecture reviews, mobile and AI questions, and startup tech consults. Book a slot on Topmate: topmate.io/sairamkotha

This article covers what Google's documentation says, what changed at I/O 2026, and how to budget a prompt before you call it.

How to read it: the Prompt API limits come from Google's page "Get started with Prompt API" (developers.google.com/ml-kit/genai/prompt/android/get-started), read on October 19, 2026. The I/O 2026 changes come from Google's post "Top AI on Android updates for building intelligent experiences" (May 26, 2026). The budget example uses sizes I assumed, and the code is a sketch I have not compiled or run on a device.

The numbers Google gives

• Input. "Input must be under 4000 tokens (or approximately 3000 English words)."
• Output. The page gives no hard number. It says: "Use cases that require long output (more than 4K tokens) should be avoided."
• Output cap you set. maxOutputTokens "defines the maximum number of tokens that can be generated in the response." The page gives no default or range.
• Counting. The page points to a countTokens reference for measuring input.

The page does not say what a token is for this model, or what happens when input reaches the limit. Measure your own strings instead of relying on a characters-per-token rule.

What the device needs

• Android API level 26 or higher.
• Not an unlocked bootloader. The page says the API "is not supported on devices with an unlocked bootloader."
• AICore. Gemini Nano runs in Android's AICore system service, which also manages downloads and updates. The Prompt API page lists model states that your app must handle: AVAILABLE, DOWNLOADABLE, DOWNLOADING and UNAVAILABLE. A model that is not ready yet is a normal state.
• Per-app quota. "AICore enforces an inference quota per app." The page gives no numbers, so you cannot plan for a specific request count.

What changed at I/O 2026

Google's I/O post adds three things, and each has a status you should check before you build on it:

• Gemini Nano 4 is in developer preview. Google says developers "can already preview and prototype" with Nano 4 through the AICore developer preview. It is "launching in flagship devices later this year." The post names no devices and no date.
• Prefix caching. The post says it "reduces inference time by storing and reusing the intermediate LLM state of processing a shared and recurring part of the prompt." The Prompt API docs list prefix caching as Experimental. It helps most when many requests share a long system prompt.
• Structured Output. Google calls it "upcoming," with no date. The Prompt API docs label "Generate structured output" as Alpha.

The same post names Firebase AI Logic's hybrid inference as a way to route between on-device and cloud models. Its modes include PREFER_ON_DEVICE and ONLY_ON_DEVICE. The post does not say whether the modes are stable.

The docs also mark "Prompt," "Set system instructions" and "Use thinking mode" as Beta. Build your launch plan around the parts that are stable, and check the labels before each release.

A budget for a prompt

These sizes are my assumptions, chosen to show the shape, not measurements:

• System instructions: 300 tokens.
• A user message with a pasted article: 2,400 tokens.
• Room for the reply: 800 tokens, set with maxOutputTokens.

That leaves about 500 tokens of margin under the 4,000-token input limit. A second turn with a long history can exceed it. The planning rule is simple: count the prompt before the call. If it does not fit, shorten the input, summarize older turns, or split the task.

// Sketch, not compiled. Check the current ML Kit Prompt API reference for exact names.
val inputTokens = /* count with the countTokens reference */
if (inputTokens >= 4000) {
    // trim the oldest turns, summarize the document, or split the task
}

A fallback when the model is not there

A feature that depends on the model must still work when the model is unavailable, unsupported or still downloading. Google's options are:

1. Show a plain state. Tell the user the feature is preparing, and keep the rest of the app usable.
2. Route to cloud. Firebase AI Logic's hybrid inference can send requests to a cloud Gemini model when Nano is unavailable. Check the current mode names and status in its docs before you ship.
3. Turn the feature off. For a feature that needs on-device data, that may be the right answer.

Pick one before launch, not after the first crash report.

What I would do

This section is my opinion, not Google's.

1. Count before you call. Do not learn the input was too long from an error.
2. Keep history short. Summarize older turns rather than sending the whole conversation.
3. Set maxOutputTokens. Keep replies inside the output budget the page recommends.
4. Handle the model states. A DOWNLOADABLE or DOWNLOADING model is normal, not a crash.
5. Measure prefix caching yourself. It is Experimental, and Google gives no number for the gain.
6. Test on a range of devices. Quota, availability and warmup vary, and Google gives no figures for them.

What I could not verify

• Exact token counts. I did not run countTokens on real strings, so I cannot say how many tokens a given paragraph is.
• Output behavior past the limit. The page advises against long output but gives no hard number and no error behavior.
• Quota numbers. The page says there is a per-app quota and gives no figures.
• Nano 4 devices. Two sources I checked list different phones, and neither list is on a Google page I could read. I have not named any device.
• Prefix caching gains and Structured Output dates. Google gives no numbers or dates.

Sources, accessed October 19, 2026: Google for Developers, "Get started with Prompt API" (ML Kit GenAI), developers.google.com/ml-kit/genai/prompt/android/get-started; Google for Developers Android Blog, "Top AI on Android updates for building intelligent experiences" (May 26, 2026), developer.android.com/blog/posts/top-ai-on-android-updates-for-building-intelligent-experiences-from-google-i-o-26; Android Developers, "Gemini Nano" AICore overview (last updated October 8, 2026), developer.android.com/ml/aicore.

If your Android feature needs a 10-page document in one prompt, what does it do on a device where Gemini Nano is still downloading?

Work with me: I take 1:1 calls on mobile architecture reviews, mobile and AI questions, and startup tech consults. Book a slot on Topmate: topmate.io/sairamkotha

Their answer is a stack of checks, with a person accountable for every change that ships. This article walks through the layers, the numbers they published, and what they still have not solved.

How to read it: the post is "How the Claude Code team does code review," published October 16, 2026, at claude.dev/blog/solving-code-review-at-anthropic, read on October 18, 2026. Every number below is the team's own, from internal measurements, and I have not checked them independently. I have not used their system. Where I add an opinion, I label it.

The problem they named

Merged pull requests more than doubled from June to September, and the share needing a revert fell by more than 40 percent. The stated goal is to minimize human review time without increasing reverts or incidents. Those two numbers together are the real claim: faster merging with fewer rollbacks, not just faster merging.

The reframing matters more than any tool. "Did a human read the code?" is a poor proxy for safety. The team moved review from a gate at the end of a process to something that runs continuously, which they describe as "more like spellcheck while you write."

The layers

1. Verify before the pull request. Claude runs tests, builds and linters, and shows the output rather than claiming success. A subagent reviews the diff in a fresh context. Recurring findings are fixed at the source, in CLAUDE.md, a REVIEW.md written by code owners, skills, or lint rules.
2. An AI reviewer on every pull request. Parallel agents hunt for bugs. A separate group votes on each finding, and only the findings that survive are posted, labeled Important, Nit, or Pre-existing. The team reports that internal pull requests getting substantive comments rose from 16 percent to 54 percent when the tool launched.
3. A security review that can block. Open bug or security findings stop a merge until they are fixed or closed with a reason.
4. Required continuous integration. Lint, types, tests across platforms, and builds. The pull request template asks how the change was tested and how to revert it.
5. Human approval scaled to risk. A classifier sets the tier from the change, not the author. Low risk needs no human once automated checks pass. Medium needs one person, and high needs two. Some paths, such as CI workflow files, always need a person.
6. A merge queue, flags and gradual release. The queue re-runs checks against main. New features reach internal users first, and releases go to a latest channel before stable.
7. Fast detection and rollback. Flags turn a feature off without a deploy, and a weekly on-call rotation watches for problems.

Two rules in the list are easy to skim past. An agent's approval never counts as a human approval. And an approval covers one specific commit, so a later change needs a new review.

The reviewer needed its own fixing

The team found that the AI reviewer could fail to converge. One command-line pull request went through more than fifty review rounds overnight. About 59 percent of what the reviewer kept flagging was code that had been there since its first pass.

Their fixes were practical: cap the review at three rounds, re-review only the diff after the first few passes, flag bug classes rather than single lines, and never repeat a finding. Their numbers before and after:

They also turn past incidents into a test set: bug-causing pull requests, their reverts and their fixes become cases the reviewer runs daily.

Where humans stay

The post lists what stays with people:

• Whether to build something at all: direction, scope and taste.
• Risky code, such as billing, the auto-updater and CI workflows.
• Architecture that others will copy.
• Reading evidence, such as test output, screenshots and findings.
• Owning outcomes: "A person is accountable for every change that ships, whoever typed it."

What is still unsolved

The post is direct about the gaps, and I think this is the most useful part for other teams.

• No circuit breakers yet. The team has no mechanism that tightens approval when reverts or incidents rise. Engineers want one before more code moves to zero-human merges.
• Bad merges are still caught by people. Detection depends on the author and on-call, as well as dashboards.
• Slowing down is not the fix. The post says slowing down lowers error rates, "mostly because you're doing less of everything." They prefer getting faster at detection and repair.
• Reviewer loops have no limit. Agent-author and agent-reviewer loops can run indefinitely without a convergence cap.

What I would take from it

This section is my opinion, not the team's.

1. Replace "who read it?" with "which check catches this risk?" Then write down the check for each risk class.
2. Scale approval to risk. Most changes do not need two people. A few paths always do.
3. Cap every automated review loop. Three rounds is a number they chose; the principle is a hard limit.
4. Turn incidents into tests. Every revert should leave behind a case the reviewer runs.
5. Build the circuit breaker before you need it. The team says it doesn't have one yet.

What I could not verify

• Every number. The 90 percent, the 16-to-54 percent, and the convergence table are internal figures, and I cannot check them.
• Transfer to other teams. Claude Code is one product with one codebase, and the team's tooling is its own.
• Current state. The post describes one moment in October 2026, and the tooling is changing.

Source, accessed October 18, 2026: "How the Claude Code team does code review," claude.dev/blog/solving-code-review-at-anthropic (October 16, 2026).

If your team's reviewers approved the last ten changes in under a minute each, which one would you have checked yourself?

#AI #ClaudeCode #CodeReview #SoftwareEngineering

Work with me: I take 1:1 calls on mobile architecture reviews, mobile and AI questions, and startup tech consults. Book a slot on Topmate: topmate.io/sairamkotha

The answer is not to write it faster. Anthropic's engineering team describes the shift from the inside: when code changes arrive at 30,000 a day, the engineer's job moves from reading every line to judging what a change is for, and to telling the agent "no" when it should not proceed. This article takes that seriously and asks what it means for the people who review and own a codebase.

How to read it: the Medium essay is by Scott Batson, "What Should Programmers Do While the AI Writes Code?", read on October 17, 2026. The Anthropic description comes from its engineering post "Redesigning source control at Anthropic" (October 13, 2026), read on October 14. I have not used Anthropic's internal system and have no inside knowledge. The claims about review practice are the post's own, and I label my opinions.

The new problem is volume, not writing

Scott's essay names the trap precisely. AI made each task faster, so he took on more tasks, and "I was working on everything, everywhere, all at once." His productivity rose and his control fell. The code was not the bottleneck anymore. Review and judgment were.

Anthropic's post makes the same point at a larger scale. It says merges went from around 2,500 a day in May 2026 to more than 30,000 a day in September, and that source control became the biggest bottleneck. It also says that Anthropic could not replicate the industry-standard global lock on the merge queue at that volume, so it invested in automated testing after a merge and before a deploy instead.

Reviewers now judge intent

The post describes what reviewers do when they cannot read every line. Reviewers "examine the intent of a change instead of reading every line," and the review tool can express complex rules about who must approve what.

Read the wording carefully. It does not say reviewers stopped caring about the code. It says the question moved from "is every line right?" to "is this change doing what it claims, and does the test suite say so?"

The engineer's job became saying no

The post's most direct sentence is about the engineer's role: the engineers' job was to tell Claude "no." It gives two examples: several systems that tried to brute-force when a change had merged to main, and a review that was too defensive about trusting another internal service.

That is a job description for a senior engineer. Anyone can accept an agent's output. The valuable skill is refusing the wrong approach, and knowing which rule the agent should not break.

What this means for a team that is not Anthropic

This section is my opinion, not Anthropic's.

1. Write down what "done" means. An agent cannot judge intent it was never given. A written success criterion, such as a failing test that must pass, is the reviewer's first tool.
2. Review intent, then spot-check. If a change cannot be explained in two sentences, send it back before reading the diff.
3. Protect one human veto. The post's examples are about a human saying no to a bad approach. Keep that role explicit in your process, not implied.
4. Measure the queue, not the output. Scott's warning was about stacking work. Track how many changes are waiting on a human, not just how many the agent produced.

What I could not verify

• Anthropic's internal review practice. I read one post. I cannot say how reviewers work day to day, and the post describes intent-based review at a company with unusual volume.
• Scott's productivity figure. "Five times as productive" is his own estimate, not a measurement I checked.
• Generalizing to your team. The 30,000-merges figure comes from a company with a very large monorepo. A small team has different constraints.

Sources, accessed October 14 and 17, 2026: Anthropic Engineering, "Redesigning source control at Anthropic" (anthropic.com/engineering/redesigning-source-control); Scott Batson, "What Should Programmers Do While the AI Writes Code?" (Medium, August 31, 2026).

A review I did this year

On a trading app's real-time connection layer, an AI agent had made a change to how a live data socket is kept alive. I did not stop at the diff. I asked what the change does when the app goes to the background and comes back.

That question led to a regression in the foreground path. When the app returned to the foreground, a keep-alive setting re-enabled the socket in the middle of a reconnect, so the socket could reopen while the reconnect was still settling. The fix was a rule: a pending disable must win over a keep-alive that arrives during the same reconnect. It came with tests for that exact order of events.

Then an external review posted 16 findings on the pull request. The temptation was to accept the list. Instead I checked the open items against the current code. Six of seven were valid, and one was overstated. I also declined two findings outright. Only the valid ones went into the change.

The engineer's job in that review was not to write the fix. It was to decide what the change was supposed to do, and to refuse the findings that did not hold.

If your team merged ten times more code next quarter, which review would you stop doing first, and which would you never give up?

#AI #SoftwareEngineering #CodeReview #Engineering

Work with me: I take 1:1 calls on mobile architecture reviews, mobile and AI questions, and startup tech consults. Book a slot on Topmate: topmate.io/sairamkotha

This article covers what changed, what breaks, and what I would change in a CI pipeline. I have not run any of this on Xcode 27 myself.

How to read it: the path change, the failing operations and the workarounds come from GitHub issues and a pull request in open-source simulator tools, read on October 16, 2026. I did not watch the video this article is based on, and I could not open its transcript, so I have not used it as a source. Where the sources disagreed, I say so.

What changed

Two things moved.

• The app. Simulator.app is gone. Device Hub replaced it. One reporter found Contents/Applications/DeviceHub.app present and Contents/Applications/Simulator.app absent.
• The framework. Tools load SimulatorKit.framework to talk to simulators. In Xcode 26 it lived under the developer directory. In Xcode 27 it moved to a sibling folder:

Xcode 26:  Xcode.app/Contents/Developer/Library/PrivateFrameworks/SimulatorKit.framework
Xcode 27:  Xcode.app/Contents/SharedFrameworks/SimulatorKit.framework

A pull request that patched one tool describes the new location as "a sibling of Developer, not a relocation within it." That detail matters for any path logic that assumes everything sits under Developer.

What breaks

The reports agree on this, with one difference noted below.

• Attaching to a simulator fails with a framework-load error when the tool looks in the old location: Attempting to load a file at path '/Applications/Xcode 27.app/Contents/Developer/Library/PrivateFrameworks/SimulatorKit.framework', but it does not exist.
• Input fails. The same pull request says every simulator HID call (tap, type, swipe, press) fails when the framework cannot be loaded.
• Silent taps. One reporter saw tap and button return success while the screen did not change.
• Opening Simulator. open -a Simulator fails, because the app is gone.
• Builds. If you point xcode-select back at an older Xcode to get the old paths, builds that need the Xcode 27 SDK fail.

The reports disagree on screenshots. The pull request says screenshots and app install are unaffected. Another report says video and screenshots gave a black screen and a crash after the path was patched. I would test screenshots on your own runner before trusting either account.

Workarounds that people used

• Pin xcode-select to an older Xcode. This fixes attach and input, but breaks builds that need the Xcode 27 SDK. Good for a short fallback, not as a fix.
• Symlink the framework into the old layout. This works locally, but it writes into the Xcode bundle and needs sandbox changes in at least one tool. I would not put it in CI.
• Use a physical device. One reporter drove an iPhone over iPhone Mirroring instead.

The pull request's approach is the one I would copy: resolve the framework from both layouts at runtime, and return the path it actually loaded, so the path logic cannot drift from the file that gets used.

What this means for a CI pipeline

This section is my opinion, not the sources'.

Our repository's workflows run on Linux. The Android Maestro job says iOS simulator coverage "needs a macOS runner and is a deliberate follow-up." I did not find a macOS runner in any workflow. The iOS Detox configuration is set up for an iPhone 16 simulator, so iOS simulator tests are most likely run locally or on a separate Mac. If your failing CI runs are on a Mac runner, the fixes below apply directly. If not, tell me where they run.

1. Pin the Xcode version explicitly on macOS runners. Do not take whatever Xcode the image defaults to. Log xcodebuild -version at the top of every job.
2. Resolve SimulatorKit from both layouts, and fail fast. A missing framework should stop the job with a clear message, not time out a test run.
3. Do not use symlinks or xcode-select swaps in CI. They pass one machine and fail on the next image update.
4. Separate the failures. A test that fails on a tap is an infrastructure failure until proven otherwise. Make the simulator check a step of its own, so a broken path does not look like a broken screen.

A resolver in shell, as an illustration and not compiled:

XCODE_CONTENTS="$(dirname "$(xcode-select -p)")"   # .../Xcode.app/Contents
SIMKIT=""
for p in "$XCODE_CONTENTS/Developer/Library/PrivateFrameworks/SimulatorKit.framework" \
         "$XCODE_CONTENTS/SharedFrameworks/SimulatorKit.framework"; do
  [ -d "$p" ] && SIMKIT="$p" && break
done
[ -n "$SIMKIT" ] || { echo "SimulatorKit not found under $XCODE_CONTENTS"; exit 1; }
echo "SimulatorKit: $SIMKIT"

What I could not verify

• The video's content. I used its title only.
• Detox's behavior on Xcode 27. I did not check whether Detox or its simulator helper uses SimulatorKit, so I cannot say whether our iOS suite is affected.
• Exact paths on later builds. The reports do not say which Xcode 27 builds they were run on, and I found no Apple documentation for Device Hub.
• Whether Apple intends SimulatorKit to stay private. Tools depend on it, so expect it to move again.

Sources, accessed October 16, 2026: quern-dev/quern pull request #176 (SimulatorKit relocation); anthropics/claude-code issue #79991 (Xcode 27 Device Hub); stablyai/orca issue #17587; software-mansion/argent issue #406.

If your simulator tests went green last week and red this week, which step did they fail on: attach, tap, or the screenshot?

#iOS #Xcode #CI #SoftwareEngineering

Work with me: I take 1:1 calls on mobile architecture reviews, mobile and AI questions, and startup tech consults. Book a slot on Topmate: topmate.io/sairamkotha

This follows up my last article on the on-device window. It covers what Apple said, why the numbers may differ, and how to budget tokens when one API can now talk to two models.

How to read it: every claim comes from Apple's WWDC26 sessions "What's new in the Foundation Models framework" (241) and "Build with the new Apple Foundation Model on Private Cloud Compute" (319), and from Apple's technote TN3193, all read on October 16, 2026. The code is a sketch I have not compiled, and I ran nothing on a device. Where Apple was silent, I say so.

The three numbers

• 4K. In session 319, comparing the two models, Apple says "The on-device model offers 4k, and with PCC you get 32K." My previous article's 4,096 came from TN3193.
• 8192. In session 241, Apple's sample code for the new context-size API is print(model.contextSize) // 8192. That is a comment in a sample, not a stated spec.
• 32,000. The Private Cloud Compute (PCC) model "has a 32,000 token context window".

I can see two readings of 4K versus 8192, and Apple does not say which is right. One is that the new on-device model has a bigger window than the one in TN3193. The other is that the sample simply shows a device where the value is 8192. Session 241 hints at the second: it says to use the context-size and token-counting APIs, available since iOS 26.4, to "adapt your app to the hardware it's running on."

What changed in the model

Per session 241, the on-device model was "rebuilt from the ground up", with better logic and tool calling, and it gains vision. You attach an image to a prompt next to your text. Apple warns that "larger images will consume more tokens and incur more latency," so images spend the same window your text does.

What the server model adds

The PCC model is a much bigger model with the same session API. Per Apple:

• One line to switch. Apple says changing one line of code switches a session to the server model. Generable output and tools work the same.
• Reasoning. It offers three levels: light, moderate and deep. Reasoning is extra text the model generates, and Apple says it "counts towards your context size limit." Deep reasoning text may be longer than the answer itself.
• Cost and limits. There are no token costs to the developer. Each user gets a daily limit, higher with iCloud+. Apple says it is available to apps with fewer than 2 million downloads and needs an entitlement. Session 319 words this as "less than 2M downloads" and session 241 as fewer than 2 million first-time downloads, so check the current terms.
• Trade-offs. On-device works offline and has no request limits. PCC needs a connection and has the daily limit. Both need an Apple Intelligence device.

What this means for budgeting

My previous article's advice still holds: do not hardcode the number. Apple added the tools to follow it:

• contextSize works on both SystemLanguageModel and PrivateCloudComputeLanguageModel.
• tokenCount(for:) counts instructions, prompts and transcripts.
• A usage property on responses reports input, output and reasoning token counts.
• For PCC limits, session 319 says to check isLimitReached on the model's quotaUsage, and to show persistent UI instead of an alert.

A pattern I would use

This section is my opinion, not Apple's.

1. Measure before you choose. Count the tokens of the real prompt with tokenCount(for:) and compare against contextSize.
2. Leave room for the answer. For PCC, that includes reasoning tokens.
3. Route by data, not vibes. Apple's own phrase, from session 319, is to decide between models "based on data, not just vibes" with the new Evaluations framework.
4. Plan for the limit. A PCC daily limit is a different failure from a full context window. Decide what the user sees in each.

A sketch, not compiled:

let model = SystemLanguageModel()
let window = model.contextSize
let promptTokens = try await model.tokenCount(for: prompt)
// leave room for the reply; if it does not fit,
// trim, split the task, or route to the server model

What I could not verify

I did not run any of this. The 8192 is a code comment in a sample, and I did not find a table of per-device windows. Apple does not give numbers for the PCC daily limit. The "4K" in session 319 is a spoken comparison, not a documented spec. Third-party posts say more about these APIs, and I did not use them. These are developer-beta APIs, and details can change.

Sources, accessed October 16, 2026: Apple Developer, WWDC26 session 241 "What's new in the Foundation Models framework" and session 319 "Build with the new Apple Foundation Model on Private Cloud Compute"; Technote TN3193, "Managing the on-device foundation model's context window" (developer.apple.com).

If Apple says to adapt to the hardware, what does your app do when contextSize returns a number you never tested?

#iOS #AI #OnDeviceAI

Work with me: I take 1:1 calls on mobile architecture reviews, mobile and AI questions, and startup tech consults. Book a slot on Topmate: topmate.io/sairamkotha

This article follows Apple's own documentation: what the window is, what spends it, a worked budget, what Apple recommends, and what to do when you hit it.

How to read it: every claim about the Foundation Models framework comes from Apple's documentation, read on October 15, 2026, mainly technote TN3193 and the API pages for SystemLanguageModel. The budget example uses sizes I assumed, and the code is a sketch I have not compiled. Where Apple's pages were silent, I say so.

The number

TN3193 states it plainly: Apple's on-device foundation model has a context window of 4096 tokens per language model session. Apple explains the reason too: an on-device model running on an iPhone needs a relatively small window to run efficiently.

Two details matter:

• It is per session, not per message. The contextSize documentation says it is the total number of tokens that can be used in a single session, including both input prompts and generated responses.
• Tokens are small. Apple says a token is roughly three to four characters in Latin-alphabet languages like English, and roughly one character per token in Chinese, Japanese and Korean.

My arithmetic: 4,096 tokens at 3 to 4 characters is about 12,000 to 16,000 English characters. That is a few pages of text for the whole conversation, instructions and replies included. For Chinese, Japanese or Korean it is closer to 4,000 characters.

What spends the window

The technote lists everything that contributes tokens to the current session:

• Your instructions.
• Every prompt you send.
• Tool information: the schemas, the arguments the model passes, and the tool's output.
• Your Generable schemas.
• All of the model's responses.

Two of these are easy to miss. For every Generable type, the framework converts its type and format information into a JSON schema and passes that text to the model, and any @Guide descriptions you write also cost tokens. For tools, the name, description and parameters go into the prompt, and each tool call's output comes back in as more tokens.

The consequence is that a session is a running transcript. Each turn adds your prompt and the model's reply, and nothing leaves until you start a new session.

A worked budget

These sizes are my assumptions, chosen to be plausible for a small feature, not measurements:

• Instructions: 300 tokens.
• Five tools at about 150 tokens each for name, description and parameters: 750 tokens.
• One Generable type with a few @Guide notes: 200 tokens.

That is 1,250 tokens spent before the user says anything, leaving 2,846. If a turn is a 200-token prompt and a 300-token reply, that is 500 tokens per turn, so the chat has room for about five turns. If a tool returns a long result, fewer.

Your numbers will differ. The point is the shape: a fixed overhead you pay once, then a per-turn cost that adds up, in a window small enough that the overhead is a third of it.

What Apple tells you to do

TN3193 is written as a diet. Its advice, in my grouping:

• Measure first. Use tokenCount(for:) on your instructions, prompts, tools, schema and transcript entries, and use the Foundation Models instrument in Xcode to watch token use while the app runs.
• Split big tasks. Run each step in a new session and assemble the results. For a long article, summarize each chunk in its own session and pass the previous summary forward.
• Ask for less. Put the target length in the prompt ("In 3 sentences...") and cap array sizes with @Guide and a maximum count.
• Shrink prompts. Aim for one to three paragraphs, use concise imperative language, and give the model only the information the task needs.
• Keep types small. Short property names, and add @Guide only where the model needs help.
• Limit tools. Keep tool and @Guide descriptions to a short phrase and give the model a maximum of three to five tools. If the model should always have a tool's data, run the tool yourself first and put the result in the prompt.
• Use retrieval. For large knowledge bases, fetch only the relevant snippets instead of loading everything.

One warning stands out. Apple says to use maximumResponseTokens only to protect against runaway generation, because a strict cap can make the model produce malformed or grammatically incorrect partial output, like "A cat is a small."

When you hit the limit

The framework throws .exceededContextWindowSize, and the session cannot respond. A new session has a fresh window but not the old state. Apple's options are to summarize the old transcript into a new session, or keep only the important entries. Its example keeps the first and last:

func newContextualSession(with original: LanguageModelSession) -> LanguageModelSession {
    let all = original.transcript
    let condensed = [all.first, all.last].compactMap { $0 }
    var newSession = LanguageModelSession(transcript: Transcript(entries: condensed))
    newSession.prewarm()
    return newSession
}

That example is Apple's, lightly shortened. Keeping the first and last entry is a starting point, not a quality guarantee, so test what your feature actually loses.

Do not hardcode 4,096

Apple added two APIs for this. SystemLanguageModel.contextSize returns the maximum context size in tokens that the model supports, and tokenCount(for:) returns the token count for your input. Apple's contextSize page marks it as back-deployed before iOS 26.4, and its tokenCount(for:) page lists availability from iOS 26.4. InfoQ's write-up says both additions are back-deployed, so check the availability on your deployment target. InfoQ's reading is that the point is to remove the need to hardcode the limit.

There is a reason beyond tidiness. The SystemLanguageModel page says Apple periodically updates the model in routine OS updates, and currently lists three model versions: iOS, iPadOS, macOS and visionOS 26.0 to 26.3, 26.4, and 27.0. Apple's guidance is to test your prompts against old and new versions and to version them with #available checks. The technote documents 4,096 for the model it describes, and I did not find a documented window for the 27.0 model. Read the limit at runtime and re-measure after each OS update.

let model = SystemLanguageModel.default
let limit = try model.contextSize
let instructionTokens = try await model.tokenCount(for: instructions)

A pattern I would use

This section is my opinion, not Apple's.

1. Budget at launch. Count the tokens of your instructions, tools and schemas once, and compute how much is left for conversation.
2. Keep a reserve. Do not plan to fill the window. Leave room for a long reply.
3. One task, one session. Start a new session for each distinct job instead of one forever chat.
4. Decide the overflow behavior in advance. Summarize, trim, or hand the task to a server model. Do not let the error be the first time you think about it.
5. Re-test on each OS release. Prompts and limits can change with the model version.

What I could not verify

I did not run any of this on a device, so the budget example and the code are untested illustrations. I did not find a documented context window for the iOS 27.0 model, so I make no claim about it. Reports about newer Apple models describe input limits as undisclosed, and I did not use them. Apple's token-per-character figure is a rule of thumb, so measure your real strings. Device and region availability, and model behavior, vary and change.

Sources, accessed October 15, 2026: Apple Developer Documentation: TN3193, "Managing the on-device foundation model's context window"; SystemLanguageModel; contextSize; tokenCount(for:); "Updating prompts for new model versions" (developer.apple.com). InfoQ, "Apple Foundation Models context" (infoq.com, March 2026).

If your on-device feature kept a conversation going for ten minutes, do you know how many tokens it would have left?

#iOS #AI #OnDeviceAI #Swift

Work with me: I take 1:1 calls on mobile architecture reviews, mobile and AI questions, and startup tech consults. Book a slot on Topmate: topmate.io/sairamkotha

This article follows the post: what broke, what Scion changes, and what a team that is not Anthropic can take from it.

How to read it: every claim about Scion comes from Anthropic's engineering post "Redesigning source control at Anthropic", published October 13, 2026, which I read on October 14. I have not used Scion and have no inside knowledge. The arithmetic sections are mine and are labeled. Where the post was silent, I say so.

The numbers behind it

The post gives these figures for Anthropic's monorepo:

• Merges went from around 2,500 code changes per day in May 2026 to over 30,000 per day in September.
• The history of the main branch alone is almost 100 GB, and a compressed working copy is over 25 GB.
• CI performs on the order of 25 million code checkouts per day, before developer environments, agent sandboxes and research jobs.
• Fetched naively, that would be over 600 PB of git traffic per day.

The post's first line says that as merges per day grew more than tenfold, source control became the biggest development bottleneck.

My arithmetic on those figures: 30,000 merges a day is about one every 2.9 seconds, around the clock. 25 million checkouts a day is about 290 per second. And 600 PB divided by 25 million is about 24 GB per checkout, which lines up with the 25 GB working copy. So the 600 PB figure is just "every checkout downloads the whole repo."

Why git broke

The post's core diagnosis is a mismatch. Git is built for distributed, offline work: every machine holds the history and can work alone. Anthropic was using it as a centralized system with one authoritative copy, whose clients are nearly always online agents and jobs. The post says the properties that make git excellent for distributed, offline work were actively undermining that use.

The team had piled up the standard workarounds: shallow clones, sparse checkouts, promisor remotes, internal git mirrors and blob-storage seeds. At its peak Anthropic ran several hundred internal git mirrors. Three things gave out:

• Seeds drifted further from the tip of main between refreshes.
• Diff and blame on partial clones got harder to keep working as the merge rate rose.
• Mirrors stopped scaling once the repo outgrew their disks or memory, and became harder to operate as the merge rate increased.

None of this means git is slow. It means a design for a thousand offline laptops met thirty thousand merges a day from machines that are always connected.

What Scion does differently

The post lists six design principles:

1. Clients are always online. A client talks to a reachable server and fetches data when it needs it.
2. Client-server asymmetry. The server handles history, blame, commits, merges and rebases. The client keeps a bounded local cache and lays out the working copy.
3. Source control as a service. Queries are exposed through an API, so many jobs need no local repository. Automated bisect and revert pipelines depend on this.
4. Horizontal scalability. No full copy of the repo exists anywhere, even on the server, and there are no packfiles or repo maintenance.
5. Cheap, lock-free commits. Commits, rebases and merges happen on the server, so there is nothing to push. Refs are discouraged except for main and release branches.
6. Optional, sparse working copies. CI can ask the server for diffs, and many writes skip checkout entirely.

The data model is a changeset, an entry in a write-ahead log, created on the server. It lists changed files relative to its first parent, with content-addressed pointers to compressed blobs. Refs are a compare-and-swap on a single Spanner row. Blame, path history and commit-graph skip lists are derived data, computed from parent values, so the internals can change without client changes.

A small fleet of servers answers all the APIs. Because everything is content-addressed, any server can cache it without coordination. The client fetches only the trees and blobs it wants, starting at a root tree and recursing, so there is no multi-gigabyte packfile and transfers are resumable.

Delivery is tiered: a scion-daemon on each node, a regional scion-cache, and the authoritative scion-server that reads from Spanner or cloud storage on a miss. The post says tiering is strictly enforced so a failing regional cache cannot overwhelm upstream servers.

What Anthropic reports

All figures below are the post's own claims, not independent measurements:

• The Scion infrastructure footprint is 200x smaller than the previous git mirroring infrastructure, in CPU and memory.
• CI and research jobs check out code two orders of magnitude faster, in seconds rather than minutes.
• CI alone realized a 25% CPU capacity win.
• Each tier keeps a cache hit rate of over 99%.
• A team of seven people built Scion and the code review tool Antique and migrated to both in five months.
• Claude Mythos Preview rewrote the git-to-Scion ingestion pipeline, producing a version that used 20x less memory and ran 4x faster.

The post does not describe the cutover plan or downtime, so I can say nothing about how the migration was done.

The part most people skip

Two paragraphs in the post are about people and process, not storage.

Human-scale practices stopped fitting. The post says Anthropic has not been able to effectively replicate the industry standard of a global repository lock on the merge queue. It has invested in post-merge and pre-deploy automated testing instead, and says Claude can typically fix conflicts faster than humans can even identify them. Reviewers now examine the intent of a change instead of reading every line, and the code review tool can express complex semantic constraints on approvals.

The engineer's job moved toward saying no. The post says the role of the engineers was to tell Claude "no," and gives two examples: multiple systems that tried to brute-force looking up when a change merged to main, and one review that was overly defensive about trusting another internal service.

Who should not copy this

The post is direct about the limits. Scion is "by no means a suitable replacement for git for general use," and it fits an organization with either tens of thousands of engineers or swarms of agents prolifically writing code. The post says Anthropic's ability to spend tokens for internal use far exceeds what any other business would realistically want to spend. And because Scion diverges from git, adopting it means replacing code review, code search and CI too, which are highly bespoke inside Anthropic.

The post does not announce a release. It says it is difficult to offer Scion in a form that would be broadly useful, and that the most impactful thing Anthropic can do is share its thinking. It names three directions for the wider ecosystem: git servers that do not mirror client behavior, server-side RPCs on git platforms, and a redesigned negotiation protocol where sparse and shallow features compose well.

What a smaller team can take from it

This section is my opinion, not the post's.

1. Count your own checkouts. Most teams have never added up how many times per day CI, agents and sandboxes fetch the repo. At agent scale that number, not developer headcount, is what breaks things.
2. Treat source control as an API for machines. If a job only needs to know what changed, asking a server beats cloning.
3. Look for human-scale locks. Merge queues, serialized deploys and manual review gates all assume a human pace. Anthropic's own post says it could not make a global merge-queue lock work at its volume.
4. Do not rebuild git. Anthropic needed this at 30,000 merges a day with a seven-person team and its own models doing the work. Try the sparse and shallow tools git already has first.

What I could not verify

I read one primary source and did not test Scion or run any of the numbers. Every performance figure is Anthropic's claim. The post does not describe the cutover plan, downtime, or whether the 600 PB figure was ever actually transferred rather than being a calculation. Several news write-ups say Anthropic will not open-source Scion or give a different reading of the post. I used only the post's own wording. Details may change.

Source, accessed October 14, 2026: Anthropic Engineering, "Redesigning source control at Anthropic" (anthropic.com/engineering/redesigning-source-control).

If your agents started merging ten times more code next quarter, which part of your pipeline would break first?

#AI #Git #SoftwareEngineering #AIAgents

Work with me: I take 1:1 calls on mobile architecture reviews, mobile and AI questions, and startup tech consults. Book a slot on Topmate: topmate.io/sairamkotha