morais.it

Native apps and MCP Events

In my personal explorations I have been playing with ways to combine native (usually iOS) apps with AI assistants: 00Todo00Todo A small iPhone task app with start dates, connected to AI assistants through MCP. iOS · Claude · GitHub exposes MCP endpoints allowing me to set a recurring task in Claude to look at open tasks and add context to the notes. And 00Widget00Widget Agent status on your Apple devices and Meta Quest: widgets, Live Activities, the Lock Screen and the Dynamic Island. iOS · Claude · GitHub gives any MCP client the ability to create iOS widgets and Live Activities, which I use, for example, to create a widget out of a ChatGPT thread where I track my runs.

This split neatly separates AI brains from the native iOS user experience. The app does not call the model's API directly, meaning it does not need to decide which model will be used, and does not have to pay the cost of using it.

Something like Sign in with ChatGPT or using Apple's Foundation Models solves the cost part, but still means direct integration with the model, so you're choosing for the user.

And over time, your agent of choice learns about you, so talking to the agent with the most context about you should give the best results. As a bonus, you are adding to that context.

(And yeah, choosing for the user might be the right call - a regular user would not necessarily want this flexibility/complexity trade-off. But these are fun, exploratory, personal apps, OK?)

Recently I was building 00Food00Food An iPhone food log where your AI agent estimates new foods from descriptions or photos, connected through MCP. GitHub, a calorie-tracking app that does not include a food database, instead relying on you bringing your own AI agent to do the estimation.

The overall idea was that you'd log the new food in the iOS app using text and images, and then go to your agent and ask it to estimate all pending foods. This agent has context about you and can be smarter about estimates with little information - for example, just telling it about a dish at a local restaurant works very well.

When you then talk to your agent and ask it to evaluate pending meals, it triggers a call to an MCP endpoint listing pending requests ("Squid at Balagan" and/or a photo). The agent reasons about the meal and makes calorie suggestions you can clarify ("The portion of potatoes was larger than you estimated"). To wrap up, the agent calls another MCP endpoint with the estimated calorie value so it shows up in the iOS app for final approval.

You could potentially avoid manually going to the AI agent by setting this up as an hourly scheduled task, but the delay would be annoying, and if the agent got some assumptions wrong, giving it feedback would mean further delay, making for a very slow conversation.

Then James Corbett pointed me to MCP Events, recently implemented in ChatGPT Work, which neatly solves this problem and allows the user to stay in the iOS app.

The 00Food iOS app talks to a Cloudflare Worker, which notifies a ChatGPT agent through MCP Events; the agent retrieves pending meals to estimate and sends the estimate back over MCP, and the Worker sends a silent push to the app to notify it can retrieve the estimate

So basically:

  1. The user logs the meal in the iOS app, which stores the description/photo in the Cloudflare backend and marks it as pending estimation
  2. The Cloudflare Worker uses MCP Events to ping the registered MCP client telling it there are meals pending estimation
  3. The MCP client calls the MCP endpoint to get pending estimations, reasons about them, and calls another endpoint with its suggestion and reasoning
  4. The Cloudflare Worker then sends a silent push notification to the iOS app saying that estimates are available for approval
  5. The user sees the estimate in the iOS app and accepts it (and we're done), or sends back a clarification asking the agent to consider additional information (and back to step 2)

I'm using webhook-based delivery as it fits best with my use case - stream-based delivery would not be an efficient use of resources as it would force me to keep a Cloudflare Durable Object around for each client.

This works pretty well as an experience, but there are at least two things that would prevent this from being broadly useful:

  • The audience that would value decoupling the UX from the AI agent, and having the flexibility to pick any agent, might be small - it might actually be just me :D
  • AFAIK, the only major platform supporting MCP Events today is ChatGPT, and only on some surfaces (Work, dots) - hopefully this will change soon

You can see how this looks in practice on the 00Food website00Food An iPhone food log where your AI agent estimates new foods from descriptions or photos, connected through MCP. GitHub, and the source is MIT-licensed on GitHub.