Will Apple Give Us Truly Personal AI?

René MagritteThe Listening Room. 1952.

A couple of weeks ago, in The George Carlin Model of AI, I said personal AI needed to work first on what George called a place for my stuff.

Should Apple occupy that whole personal space, kinda like René Magritte visualized in a surrealist painting seventy-five years ago?

I think that’s where they’re going. You can see hints of it in this headline, grabbed from an announcement last month:

Siri is Apple’s Clippy. Maybe worse, because it’s still alive and unloved after fifteen years of relentless promotion and disappointment. (Start reading down from the Reception subhead on Siri’s Wikipedia page for a partial account of Siri’s failings. A lot there.) But Apple is investing the next generation of Siri—Siri AI— with huge new responsibilities—to you. But is it really—

?

On the privacy topic, Apple smokes a lot of its own exhaust. Find examples  herehereherehere, and here. A couple of years back, in response to the first of those, I wrote here and here about how Apple falters on the privacy front, despite its many promises. But I give it points for staying on the case, which will get a lot bigger with its next operating system.

Here is a June 8 press release that lays out what Siri AI will do, under this headline:

And here’s a video in the style of a movie trailer, laying out Siri’s fancy new features. Excerpts:

Coming to a pocket near you… The Siri AI glowup hits the small screen. See Siri AI in the role of a lifetime: your personal assistant.

Ask Siri anything and revisit your conversations with ease in a new dedicated app.

Watch Siri embark on a quest to provide nutritional details about your meal. Yummy.

Cheer as Siri AI handles every detail of your soccer watch party. Go! Siri is giving main character energy and puts the personal in personal assistant.

See next-generation Apple intelligence extend, reframe, and clean up your photos. Booya!

Create photorealistic images and flex your imagination everywhere from contact posters to fun wallpapers in the all-new Image Playground.

Safari has never been more organized with topics, or timely with Notify Me. Rest easy knowing your data is protected as compromised passwords are automatically updated with just a tap.

And see Apple intelligence stand guard to protect your privacy. A faithful Sentinel keeping your private info private. With Siri and Apple intelligence, your data stays protected.

All this and so much more coming to an iPhone near you.

None of this is a place for your stuff.

I think it needs to be a box: a dedicated one, like your closet or garage. I also think it’s coming, because Apple knows why Mac Minis are selling out:

I think this suggests that Apple will pitch the next Mac Mini (an M5 one) as the personal AI machine, meaning the place for your stuff.

Now let’s look at some of the specifics behind the promotional jive:

1. On-Device “Personal Context”: A new architecture (not the old Siri) maps your device locally, using Apple Silicon’s Neural Engine to index information across your Apple applications: contacts, calendar, reminders, messages, emails, documents, photos. As for your non-Apple stuff, such as my million-plus photos that are not in Apple’s Photos app, it looks like it’s already on the case. When I search for “tunnel” across my photo directories with my laptop (2023 MacBook Pro running Tahoe 26.5.1), I get every shot where that word appears, plus lots of stuff that is either a tunnel or looks kind of like one. Example:

Clearly an AI does some pattern recognition there, but is that “personal context”? I dunno.

It does have “Semantic Indexing,” which makes informed presumptions about the meaning of your data, and not just your keywords. Big AI does this now, but Siri will do it just for you, on your stuff, inside your place for it. Note what it says under the “Apple Intelligence in Apps” subhead here:

Express yourself through photos and images, save time with Safari, and get more done with Apple Intelligence seamlessly integrated into your everyday apps and experiences.

But do we want “seamless” everything? We need edges and boundaries to make sense of the world. In the absence of those, I want the option to turn that off, or not turn it on

2. Private Cloud Compute (PCC) is how Apple describes another place for your stuff: kind of a private office in Apple’s hi-rise downtown. Specifics:

For advanced features that need to reason over complex data with larger foundation models, we created Private Cloud Compute (PCC), a groundbreaking cloud intelligence system designed specifically for private AI processing. For the first time ever, Private Cloud Compute extends the industry-leading security and privacy of Apple devices into the cloud, making sure that personal user data sent to PCC isn’t accessible to anyone other than the user — not even to Apple. Built with custom Apple silicon and a hardened operating system designed for privacy, we believe PCC is the most advanced security architecture ever deployed for cloud AI compute at scale.

The authors of that text are Apple Security Engineering and Architecture (SEAR), User Privacy, Core Operating Systems (Core OS), Services Engineering (ASE), and Machine Learning and AI (AIML)—all inside the company. They say lots more at that last link, all helpful to know. So is Expanding Private Cloud Compute, by the same teams.

3. Systemwide app actions: This new assistant can, for example, cross-reference a tracking number from your email and a message thread to find who asked for it, pull out other relevant information, then automatically drop it into a reply for you to review or edit before you send it, all in your virtual cabin (device) or office (private cloud).

4. Controlled federation, anonymized gateways, a privacy shroud, and other jive required to make this work:

From this press release.

I gather, from Apple’s literature, that Siri strips out your IP address and personal identifiers before making a query to an external AI. The external AI agent sees only the isolated query. This prevents the external AI from examining the personal stuff in your online home.

Now here is a tough question: What if only a giant can put together most or all of what we need? Three giants currently furnish most of our personal spaces in the digital world:

  • Apple (iOS and MacOS devices, Safari browser, etc.)
  • Google (Android devices, Gemini, Chrome browser, etc.)
  • Microsoft (Windows OS and devices, apps, etc.)

With iOS and MacOS 27, Apple moves to the front of that pack in the personal AI space, and will likely be the only giant to offer something that looks like a place for your stuff. Given its role in the surveillance fecosystem, Google can’t be trusted. Microsoft still has Micro in its name, but it has become much more of an enterprise company in recent years. So, among giants, Apple is it.

Now let’s talk about agents.

Apple sees you with just one: Siri AI. But you will probably need many agents: one or more for health (in various specialties), financial (banking, investment, credit), travel (airlines, car rental, hotels), home economics (property, stuff in storage, scheduling the kids, keeping the car working), legal (all your contractual commitments, plus much better customer-company interactions than are possible today).

This can get very complicated. As Opaque explains,

Here’s the thing: if a single chatbot request is too risky to run unverified, what does that say about agents?

A chatbot is one request in, one answer out. An agent runs that risk in a loop: reading email, opening files, calling tools, handing work to other systems, unattended and at machine speed.

No breach required. An agent doing exactly the job you gave it moves your data constantly into places you don’t control and mostly can’t see.

Now wire thousands of agents together, the way every enterprise is planning to this year. Whatever the per-step risk is, compounding turns it into a certainty.

Apple just deployed Confidential AI to protect the smallest risk surface in AI. Enterprises are wiring up the largest with nothing underneath it.

Opaque sells arms to enterprises, so it’s not in the personal AI business. But it does make a good point in its opening sentence:

“Apple looked at a simple chatbot, the single most contained form of GenAI there is, and decided the data it leaks is too dangerous to ship to their customers without Confidential AI underneath it.”

To Apple, the more personal the context, the higher the privacy stakes. That’s why it believes personal AI has to run—

  • on-device (the place for your stuff) and
  • in a privacy-walled cloud infrastructure (your private office in Apple’s high-rise cloud)

The former can actually cover a lot of ground in your life, just by helping you get on top of all the stuff in your digital home. It can also handle some simple interactions with outside entities, such as MyTerms ceremonies and record-keeping.

But you’ll need much more from your personal AI if you’re going to scale your life out into the larger world, where nearly every company, every government agency, everything you might subscribe to, and even every church and nonprofit, wants to have AI agents for interacting with you and your digital agents.

As of today, Apple isn’t ready for that. Nor is anyone else. But researchers are helping. In Too Private to Tell: Practical Token Theft Attacks on Apple Intelligence, four researchers from Ohio State University say this in their abstract:

Apple Intelligence is a generative AI (GenAI) service provided by Apple on its devices. While offering a similar set of features as other similar GenAI services, Apple Intelligence is claimed to be designed with an extra focus on user security and privacy through a two-stage authentication and authorization design using anonymous access tokens. In this paper, we present our investigation into this token issuance mechanism with a goal to reveal possible vulnerabilities using traffic analysis, reverse engineering, and cross comparison with Apple’s public documentation. Specifically, we present the Serpent attack, the first practical cross-device token replay attack against Apple Intelligence that allows the attacker to steal the access tokens from the victim’s device and utilise them on a different device, with all usage rate-limited against the victim. We have achieved successful attacks on the latest macOS 26 Tahoe and demonstrated that an attacker, who even has used up its own allowance, can immediately regain access to Apple Intelligence service. We have responsibly disclosed the vulnerabilities to the vendors and received confirmation from Apple with CVE assigned and bounty given. Our results highlight a general lesson for built-in AI services: Anonymising identity does not by itself make the AI service secure; Enforcing non-transferability requires cryptographic binding to the rightful user.

We assume that Apple is addressing those concerns, plus a near-countless number of others, with macOS 27 and iOS 27. We’ll see later this year, presumably. (Apple is better with promises and forecasts than most other giants, but not perfect.)

Humans invented privacy with the technologies we call clothing and shelter. We don’t have clothing yet in the digital world, or we wouldn’t be walking worse-than-naked across the Net, covered with thousands of invisible data-sucking ticks called cookies and tracking beacons: parasites that report who-knows-what to god-knows-who, across thousands of unseen and unknown paths.

But we might get shelter, or the beginning of a working model for it—a place for our stuff—from Apple and these other companies and projects.

Apple seems to understand some of this, at least architecturally, to some degree. I think others (including those listed here) understand it more deeply. But none of them have Apple’s heft.

As for the enterprise side of this, there are growing bodies of work coming from Nitin BadjatiaIain Henderson, and Jamie Smith. All three see empowered customers coming to the marketplace with agentic AI capabilities that will strip the gears of existing enterprise systems, including those with AI agents.

In Confidential AI Just Hit Escape Velocity (published on 13 June), Aaron Fulkerson, CEO of Opaque, writes this:

Apple just set the bar every enterprise will be measured against

Escape velocity is the moment a category stops needing evangelism, when the question flips from “do I really need this?” to “why don’t you have it?” Three things flipped it this month.

First, the existence proof landed at the hardest difficulty setting. Apple just rolled out the largest Confidential AI deployment in history: every iPhone, at consumer latency, consumer cost, consumer scale. Every objection enterprises have leaned on, too slow, too expensive, more than we need, just got falsified a billion times over by a phone.

Second, this is already how the giants operate. Meta runs WhatsApp message AI through private processing. Google built Private AI Compute so Gemini can process your personal data in a sealed environment that, in Google’s own words, not even Google can access. Anthropic and TikTok run their own implementations. And Microsoft, Google, and NVIDIA ship the underlying confidential infrastructure across their clouds and silicon. The pattern is consistent: every company with world-class security talent, when forced to put AI against sensitive data at scale, lands on the same architecture. When that many teams solve the same problem independently and arrive at one answer, you’re looking at convergence.

On our side—the customer’s side—we need confidential personhood, based on personal sovereignty: root for the person. In other words, personal AI needs to be operated by the person, not just for the person.

So let’s suppose Opaque succeeds perfectly. Enterprises will have attestable hardware, secure enclaves, confidential containers, encrypted memory, verifiable runtimes, machine-speed agents, and other whatevers we’ve been reading about.

We will need the same. The flow should go like this:

Natural person
    ↓
Personal AI
    ↓
Personal terms (MyTerms)
    ↓
Confidential runtime
    ↓
Outside agents and services
    ↓
Network

Note also that the flow here is top-down from the person, the individual—rather than bottom-up from “the consumer” or “the user.”

Almost everybody talking about agentic AI today is looking only at the lower half. But that half won’t run without our permissions from the upper half. That’s why we (the working group I chaired) worked for nine years on  IEEE 7012-2025—Standard for Machine-Readable Personal Privacy Terms, nicknamed MyTerms. As I say at that link, MyTerms is the only way we’ll get personal privacy in the digital world. Apple, please adopt it. Everyone else, jump on board too. It’s radically simple to implement:

MyTerms are contractual agreements about personal privacy that you proffer as the first party, and the company agrees to as the second party. With MyTerms, you don’t “consent” to the company’s privacy policies or whatever they say about their use of cookies. They agree to your privacy requirements, which will limit the use of cookies and tracking tech to only what you allow. You are not a mere “user” or “client.” You are an independent human being operating with full agency.

In a way, Aaron Fulkerson’s post argues a need for work on the upper half. Because, while he says, “the request never travels on trust,” our social and economic lives are based entirely on trust: contracts, promises, agreements, agency, representation, delegation.

If my personal agent books a hotel, negotiates a subscription, grants limited use of my health data, tells my bank to move money, buys something, or participates in market intelligence that flows both ways, those acts and processes aren’t just computations and transactions. They are relationships. And those require identity, delegated authority, obligations, records, audit trails, and remedies. Those all need to start with My Terms.

At scale, remedies also need to be based on ODR (online dispute resolution), which is thankfully a mature field—and one that MyTerms will expand.

I suspect Apple, Opaque, and MyTerms are each solving a different problem posed by a place for my stuff:

LayerQuestionExample
Confidential computingCan I trust the machine?Opaque, et al.
Personal contextDoes the machine know me?Apple, et al.
Personal sovereignty (confidential personhood)Does the machine represent me?MyTerms
Dispute & accountabilityWhat happens when things go wrong?ODR

In each case, the place for my stuff is a machine: my (or your) machine, and possibly my (or your) private cloud. Nobody is building that whole stack yet. Nor should anybody. Not if we want each layer to scale.

So here is a question. What if:

  • Apple provides the shelter (then competitors follow),
  • Opaque (and its competitors) provide the locks,
  • Linux and open source hacks provide the plumbing, and
  • MyTerms provides the constitution—or at least some solid ground under a new constitution for personal agency, independence, and privacy online?

If personal AI becomes ubiquitous, agents will do things that matter legally and socially. The questions that matter then are, “Under whose authority?” and “How is that authority secured?”

The answers to both require contracts in which the person is the first party. Fortunately, contract law is well established everywhere, and contract itself is specified by Article 6 of the GDPR as one of the lawful bases for processing personal data. (Dive deeper here if you like.)

So, while we wait for Apple to drop other giant shoes, let’s start putting MyTerms to use. Our home—places for our stuff—on the Net won’t be secure without them.



Leave a Reply

Your email address will not be published. Required fields are marked *