The largest coming conflict in the new AI world is not the one between AI giants or the one between those giants and governments. It will be the conflict between containment and expansion of personal agency. On the side of containment are expanded surveillance, guesswork, and entrapment in walled corporate gardens. On the side of expansion are tools that serve self-knowledge and expression of personal intent in the marketplace, including the intent to enjoy personal privacy, and have that work for both sides.
On the containment side, personal intent is already inferred by corporate agents. On the expansion side, personal intent will be expressed by personal agents—ones owned and operated by independent and self-sovereign human beings. In the long run, agents on both sides will work cooperatively to expand communication and economic activity. But that can only happen when mechanisms that assure trust are in place. We don’t have those yet. Not while surveillance continues to rule, and personal privacy is what corporate “consent” systems allow. Which rounds to none.
But it’s still early. The Digital Age has been with us for decades at most and will persist for centuries to come. AI as we know it today is much younger.
I coined “The Intention Economy” in a 2006 column with that title. That same year, I started ProjectVRM at Harvard’s Berkman (now Berkman Klein) Center. My book The Intention Economy: When Customers Take Charge (Harvard Business Review Press, 2012) posited a future described this way:
Over the coming years customers will be emancipated from systems built to control them. They will become free and independent actors in the marketplace, equipped to tell vendors what they want, how they want it, where and when—even how much they’d like to pay—outside of any vendor’s system of customer control…
Demand will no longer be expressed only in the forms of cash, collective appetites, or the inferences of crunched data over which the individual has little or no control. Demand will be personal. This means customers will be in charge of personal information they share with all parties, including vendors.
Customers will have their own means for storing and sharing their own data, and their own tools for engaging with vendors and other parties…
Thus relationship management will go both ways. Just as vendors today are able to manage relationships with customers and third parties, customers tomorrow will be able to manage relationships with vendors and fourth parties, which are companies that serve as agents of customer demand, from the customer’s side of the marketplace.
Relationships between customers and vendors will be voluntary and genuine, with loyalty anchored in mutual respect and concern, rather than coercion…
Likewise, rather than guessing what might get the attention of consumers—or what might “drive” them like cattle—vendors will respond to actual intentions of customers. Once customers’ expressions of intent become abundant and clear, the range of economic interplay between supply and demand will widen, and its sum will increase. The result we will call the Intention Economy…
The volume, variety and relevance of information coming from customers in the Intention Economy will strip the gears of systems built for controlling customer behavior, or for limiting customer input. The quality of that information will also obsolete or re-purpose the guesswork mills of marketing, fed by crumb-trails of data shed by customers’ mobile gear and Web browsers. “Mining” of customer data will still be useful to vendors, though less so than intention-based data provided directly by customers.
In economic terms, there will be high opportunity costs for vendors that ignore useful signaling coming from customers. There will also be high opportunity gains for companies that take advantage of growing customer independence and empowerment.
The Intention Economy inspired Sir Tim Berners-Lee’s Solid Project, Consumer Reports‘ Permission Slip, work on personal AI at Kwaai.ai (which I serve as Chief Intention Officer)—among the many other efforts listed by ProjectVRM, which is still active. You can follow and join here.
But, while tools for independent personal agency are still in their infancy, work proceeds on the corporate side, which imagines that personal intent is best inferred and controlled, rather than obeyed. As both muddle in their own ways, are there middle grounds to explore?
Shuwei Fang of Reuters and the Harvard Kennedy School is looking into that space. In The information ecosystem is being redrawn by AI, she writes,
3. From artefacts to liquid information
Once machines are the primary audience, the artefacts we have built around information (the article, the bulletin, the documentary) become structurally optional.
Fragmentation was the old problem. Now information can exist without a container at all. And when information becomes liquid, the market bifurcates. Economic value concentrates at two extremes of a ‘barbell’; premium brands competing on differentiation and direct audience relationships; and commodity infrastructure operating at massive scale and razor thin margins. The middle, where most publishers currently sit, hollows out.
4. From attention to intention
When content is liquid and an intermediary controls the interface, something more consequential than format changes. The currency does. Clicks, pageviews, time on site; these were always crude proxies for what we actually wanted to know: what do people need? Now there are systems that can infer the answer directly. I have written elsewhere about this shift from attention to intention as the defining transition of this era.
Good points. Now, what about systems that start with individual customers and their agency, their intent?
Shuwei’s linked piece, From Attention Merchants to Intention Architects: The invisible infrastructure reshaping human curiosity, has this paragraph:
Researchers are beginning to document the foundations of this phenomenon, what some at Harvard Data Science Review and elsewhere call the ‘intention economy,’ where AI systems collect, commodify, and potentially manipulate user intent. But this only scratches the surface. What they’re witnessing is accompanied with a fundamental restructuring in how information flows through society. In the intention economy now emerging, AI systems could compete to anticipate and shape what those eyeballs seek before they even know they’re seeking it. The infrastructure being built right now, largely invisible to most of us, won’t just determine what we see; it will determine what we want to see before we know we want it.
The first link in that paragraph goes to a piece that hijacked the meaning of intention economy. I tried to pull it back with The Real Intention Economy, and I’m doing the same here. I hope Shuwei reads it. The book too.
Meanwhile, let’s get back to Shuwei’s From Attention Merchants to Intention Architects: The invisible infrastructure reshaping human curiosity, which concludes,
But here’s what gives me hope: patient capital and policymakers are beginning to recognize infrastructure as the leverage point. The standards aren’t set yet. The architecture remains fluid. If these signals are correct, we might be at a rare moment where we can see a paradigm shift coming. Unlike previous shifts that caught democracy off guard (radio’s consolidation, television’s commercialization, social media’s polarization) we might actually have warning this time. If curiosity is becoming the new scarcity, if intention shapes outcomes more than attention ever could, then whoever builds the curiosity infrastructure could write the future of human understanding. And unlike attention, which is zero-sum and depletable, curiosity can grow through exercise, each question potentially spawning new questions, expanding rather than exhausting with use. This offers hope: the right infrastructure could create abundant understanding where the attention economy leveraged scarcity.
For centuries, democracy fought for the right to know; freedom of information, transparency, the end of censorship. If AI makes all information instantly accessible, we might face a new frontier: ensuring the courage to question survives the comfort of infinite answers. The intention economy might not be inevitable, it could be a design space waiting for architects. The question isn’t whether we’ll have curiosity infrastructure, we probably will. The question is whether we’ll build it to expand human wonder or contract it.
Indeed.
To expand human wonder, we need Personal AI, not just the mainframe kind we have now. And within personal AI, we need mechanisms that enable and express far more personal agency than can ever be provided through the kinds of conversations we have today with Big AI.
For full personal agency to thrive, we must have personal privacy. Because what we—the people—don’t say, and keep to our own private selves, may be far more meaningful and leveraged than anything that leaks out through AI queries. For that, we can start with contractual agreements such as those outlined by MyTerms (IEEE 7012).
We will indeed use AI agents to express our intentions (including the one to be left alone). But they will be our agents, not ones surveilling us.
We’ve been waiting eighteen years for those agents to arrive. It may take another one, three, five, or twenty years for that to happen. But it will.
Leave a Reply