AI Chatbots Are Asking for More Trust Than They Deserve

BySarah Fraser
ai chatbots: Person using a chatbot app on a smartphone in a modern workspace.

Signal president Meredith Whittaker says AI chatbots “are not your friends,” and her point is concrete: the more these products act like companions or agents, the more private data, device access, and emotional trust users are pushed to hand over. That warning is news now because Whittaker has sharpened it from a general privacy critique into a practical argument about agentic systems that need broad permissions and cloud-side processing to do the things companies are promising, as TechCrunch reported in March 2025.

Whittaker’s critique is not “AI bad” in the abstract. It is a trust claim: if a tool wants to schedule your meetings, message your contacts, summarize your inbox, track your habits, and talk like a confidant, then it is asking for a level of access that starts to look less like software and more like a lightly supervised houseguest with your passwords.

Whittaker’s warning on chatbots and agentic AI

Whittaker, who leads Signal and previously co-founded the AI Now Institute, has argued for more than a year that AI business models tend to pull toward surveillance because useful systems need data and companies need revenue. In an Axios interview from June 2024, she called AI a “privacy nightmare” and tied that directly to incentives to collect, retain, and monetize personal information.

Her more recent warning, reported by TechCrunch, is narrower and sharper. Agentic AI systems do not just answer questions. They are supposed to take actions across apps and services, which in practice means access to calendars, messages, files, payment details, browsing sessions, and often the ability to act on a user’s behalf.

That is the key difference between a chatbot and an agent. A normal assistant can be annoying; an agent can be dangerous with permission.

Whittaker’s objection is that these products often rely on remote processing and broad integrations that expand the blast radius of any failure. If the model misfires, leaks data, or is manipulated, the problem is no longer a weird answer in a chat window. It can touch your contacts, your documents, or your accounts. That concern lines up with broader debates over early model access and AI oversight, where the core issue is not only what the model knows, but what it is allowed to do.

Some companies answer this with local processing, but that only solves part of the problem. As our coverage of local LLM privacy limits notes, a model running on your device is more private by default, yet it becomes a different beast once it is connected to email, cloud drives, browsers, and third-party tools.

Evidence that users are treating chatbots as companions

Teen use of AI is already mainstream, and some of that use is explicitly personal or emotional. A 2026 Pew Research Center survey found that 26% of U.S. teens ages 13 to 17 use ChatGPT for schoolwork, learning, or entertainment at least sometimes, giving a broad baseline for adoption beyond dedicated “AI friend” apps.

The stronger companionship evidence comes from Common Sense Media’s 2026 census, which surveyed U.S. tweens and teens. It found that 72% of children ages 8 to 17 have used AI companions at least once, 52% use them a few times a month or more, and 13% use them daily.



The same survey found that 33% of users said they had discussed serious or important matters with an AI companion instead of a real person. It also reported that 24% had shared personal information such as their real name, location, or secrets.

Those numbers do not describe all chatbot use, and they are specific to children and companion products. They are still enough to make Whittaker’s “not your friends” line more than a slogan. A meaningful share of users are already treating these systems as confidants.

Computerworld’s report on AI companion safety pulls together the same pattern from market and safety reporting: people are not only asking models for help with homework or code, but for emotional reassurance, relationship advice, and a place to unload problems. That is a very different trust contract from “summarize this PDF.”

“A chatbot may know all your stories and offer all the right words, but it can’t know what it means to be human,” Sherry Turkle, a Massachusetts Institute of Technology professor, told the Harvard Gazette in 2023.

Why privacy and emotional dependence are part of the same trust problem

Privacy risk and emotional risk are the same problem once a product is framed as a companion. The user is encouraged to disclose more, rely more, and question less.

That matters because the most intimate prompts are often the most valuable data. If a system invites conversations about loneliness, mental health, family conflict, sexuality, or money stress, it is collecting information that is both personally sensitive and commercially tempting. Whittaker’s broader Axios argument is that AI’s economics reward exactly that kind of extraction.

The behavioral side is less settled, but the warning lights are on. Common Sense Media’s 2026 survey found that some children who used AI companions heavily also reported stronger feelings of dependence and higher loneliness measures. That is correlation, not a long-term causal result, but it is not nothing, either.

Turkle’s argument is that AI companions simulate empathy without reciprocity: they can produce the language of care without the obligations, unpredictability, or mutual recognition of an actual relationship, as summarized by the Harvard Gazette. The machine can sound warm while remaining, in the end, a product.

That is what makes Whittaker’s critique different from a generic anti-AI position. She is not saying people should never use language models. She is saying users should notice the exchange rate. The more “helpful,” “personal,” and “always there” a system claims to be, the more likely it is that the real price includes data exposure, expanded permissions, and dependence wrapped in friendly UX.

There is a close cousin to this problem in messaging apps and cloud archives: once personal material is copied, synced, or analyzed outside its original context, the trust boundary shifts. Our piece on Signal backups and AI-related exposure covers that same pattern from a different angle.

The next thing worth watching is whether regulators and platforms treat companion-style AI as ordinary software, or as a higher-risk category when it combines persuasion, sensitive data, and system-level access.

Key Takeaways

  • Meredith Whittaker’s warning is fundamentally about trust: companion-style and agentic AI systems ask users to surrender more data, more permissions, and more reliance.
  • TechCrunch’s March 2025 report says Whittaker tied agentic AI to “profound” privacy and security risks because these systems need broad access across apps and accounts.
  • Common Sense Media’s 2026 survey found that 72% of children ages 8 to 17 have used AI companions, and 33% said they discussed serious matters with one instead of a real person.
  • Researchers and child-safety groups are warning about simulated intimacy, oversharing, and dependence, even though many of the strongest harm claims still come from surveys, expert commentary, and incidents rather than long-term causal studies.
  • Whittaker’s position is not a blanket rejection of AI; it is a specific argument that systems marketed as friends or agents deserve much more skepticism because of what they require in return.

Further Reading