Why we still don't know how AI is actually used

A group of researchers has launched the AI Observatory, an independent project designed to gather and analyze real conversations between humans and AI. By aggregating thousands of interactions from existing datasets, they discovered that company-released reports ignore a significant portion of reality. For instance, while companies might highlight coding or productivity tasks, the Observatory found that nearly half of all conversations involve personal topics, health questions, or even illicit content. Furthermore, the way people interact with AI changes based on the specific model they choose, with some platforms becoming hubs for news and politics and others used more for roleplay or personal support. Because companies treat their own usage data as proprietary, independent researchers are forced to patch together a picture from fragmented sources to understand the actual societal impact of these tools.

Much of the conversation around AI assumes these systems will soon reach a point of recursive self-improvement—the idea that an AI could essentially teach itself to become smarter without human help. However, recent testing shows this is harder than it sounds. Researchers challenged AI agents to perform original, open-ended scientific research—the kind that requires intuition, taste, and the ability to pivot when a theory isn't working. While the AI was excellent at the engineering side—running experiments and organizing data—it failed to produce anything of substance. It struggled to judge which approaches were worth pursuing and lacked the creative judgment to know when to start over. This happens because AI models are currently trained using reinforcement learning, a process where they are rewarded for hitting specific, clear goals. When a task has no clear, objective right answer, the AI acts in a formulaic way, lacking the human spark of critical thinking.

The gap between the AI we have today and the self-improving super-intelligence often discussed in marketing materials is significant. If we rely on corporate narratives to understand how AI is used, we risk making policy decisions based on incomplete evidence. Whether it is a student using AI for homework or a researcher failing to get a novel discovery from an automated system, the reality is that these tools are far from being the all-knowing, self-perfecting agents we are often promised. We are still in the early, messy days of figuring out what this technology is actually good for, and it is crucial that we look beyond the glossy reports released by the companies that sell it.