Can you trust ChatGPT with your medical records?

OpenAI is moving its health data tools out of a separate, contained space and into your main conversation window. Previously, you had to use a specific health-focused hub to keep your queries private and separate from your general chats. Now, you can allow ChatGPT to reference your connected medical information whenever you ask a question in the main app. OpenAI states that it works with physicians to improve how the underlying AI handles medical topics and that it does not use your uploaded health data to train its models. Despite these updates, the company maintains that these tools are not meant to diagnose or treat health conditions, and users are explicitly told to verify all medical information with a licensed professional.

Large language models—the technology powering ChatGPT—function like incredibly advanced, highly literate autocomplete engines. When you ask them a medical question, they aren't looking at your symptoms and analyzing them like a physician would; instead, they are statistically predicting the next word in a sentence based on the vast archive of text they were originally taught on. While OpenAI has developed specific tests, known as benchmarks, to measure how well their models answer medical questions, these tests have limitations. A benchmark is essentially a standardized quiz designed to see if an AI gets the right medical answer. However, passing a quiz is not the same as having clinical judgment. Because these models are designed to be helpful and conversational, they can occasionally present incorrect or dangerous advice with extreme confidence, a well-known tendency in the industry often called hallucination.

Giving an AI assistant access to your lab results, doctor notes, and activity history can certainly make for a more helpful conversation, but it shifts the burden of verification onto you. If you connect your medical records, you are inviting a system that can make statistically plausible but medically incorrect statements to help summarize your health data. While companies are racing to integrate these tools into our medical lives, the technology lacks the accountability of a human doctor, who is licensed and legally responsible for their advice. Before connecting your records, it is worth remembering that ChatGPT is an assistant, not a clinician, and it is prone to mistakes that, in a medical context, could have serious consequences for your health.