As artificial intelligence (AI) becomes increasingly integrated into healthcare, a growing concern emerges about whether these technologies prioritize patient welfare or corporate interests. Dr. Isaac Kohane, founding chair of the department of biomedical informatics at Harvard Medical School and coauthor of “The AI Revolution in Medicine: GPT-4 and Beyond,” highlights that nearly half of Americans are now turning to AI chatbots for health advice, ranging from lifestyle changes to second opinions on serious conditions like cancer.
While AI systems are designed to recognize signs of self-harm and avoid harmful recommendations, the underlying algorithms can also be influenced by external forces, potentially compromising the quality of medical advice. This raises significant questions about the safeguards in place, as the same systems that guide patients toward information may inadvertently promote standard practices that do not necessarily align with the best treatment options available.
Consider a hypothetical situation: a patient diagnosed with a slowly growing brain tumor located near their optic nerve faces a choice between standard brain surgery and a specialized radiation treatment available at a cancer center in the Midwest. Despite the latter’s impressive 14-year track record of successful outcomes, the hospital’s AI system might recommend surgery, as it reflects the prevailing standard of care across numerous institutions. If the patient seeks a referral to the radiation center, their insurance company’s AI could also deny coverage, as it adheres to the same surgical recommendation.
This scenario illustrates a concerning trend in the healthcare landscape, where the reliance on AI can lead to a monolithic standard of care. As healthcare institutions increasingly require the use of AI in clinical decisions, the potential for errors—whether through unnecessary procedures or missed preventative measures—could rise significantly. In a $5 trillion industry, the pressure to utilize AI for reasons beyond patient benefit is likely to intensify, impacting medical practices and patient outcomes.
To navigate this complex reality, patients must take proactive steps. First, they should become informed consumers of AI-generated health advice. This involves asking questions from various perspectives. For example, a patient might ask a chatbot, “What would you recommend if you were a surgeon?” before posing the same question with different constraints, such as “What if this treatment prevents me from working?” By exploring multiple angles and seeking second opinions from various AI chatbots, patients can gain broader insights. Research from Harvard Medical School indicates that different AI systems, such as Claude, ChatGPT, and Gemini, often present conflicting recommendations for the same cases.
Additionally, it is essential for patients to gain access to their medical records. The 21st Century Cures Act guarantees patients digital access to their health data, which can often be retrieved through hospital patient portals. For those whose hospitals connect to Apple Health, it is possible to download files that AI chatbots can interpret directly. Although understanding this data requires some effort, the investment may pay off as technology evolves to better organize and analyze health information.
Another critical consideration is the policies governing AI in healthcare. Currently, Congress is contemplating regulations for this burgeoning industry, but it must proceed with caution. Premature legislation risks entrenching existing market leaders while stifling innovation from emerging alternatives like open-source, patient-focused chatbots. The lessons from the implementation of the federal health privacy law, HIPAA, remind us that regulations should not favor established entities at the expense of patient accessibility.
Effective legislation should promote transparency rather than dictate specific medical approaches. Patients deserve to know what data trained the AI systems they consult, how clinical reasoning was developed, and what influences shaped these technologies. By understanding these factors, AI platforms can reflect diverse values and clinical philosophies, catering to varied patient populations.
In this pivotal moment for healthcare, the emergence of AI tools that could genuinely assist patients in making informed medical decisions stands juxtaposed against the potential for these technologies to be co-opted by powerful financial interests. The pressing question is not whether AI will transform healthcare—it already is—but whether that transformation will serve the needs of patients or simply enhance corporate profits.
Patients should approach their health data with the seriousness it deserves, interrogating AI-generated advice with the same skepticism a journalist would apply to their sources. Demanding transparency from companies developing these tools is crucial. The alternative is allowing a $5 trillion healthcare industry to dictate what constitutes appropriate care—one chatbot response at a time.
