I think there are some techbros out there with sleazy legal counsel that promises they can drench the thing in enough terms and conditions to relieve themselves of liability, similar to the way that WebMD does. Also, with healthcare access the way it is in America, there are plenty of people who will skim right past the disclaimer telling them to go see a real healthcare provider and just trust the “AI”. Additionally, there’s enough slimy NP professional groups pushing for unsupervised practice that they could just sign on their NP licenses for prescriptions, and the malpractice laws currently in place would be difficult to enforce depending on outcomes and jurisdictions.

This doesn’t get into the sowing of discord and discontent with physicians that is happening even without these products existing in the first place. Even the claims that an AI could potentially, maybe, someday sorta-kinda replace physicians makes people distrust and dislike physicians now.

Separately, I have some gullible classmates in medical school that I worry about quite a lot, because they’ve bought into the line that chat GPT passed the boards, so they take its’ hallucinations as gospel and argue with our professor’s explanations as to why the hallucination is wrong and the correct answer on a test is correct. I was not shy about admonishing them and forcefully explaining how these “generative AIs” are little more than glorified text predictors, but the allure of easy answers without having to dig for them and understand complex underlying principles is very alluring, so I don’t know if I actually got through to him or not.

There are way too many techbros trying to push the idea of turning chat gpt into a physician replacement. After it “passed” the board exams, they immediately started hollering about how physicians are outdated and too expensive and we can just replace them with AI. What that ignores is the fact that the board exam is multiple choice and a massive portion of medical student evaluation is on the “art” side of medicine that involves taking the history and performing the physical exam that the question stem provides for the multiple choice questions.

I once had ideas about building a machine learning program to assist workflows in Emergency Departments, and its’ training data would be entirely generated by the specific ER it’s deployed in. Because of differences in populations, the data is not always readily transferable between departments.

People just need to understand that the true medical uses are as tools for physicians, not “replacements” for physicians.