SAN FRANCISCO (KCBS RADIO) – A new study out of UCSF indicates that artificial intelligence (AI) language models could help triage patients in the emergency room. It was published May 7 in the JAMA Network Open journal .
For more, stream KCBS Radio now .
It is one of the first studies to use real data.
“We gave the AI model pairs of patients – one patient had a more serious condition, such as a stroke or a heart attack and so forth, and the other had a less serious condition, such as maybe a broken wrist or a broken toe,” said Christopher Williams , a postdoctoral scholar at UCSF and lead researcher on the study. He spoke with KCBS Radio’s Patti Reising for the latest episode of “As Prescribed”.
He said his team used the Chat GPT-4 large language model. According to a press release from UCSF , de-identified patient data was used and the language model was accessed via UCSF’s secure generative AI platform, which has broad privacy protections.
The language model used for this research is trained to predict the next word in a sentence, and then to create full sentences as it is trained on large datasets. Williams said they chose it in part because ChatGPT is one of the more popular AI models in use today.
“Using only the patient’s symptoms, the AI model was able to accurately identify which patient in the pair had a more serious condition at 89% of the time,” he said. “And we found that the AI was just as good at doing this as a physician.”
While Williams made it clear that implementation of AI in clinical settings is far off, he said the research is an important step towards creating a groundwork of knowledge about AI in medicine. Williams also told Reising that his team decided to use a language learning model because ER doctors must triage just off notes on talks with patients.
“If you’re looking at the notes written by doctors… as someone who’s contributed to some of these notes myself, I know that I’m not as clean in my writing as I can be because time is – is limited, and therefore we often will be abbreviating our notes and, trying to use a bunch of shorthand,” he said. “And so just in, in the initial stage, trying to work out, well, how well can a language model, be applied to the shorthand type notes of doctors is certainly very interesting.”
As AI becomes more popular, Williams said research such as this study from UCSF is important.
“I think from a purely clinical perspective, there are several, kind of, issues at stake,” he said. “Firstly, we need to understand what the limitations of these models and in particular things like bias within the models, because ultimately going back to, you know, your first kind of principles of medicine do no harm. It’s of paramount importance that we, you know, deploy models which are safe and, and equitable, and therefore it's really, an imperative that we can understand what are the limitations of these types of models.”
Listen to this week’s “As Prescribed” to learn more. You can also listen to last week’s episode about why going to a female doctor might be better for your health here .
DOWNLOAD the Audacy App
SIGN UP and follow KCBS Radio
Facebook | Twitter | Instagram
"As Prescribed" is sponsored by UCSF.
Most Popular
Comments / 0