<body><div><figure><img src="https://img.particlenews.com/image.php?type=thumbnail_580x000&amp;url=45cyCN_0wPrufzO00" alt="https://img.particlenews.com/image.php?url=45cyCN_0wPrufzO00" width="1067" height="712" ><figcaption>Hospitals appear to be teaming up with companies to experiment with AI (Image: Getty)</figcaption></figure>There are fears about the impact of AI used in hospitals due to a significant malfunction, according to researchers.OpenAI's Whisper tool claims to have near "human level robustness and accuracy," yet it has been found to make up things never said, according to an Associated Press investigation published Sunday with witness testimony from more than a dozen software engineers, developers and academic researchers.The experts said some of the invented text, referred to as hallucinations among industry folk, have included racial commentary, violent rhetoric and even imagined medical treatments.<ul><li>'Brain dead' chef, 28, become first person to recover from 'locked in syndrome'</li></ul>Whisper is used across various industries to translate and transcribe interviews and interactions as well as generate text for popular consumer technology and create subtitles for videos.However, the experts are most concerned about how it is used in some medical centers to transcribe patient consultations with doctors and warn it should not be used in "high-risk domains."<figure><img src="https://img.particlenews.com/image.php?type=thumbnail_580x000&amp;url=0WUxsQ_0wPrufzO00" alt="https://img.particlenews.com/image.php?url=0WUxsQ_0wPrufzO00" width="1070" height="712" ><figcaption>OpenAI's Whisper tool has been found to make up sentences (Image: Getty)</figcaption></figure>The extent of the problem is unclear, though researchers and engineers say hallucinations are happening frequently.A University of Michigan researcher, for example, said he found hallucinations in eight out of every 10 audio transcriptions.Another machine learning engineer said there were hallucinations in about half of the over 100 hours of Whisper transcriptions he reviewed.A third developer said he found hallucinations in nearly every one of the 26,000 transcripts he created with Whisper.DON'T MISS...Musk sues OpenAI and CEO Altman - claims company abandoned non-profit missionAI expert sounds alarm as 'fast and unpredictable' tech threatens 'catastrophe'OpenAI warned of AI breakthrough that 'threatens humanity' before firing CEOAnother recent study by computer scientists uncovered 187 hallucinations in more than 13,000 clear audio snippets they examined, which would lead to tens of thousands of faulty transcriptions over millions of recordings, according to the researchers.Alondra Nelson, who led the White House Office of Science and Technology Policy for the Biden administration until last year, warned of "really grave consequences.""Nobody wants a misdiagnosis," said Nelson, a professor at the Institute for Advanced Study in Princeton, New Jersey. "There should be a higher bar."The prevalence of such hallucinations has led experts, advocates and former OpenAI employees to call for the federal government to consider AI regulations. At minimum, they said, OpenAI needs to address the flaw.“This seems solvable if the company is willing to prioritize it,” said William Saunders, a San Francisco-based research engineer who quit OpenAI in February over concerns with the company’s direction. “It’s problematic if you put this out there and people are overconfident about what it can do and integrate it into all these other systems.”An OpenAI spokesperson said the company continually studies how to reduce hallucinations and appreciated the researchers’ findings, adding that OpenAI incorporates feedback in model updates.While most developers assume that transcription tools misspell words or make other errors, engineers and researchers said they had never seen another AI-powered transcription tool hallucinate as much as Whisper.</div></body>

Mountain View, CA

NewsBreak

OpenAI tool used in hospitals is 'making up treatments,' sparking concern from scientists

There are fears about the impact of AI used in hospitals due to a significant malfunction, according to researchers. OpenAI's Whisper tool claims to have near \"human level robustness and accuracy,\" yet it has been found to make up things never said, according to an Associated Press investigation published Sunday with witness testimony from more than a dozen software engineers, developers and academic researchers.

There are fears about the impact of AI used in hospitals due to a significant malfunction, according to researchers. OpenAI's Whisper tool claims to have near "human level robustness and accuracy," yet it has been found to make up things never said, according to an Associated Press investigation published Sunday with witness testimony from more than a dozen software engineers, developers and academic researchers.