Artificial intelligence is increasingly being integrated into healthcare, helping to analyze images, make diagnoses, and predict disease progression. For instance, AI is already capable of creating cancer treatment plans on par with current medical standards. However, a new study has revealed a worrying trend: many developers of such systems are poorly acquainted with safety requirements for medical AI, reports infohub.kz.

Researchers from Nanyang Technological University surveyed 122 medical AI developers from Singapore, China, Hong Kong, the UK, and other regions. The results were published in the journal npj Digital Medicine.

It turned out that only 57% of participants were aware of at least one regulatory framework for neural networks, such as the European AI Act or Singapore's guidelines for using AI in healthcare. Moreover, nearly two-thirds of developers work in organizations where no regulatory requirements for medical AI development are applied at all.

Experienced developers and employees of commercial companies had the best understanding of regulatory requirements. Young professionals and academics were much less likely to be familiar with the rules.

Despite the findings, most participants believe that responsibility for the operation of medical AI should rest not only with government bodies but also with developers themselves. In their view, system users, ethics committees, and organizations providing data for algorithm training should also be involved in ensuring safety.

The study authors note that developers know their models better than anyone else. They understand the quality of training data and the circumstances under which AI can produce unreliable outputs. If developers are unfamiliar with safety requirements, this could lead to errors when implementing such technologies in clinical practice, posing risks to patients and undermining trust in medical AI.

According to the authors, the safe integration of artificial intelligence into healthcare requires enhancing developers' knowledge of the rules that these technologies must comply with.