A research group led by scientists from the RIKEN Center for Advanced Intelligence Project (AIP) have developed a novel system that can automatically detect abnormalities in fetal hearts in real-time using artificial intelligence (AI). This technology could help examiners to avoid missing severe and complex congenital heart abnormalities that require prompt treatments, leading to early diagnosis and well-planned treatment plans, and could contribute to the development of perinatal or neonatal medicine.
Congenital heart problems — which can involve abnormalities of the atrium, ventricle, valves or blood vessel connections — can be very serious, and account for about 20% of all newborn deaths. Diagnosis of such problems before the baby is born, allowing for prompt treatment within a week after birth, is known to markedly improve the prognosis, so there have been many attempts to develop technology to enables accurate and rapid diagnosis. However, today, fetal diagnosis depends heavily on observations by experienced examiners using ultrasound imaging, so it is unfortunately not uncommon for children to be born without having been properly diagnosed.
In recent years, machine learning techniques such as deep learning have been developing rapidly, and there is great interest in the adoption of machine learning for medical applications. Machine learning can allow diagnostic systems to detect diseases more rapidly and accurately than human beings, but this requires the availability of adequate datasets on normal and abnormal subjects for a certain disease. Unfortunately, however, since congenital heart problems in children are relatively rare, there are no complete datasets, and up until now, prediction based on machine learning was not accurate enough for practical use in the clinic. However, the RIKEN AIP-led group, which also involves collaborators from Fujitsu Ltd. and Showa University, decided to take on this challenge, and has successfully developed new machine learning technology that can accurately predict diseases using relatively small and incomplete datasets.
In general, experts of fetal heart diagnosis seek to find whether certain parts of the heart, such as valves and blood vessels, are in incorrect positions, by comparing normal and abnormal fetal heart images based on their own judgement. The researchers found that this process is similar to the “object detection” technique, which allows AIs to distinguish the position and classify multiple objects appearing in images.
A set of “teacher” data — meaning data from which the AI is to learn — is prepared through “annotation” — the attachment of meanings of objects — and used to train the object detection system. To develop the current system, the researchers used normal heart images to annotate correct positions of 18 different parts of the heart and peripheral organs, and developed a novel “Fetal Heart Screening System,” which allows the automatic detection of heart abnormalities from ultrasound images. When there are differences between the test and learned data, the system judges that there is an abnormality if the difference is greater than some confidence value. The process is quick and can be performed in real-time, with the results appearing immediately on the examination screen. The system can also help harmonize diagnoses among different hospitals with different levels of medical expertise or equipment.
“This breakthrough was possible thanks to the accumulated discussions among the experts on machine learning and fetal heart diagnosis. RIKEN AIP has many AI experts and opportunities for collaboration like this project. We hope that the system will go into wide-spread use by means of the successful cooperation among clinicians, academia, and the company,” says Masaaki Komatsu, a RIKEN AIP researcher who led the project.
The researchers now plan to carry out clinical trials at university hospitals in Japan, adding larger number of fetal ultrasound images to allow the AI to learn more in order to improve the screening accuracy and expand its target. Implementing this system could help correct medical disparities between regions through the training of examiners or by remote diagnosis using cloud-based systems.
