Online crowd-sourcing—in which a task is
presented to the public, who respond, for free, with various solutions and
suggestions—has been used to evaluate potential consumer products, develop
software algorithms, and solve vexing research and development challenges. But
diagnosing infectious diseases?
Working on the assumption that large groups
of public non-experts can be trained to recognize infectious diseases with the
accuracy of trained pathologists, researchers from the University of California,
Los Angeles (UCLA) Henry Samueli School of Engineering and Applied Science and
the David Geffen School of Medicine at UCLA have created a crowd-sourced online
gaming system in which players distinguish malaria-infected red blood cells
from healthy ones by viewing digital images obtained from microscopes.
The UCLA team found that a small group of
non-experts playing the game (mostly undergraduate student volunteers) was
collectively able to diagnosis malaria-infected red blood cells with an accuracy
that was within 1.25% of the diagnostic decisions made by a trained medical
The game, which can be accessed on cell
phones and personal computers, can be played by anyone around the world,
“The idea is, if you carefully combine
the decisions of people—even non-experts—they become very competitive,”
said Aydogan Ozcan, an associate professor of electrical engineering and
bioengineering and the corresponding author of the crowd-sourcing research.
“Also, if you just look at one person’s response, it may be OK, but that
one person will inevitably make some mistakes. But if you combine 10 to 20,
maybe 50 non-expert gamers together, you improve your accuracy greatly in terms
Crowd-sourcing, the UCLA researchers say,
could potentially help overcome limitations in the diagnosis of malaria, which
affects some 210 million people annually worldwide and accounts for 20% of all
childhood deaths in sub-Saharan Africa and
almost 40% of all hospitalizations throughout that continent.
The current gold standard for malaria
diagnosis involves a trained pathologist using a conventional light microscope
to view images of cells and count the number of malaria-causing parasites. The
process is very time consuming, and given the large number of cases in
resource-poor countries, the sheer volume presents a big challenge. In
addition, a significant portion of cases reported in sub-Sahara Africa are actually false positives, leading to
unnecessary and costly treatments and hospitalizations.
By training hundreds, and perhaps thousands,
of members of the public to identify malaria through UCLA’s crowd-sourced game,
a much greater number of diagnoses could be made more quickly—at no cost and
with a high degree of collective accuracy.
“The idea is to use crowds to get
collectively better in pathologic analysis of microscopic images, which could
be applicable to various telemedicine problems,” said Sam Mavandadi, a
postdoctoral scholar in Ozcan’s research group and the study’s first author.
Ozcan and Mavandadi emphasized that the same
platform could be applied to combine the decisions of minimally trained health
care workers to significantly boost the accuracy of diagnosis, which is
especially promising for telepathology, among other telemedicine fields.
The new UCLA study, “Distributed
Medical Image Analysis and Diagnosis Through Crowd-Sourced Games,” has
been accepted for publication in PLoS ONE.
In addition to developing the crowd-sourced
gaming platform that allows players to assist in identifying malaria in cells
imaged under a light microscope, Ozcan’s research group created an automated
algorithm for diagnosing the same images using computer vision, as well as a
novel hybrid platform for combining human and machine resources toward
efficient, accurate and remote diagnosis of malaria.
“The most exciting aspect is that this
is an entirely novel approach in the area of visual diagnostics, which really
challenges diagnostic algorithms used to date,” said Karin Nielsen, a
professor of infectious diseases in the department of pediatrics at the Geffen
School of Medicine. “It is diagnostics outside the box—that is, the study
introduces an entirely new concept in diagnostics with the use of games for
this purpose. The potential applications of this new approach are
How the game works:
playing the game, each player is given a brief online tutorial and an
explanation of what malaria-infected red blood cells typically look like using
sample images. After completing a short training phase, players go through the
actual game, in which they are presented with multiple frames of red blood cell
images and can use a “syringe” tool to “kill” the infected
cells one-by-one and use a “collect-all” tool to designate the
remaining cells in the frame as “healthy.”
Within each frame, there are a certain
number of cells whose status (i.e., infected or not) is known by the game but
not by the players. These control cell images allow Ozcan’s team to dynamically
estimate the performance of gamers as they go through each frame and also helps
the team assign a score for every frame the gamer passes through.
“I believe that, similar to other very
innovative ideas, one of the major challenges will be the skepticism of
traditional microscopists, pathologists and clinical laboratory personnel, not
to mention malaria experts, who will initially view with suspicion a gaming
approach in malaria diagnostics,” said Nielsen, also an author of the
study. “It is a very revolutionary proposal and it might take a few
clinical studies in the field to document the efficacy of this platform in order
to convince traditional microbiologists and other infectious disease
“Scaling up accurate, automated and
remote diagnosis of malaria through a crowd-sourced gaming platform may lead to
significant changes for developing countries,” Ozcan said.
“It could eliminate the current overuse
and misuse of anti-malarial drugs, improve management of non-malaria fevers by
ruling malaria out, lead to better use of existing funds, and reduce risks due
to long-term side effects of antimalarial drugs on patients who don’t need
treatment,” Mavandadi added.
Ozcan’s team hopes to bring the platform
into the field through clinical trials to help validate its use and facilitate
implementation of the technology worldwide. Nielsen and Ozcan plan to implement
it at clinical sites in countries such as Mozambique,
Malawi, and Brazil.
In addition, the same crowd-sourcing and
gaming-based microanalysis and medical diagnosis platform could be further
scaled up for a variety of other biomedical and environmental applications in
which microscopic images need to be examined by experts, the researchers said.