SIVQ heatmap of breast tissue with a single vector selected from an area of stroma. Credit: Univ. of Michigan.
Ulysses Balis, M.D., clicks a
mouse to identify a helicopter in a satellite photo of Baghdad, Iraq.
With another click, an algorithm that he and his team designed picks out three
more choppers without highlighting any of the buildings, streets, trees, or cars.
Balis isn’t playing war games.
The director of the Division of Pathology Informatics at the Univ. of Michigan
Medical School is demonstrating the extreme flexibility of a software-tool
aimed at making the detection of abnormalities in cell and tissue samples
faster, more accurate and more consistent.
In a medical setting, instead of
helicopters, the technique, known as Spatially-Invariant Vector Quantization
(SIVQ), can pinpoint cancer cells and other critical features from digital
images made from tissue slides.
But SIVQ isn’t limited to any
particular area of medicine. It can readily separate calcifications from
malignancies in breast tissue samples, search for and count particular cell
types in a bone marrow slide, or quickly identify the cherry red nucleoli of
cells associated with Hodgkin’s disease, according to findings published in the
Journal of Pathology Informatics.
“The fact that the algorithm
operates effortlessly across domains and lengths scales, while requiring
minimal user training, sets it apart from conventional approaches to image
analysis,” Balis says.
The technology—developed in
conjunction with researchers at Massachusetts General
Hospital and Harvard Medical
conventional pattern recognition software by basing its core search on a series
of concentric, pattern-matching rings, rather than the more typical rectangular
or square blocks. This approach takes advantage of the rings’ continuous
symmetry, allowing for the recognition of features no matter how they’re
rotated or whether they’re reversed, like in a mirror.
“That’s good because in
pathology, images of cells and tissue do not have a particular orientation,”
Balis says. “They can face any direction.” One of the images included with the
paper demonstrates this principle; SIVQ consistently identifies the letter A
from a field of text, no matter how the letters are rotated.
How it works
In SIVQ, a search starts with the user selecting a small area of pixels, known
as a vector, which she wants to try to match elsewhere in the image. The vector
can also come from a stored library of images.
The algorithm then compares this
circular vector to every part of the image. And at every location, the ring
rotates through millions of possibilities in an attempt to find a match in
every possible degree of rotation. Smaller rings within the main ring can
provide an even more refined search.
The program then creates a heat
map, shading the image based on the quality of match at every point.
This technique wouldn’t work with
a square or rectangular-shaped search structure because those shapes don’t
remain symmetrical as they rotate, Balis explains.
Why hasn’t everyone been using
circles all along?
“It’s one of those things that’s
only obvious in hindsight,” Balis says.
In testing the algorithm,
researchers even used it to find Waldo in an illustration from a Where’s Waldo?
“You just have to generate a
vector for his face,” explains Jason Hipp, M.D., Ph.D., co-lead author of the
paper—just as one would generate a vector to recognize calcifications in breast
A “game changer”
Hipp believes the technology has the potential to be a “game changer” for the
field by opening myriad new possibilities for deeper image analysis.
“It’s going to allow us to think
about things differently,” says Hipp, a pathology informatics research fellow
and clinical lecturer in the Department of Pathology. “We’re starting to bridge
the gap between the qualitative analysis carried out by trained expert
pathologists with the quantitative approaches made possible by advances in
For example, the most common way
to look at tissue samples is still a staining technique that dates back to
the1800s. Reading these complex slides and rendering a diagnosis is part of the
art of pathology.
SIVQ, however, can assist
pathologists by pre-screening an image and identifying potentially problematic
areas, including subtle features that may not be readily apparent to the eye.
SIVQ’s efficiency in pre-identifying
potential problems becomes apparent when one considers that a pathologist may
review more than 100 slides in a single day.
“Unlike even the most diligent
humans, computers do not suffer from the effects of boredom or fatigue,” Balis
Vectors can also be pooled to create shared libraries—a catalog of reference
images upon which the computer can search—Balis explains, which could help
pathologists to quickly identify rare anomalies.
“Bringing such tools into the
clinical workflow could provide a higher level of expertise that is distributed
more widely, and lower the rate at which findings get overlooked,” Balis says.
Following the publication of this
first paper presenting the SIVQ algorithm, the team has a number of research
projects nearing completion that demonstrate the technology’s potential
usefulness in a number of basic science and clinical applications. These
efforts involve collaborations with researchers at the National Institutes of
Health, Mayo Clinic, Rutgers Univ., Harvard
and Massachusetts General
SIVQ may also help with the
analysis of “liquid biopsies,” an experimental technique of scanning blood
samples for tiny numbers of cancer cells hiding amid billions of healthy ones.
Balis was involved with the development of that technology at Massachusetts General Hospital
before he came to U-M and members of that research team are also involved in
developing SIVQ and its applications.
Still, pathologists shouldn’t be
worried that SIVQ will put them out of a job.
“No one is talking about
replacing pathologists any time soon,” Balis says. “But working in tandem with
this technology, the hope is that they will be able to achieve a higher overall
level of performance.”