Simplifying Semantic Image Annotation and Navigation
 |
| Courtesy of IMAGINATION |
Semantic image annotation and search, a technology that links the content of photos to concepts so as to make the images understandable by computers, has typically required end users to use a manually developed ontology — a lexicon of predefined concepts used to assign machine-readable semantic meaning to information — and then train the software to correctly annotate different images. For example, an apple would need to be defined in an ontology for fruit, and then photos of apple trees could be tagged as such.
However, an innovative new system, known as ImageNotion, strips away much of that complexity for the end user, combining semantic annotation with a variety of other technologies, from text mining and object recognition to face detection and face identification, in order to permit many more images to be accurately annotated with little or no user intervention.
“When you mention ontologies to most people they just switch off. A photographer, an image agency employee or a Web user doesn’t want and shouldn’t have to learn how the technology works, they just want to be able to use it,” explains Gabor Nagypal, who oversaw development of the ImageNotion software as technical and scientific coordinator of the EU-funded IMAGINATION project. “Because of that, our goal has been to make the technology transparent and intuitive to use,” he adds.
The incorporation of other technologies, such as face detection and recognition, means that considerably less has to be done by the user to ensure photos are tagged correctly. The face detection system, which identifies the presence of people in a photograph, functions with around 80 percent accuracy, Nagypal says. Meanwhile, face recognition, which seeks to identify who those people are — a trickier technology to perfect — starts to produce consistent results after five to 10 images of a person have been tagged.
Another key innovation is that the creation of ontology elements, which the IMAGINATION team refers to as “imagenotions,” is an integral part of the software, instead of being dealt with as a separate process. Imagenotions can, therefore, easily be changed and updated by end users at any time.
“If they realize that a notion does not exist for a particular person or object, they can easily add it to the system. For example, the system would automatically identify the first photo it comes across of Angela Merkel as a person and a woman. The user could then define her profession as German chancellor and state that the photo was taken at a certain EU meeting. Those terms could then, with a few clicks, be used to pull up a list of other photos from the meeting and connect to information about other people who were present,” Nagypal explains.
In the case of some photos, such as already archived images, text mining software extracts much of that information from the caption and draws connections with the output of the face detection and recognition systems automatically. Navigating between images is simplified by a system of embedded links, so that clicking on the face of Chancellor Merkel will take the user directly to a display of other images of her.
The software is currently being tested by the French photo agency Photo12 and the Italian State Library of Modern and Contemporary History, both IMAGINATION project partners, as well as by several German photographers. A Web-based beta version also is available for testing on the Web site of ImageNotion, a spinoff company set up by some project members to start commercializing the software.
“We are currently refining the software, but we hope to be ready to start selling it as a stand-alone or web-based product next year. First, we will target professional users, primarily photo agencies, archives and libraries who need a quicker and easier way to annotate and find images, then we will look to provide it as a software service for individual photographers and, finally, as an embedded application, which could be used on other Web sites such as Facebook or Flickr,” Nagypal explains.
“We are looking for investors and partners to help us get it off the ground,” he adds.
IMAGINATION received funding from the ICT strand of the EU’s Sixth Framework Programme for research.
Courtesy of ICT Results
