Research & Development World

  • R&D World Home
  • Topics
    • Aerospace
    • Automotive
    • Biotech
    • Careers
    • Chemistry
    • Environment
    • Energy
    • Life Science
    • Material Science
    • R&D Management
    • Physics
  • Technology
    • 3D Printing
    • A.I./Robotics
    • Software
    • Battery Technology
    • Controlled Environments
      • Cleanrooms
      • Graphene
      • Lasers
      • Regulations/Standards
      • Sensors
    • Imaging
    • Nanotechnology
    • Scientific Computing
      • Big Data
      • HPC/Supercomputing
      • Informatics
      • Security
    • Semiconductors
  • R&D Market Pulse
  • R&D 100
    • Call for Nominations: The 2025 R&D 100 Awards
    • R&D 100 Awards Event
    • R&D 100 Submissions
    • Winner Archive
    • Explore the 2024 R&D 100 award winners and finalists
  • Resources
    • Research Reports
    • Digital Issues
    • R&D Index
    • Subscribe
    • Video
    • Webinars
  • Global Funding Forecast
  • Top Labs
  • Advertise
  • SUBSCRIBE

Video-Processing Algorithm Amplifies Small Motions in Large Motions

By R&D Editors | June 17, 2015

A key to the new motion-magnification algorithm is a technique for very precisely   extracting foreground objects from their backgrounds. Courtesy of the researchersFor several years now, the research groups of MIT professors of computer science and engineering William Freeman and Frédo Durand have been investigating techniques for amplifying movements captured by video but indiscernible to the human eye. Versions of their algorithms can make the human pulse visible and even recover intelligible speech from the vibrations of objects filmed through soundproof glass.

Earlier this month, at the Computer Vision and Pattern Recognition conference, Freeman, Durand, and colleagues at the Qatar Computing Research Institute (QCRI) presented a new version of the algorithm that can amplify small motions even when they’re contained within objects executing large motions. So, for instance, it could make visible the precise sequence of muscle contractions in the arms of a baseball player swinging the bat, or in the legs of a soccer player taking a corner kick.

“The previous version of the algorithm assumed everything was small in the video,” Durand says. “Now we want to be able magnify small motions that are hidden within large motions. The basic idea is to try to cancel the large motion and go back to the previous situation.”

Canceling the large motion means determining which pixels of successive frames of video belong to a moving object and which belong to the background. As Durand explains, that problem becomes particularly acute at the object’s boundaries.

If a digital camera captures an image of, say, a red object against a blue background, some of its photosensors will register red light, and some will register blue. But the sensors corresponding to the object’s boundaries may in fact receive light from both foreground and background, so they’ll register varying shades of purple.


See the MIT researchers’ algorithms amplify movements of a shaky gate captured on video. Courtesy of the researchers

Ordinarily, an algorithm separating foreground from background could probably get away with keeping those borderline pixels: A human viewer probably wouldn’t notice a tiny fringe of purple around a red object. But the purpose of the MIT researchers’ motion amplification algorithm is precisely to detect variations invisible to the naked eye. Changes of color at an object’s boundaries could be interpreted as motions requiring magnification.

So Durand, Freeman, and Mohamed Elgharib and Mohamed Hefeeda of QCRI instead assign each boundary pixel a weight, corresponding to the likelihood that it belongs to the foreground object. In the example of the red object against a blue background, that weight would simply depend on whether the shade of purple is bluer or redder. Then, on the basis of the pixels’ weights, the algorithm randomly discards some and keeps others. On average, it will make the right decision, and it will disrupt any patterns of color change that could be mistaken for motion.

The problem of identifying the same object from frame to frame, Durand says, is related to the problem of image stabilization, which attempts to remove camera jitter from video. Identifying the motion of a single object, however, is more difficult than determining the motion of the image as a whole.

The MIT and QCRI researchers make a few assumptions to render the problem more tractable. First, they assume a correlation between the direction and rate of motion of adjacent pixels. Second, they assume “smoothness” — that the direction and rate of motion will be consistent over time. Finally, they assume that pixels’ trajectories across frames can be captured by linear mathematical relationships, which enables their algorithm to analyze pixels individually.

Then, rather than looking for correlations between one frame and the next, their algorithm considers five frames at a time, using consistencies across frames to resolve ambiguities between adjacent frames.

Once the algorithm has identified the pixels correlating to a single moving object, it corrects for the object’s motion and performs the same motion magnification procedure that previous versions did. Finally, it reinserts the magnified motions back into the original video stream.

Related Articles Read More >

Why IBM predicts quantum advantage within two years
Aardvark AI forecasts rival supercomputer simulations while using over 99.9% less compute
This week in AI research: Latest Insilico Medicine drug enters the clinic, a $0.55/M token model R1 rivals OpenAI’s $60 flagship, and more
How the startup ALAFIA Supercomputers is deploying on-prem AI for medical research and clinical care
rd newsletter
EXPAND YOUR KNOWLEDGE AND STAY CONNECTED
Get the latest info on technologies, trends, and strategies in Research & Development.
RD 25 Power Index

R&D World Digital Issues

Fall 2024 issue

Browse the most current issue of R&D World and back issues in an easy to use high quality format. Clip, share and download with the leading R&D magazine today.

Research & Development World
  • Subscribe to R&D World Magazine
  • Enews Sign Up
  • Contact Us
  • About Us
  • Drug Discovery & Development
  • Pharmaceutical Processing
  • Global Funding Forecast

Copyright © 2025 WTWH Media LLC. All Rights Reserved. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of WTWH Media
Privacy Policy | Advertising | About Us

Search R&D World

  • R&D World Home
  • Topics
    • Aerospace
    • Automotive
    • Biotech
    • Careers
    • Chemistry
    • Environment
    • Energy
    • Life Science
    • Material Science
    • R&D Management
    • Physics
  • Technology
    • 3D Printing
    • A.I./Robotics
    • Software
    • Battery Technology
    • Controlled Environments
      • Cleanrooms
      • Graphene
      • Lasers
      • Regulations/Standards
      • Sensors
    • Imaging
    • Nanotechnology
    • Scientific Computing
      • Big Data
      • HPC/Supercomputing
      • Informatics
      • Security
    • Semiconductors
  • R&D Market Pulse
  • R&D 100
    • Call for Nominations: The 2025 R&D 100 Awards
    • R&D 100 Awards Event
    • R&D 100 Submissions
    • Winner Archive
    • Explore the 2024 R&D 100 award winners and finalists
  • Resources
    • Research Reports
    • Digital Issues
    • R&D Index
    • Subscribe
    • Video
    • Webinars
  • Global Funding Forecast
  • Top Labs
  • Advertise
  • SUBSCRIBE