Methods videos reduce research variability, improve reproducibility

Traditionally, discussions about experimental reproducibility have focused on statistics, sample sizes and publication bias. However, procedural quality and training also play an important role in reducing variability. 

JoVE, a video platform for scientific content, is aiming to improve the reproducibility of research by providing detailed visual demonstrations of experimental procedures. This is important in research that involves animal testing, where variations in methods can influence physiological responses and skew data. 

Videos increase the success rate up to 90%

JoVE is a PubMed-indexed, peer-reviewed video journal with both text and video formats. Authors submit papers to JoVE, and after being peer reviewed, a video team travels to the lab to produce video content.

Written methods sections often do not effectively communicate spatial and temporal information, said Ron Myers, senior director of editorial at JoVE. This could include details like the angle of forceps or the amount of pressure applied to a scalpel. Text-based methods sections are also open to interpretation, he added, which can lead to different outcomes or even surgical failure.

According to a case study at Michigan State University, the use of JoVE video improved the student success rate for catheter implantation from 40% to 90%. The videos also reduced the learning curve for students to successfully implant a catheter for the first time from eight tries down to one. 

“If somebody is trying a new technique, they tend to feel much more confident after seeing a video and how it’s done, compared to coming into a training session cold,” said Norman Peterson, a veterinary reviewer for JoVE and other publications. Peterson’s research has focused on rodent models, primarily mice, in biologics development. 

Video content is also useful for showing logistics that are often left out of written methods sections, Peterson said, such as the layout of equipment or the timing of anesthesia administration. Written protocols provide guidance, consistency and support regulatory compliance, but may have gaps that can be more effectively addressed through visualization.

Reducing variability in animal labs 

In animal labs, factors like infection and pain can also impact reproducibility. Aseptic technique and pain control methods are two areas where errors in methodology are most commonly identified in reviews performed by Peterson. These errors can lead to variability in data. 

It doesn’t have to be a noticeable infection, Peterson said. “Even introducing perhaps 1,000 bacteria can set off certain cytokines and reactions within the body, and depending on what the investigator is looking at, that can skew results in the wrong direction.”

Pain can affect cortisol levels as well as blood pressure and other factors, which also adds variability to results. “Every animal responds to pain differently. That’s going to increase the variability in your results…Variability is our enemy, so finding the appropriate analgesics is very important,” Peterson said. 

“Not only does reducing pain help science, but it’s humane,” Peterson said, “It’s the right thing to do.”