A study in Science Advances maps the landscape of innovation to identify disruptive studies and patents that challenge existing paradigms and inspire waves of follow-up research. The most widely used metric for tracking progress in science, the CD index, focuses on a paper’s local citation patterns, meaning it looks at whether papers citing a focal work also cite that work’s references. The researchers argue this narrow view misses the bigger picture, making it unreliable for detecting simultaneous discoveries.
The embedding disruptiveness measure in action. Each scientific paper is assigned two vectors based on its position in the broader citation network — a past vector representing the research it builds on and a future vector representing the research it inspires (panels A–C). For a developing paper (D), future research continues to rely heavily on the same prior work, keeping the two vectors close together. For a disruptive paper (E), future research diverges sharply from earlier foundations, pushing the two vectors apart. The greater that distance, the higher the paper’s disruptiveness score. Credit: Munjung Kim
“Science doesn’t evolve incrementally, but sometimes we see abrupt changes. Scholars are interested in when and why exactly the disruption happens,” said Sadamori Kojaku, an assistant professor of systems science and industrial engineering at Binghamton University who worked on the research. “And to do that, we need to create a metric to kind of tell scholars, ‘OK, this is the disruption happening in a given year.’”
How citation patterns miss simultaneous discoveries
On Nov. 11, 1974, two independent teams simultaneously announced the discovery of a new particle that challenged the established three-quark model, sparking what became known as the November Revolution in particle physics. Both groups published in the same issue of Physical Review Letters and cited each other. This mutual citation caused their disruption index scores to fall from the top 1% to 4% range to the bottom 0% to 1%. EDM placed both of these papers in the top 5% to 7%.
The CD index works by sorting all future papers that cite a focal paper into two buckets. One for papers that cite the focal paper but not its references, indicating that the focal paper redirected the field away from what came before. The other bucket is for papers that cite the focal paper and at least one of its references, meaning the paper is building on the existing knowledge. The disruption score is the balance between the buckets. A paper with more disruptive citations scores close to 1, while a paper with mostly consolidating citations scores close to -1.
When the 1974 papers cited each other, any future papers that cited both of them counted as a consolidating citation for the papers, not a disruptive one. That citation link between the two papers reclassified the majority of future citations from the disruptive bucket into the consolidating bucket, dramatically decreasing the papers’ scores.
How the new measure works
Using a machine-learning technique known as neural embedding, the researchers built a map of approximately 55 million scientific papers and 7.4 million patents. Each paper is represented by two points — one reflecting the research it built upon, another reflecting the research it inspired. When a paper is truly disruptive, these two points are far apart, meaning it redirects future research away from what came before it.
The new measure, called the embedding disruptiveness measure (EDM), represents each paper as two vectors: a past vector and a future vector. The past vector captures what the paper builds on, while the future vector captures what it inspires. A disruptive paper causes its descendants to diverge from its ancestors. The greater the distance between the two vectors, the more disruptive the paper.
The system can identify breakthroughs, like Nobel Prize-winning papers, but unlike other disruption indexes, it is sensitive to broader contexts and can better identify “simultaneous discoveries.” Knowing when breakthroughs occur can help us better understand the conditions that lead to disruptive moments and fuel more breakthrough science.
EDM avoids the simultaneous discovery problem by operating across the entire citation network, not just the immediate citations of a focal paper. Two simultaneous discovery papers that inspired similar bodies of follow-up research will have almost identical future vectors regardless of whether they cited each other. The broader pattern of who used the work and in what context dominates over any single citation link.
“By having more accurate metrics, we can actually investigate where the disruption is happening in the map of science,” Kojaku said. “It can have significant implications for science policy. It’s also helpful for prioritizing funding. We now have the quantitative metrics to investigate at which stage of research the disruptive work occurs and matters most.”
Finding the discoveries history missed
Among highly cited paper pairs in the American Physical Society (APS) dataset, EDM identified 64 out of 80 candidate pairs as genuine simultaneous discoveries.
The researchers acknowledged that measuring changes in disruptiveness over time would require computationally expensive retraining. Other limitations include the exclusion of papers with very few citations, the inability to detect simultaneous discoveries in isolated scholarly communities with no cross-citation, and reduced interpretability compared with simpler citation-counting approaches.
After reviewing the impact of research papers, the researchers are considering a follow-up paper focused specifically on tracing the trajectory of individual researchers.
What EDM could mean for science and innovation
The paper states that EDM could have significant implications for science policy and for prioritizing funding, specifically by providing quantitative metrics to investigate at which stage of research disruptive work occurs and matters most.
They also argue that the method could enable more robust identification of simultaneous discoveries, facilitating more accurate attribution of transformative contributions. They noted that the Nobel prize commonly honors several individuals for a single advance, yet still causes controversy over neglected contributors, and that this dynamic can cause some transformative works to be overshadowed by better-cited papers.
The authors say EDM’s ability to identify simultaneous disruptions may facilitate investigation into the drivers of innovation and recognition dynamics in the scientific community.
The method was also applied to patents and performed well, the researchers said, suggesting that EDM could have applicability beyond academic publishing.
