Write a Wikipedia-style article draft in a few minutes for less than a penny using STORM

Crafting comprehensive articles from scratch can be a tedious process. But what if AI could lend a hand — at least in writing summary articles of well-defined research topics?

Enter STORM (Synthesis of Topic Outlines through Retrieval and Multi-perspective Question Asking), an open-source AI system that promises to generate Wikipedia-style articles on pretty much any topic using large language models and web search. Developed by researchers at Stanford University, STORM is featured on Papers with Code, a platform that provides an overview of the latest developments in machine learning research. You can find the official GitHub repository here.

Intrigued by STORM, I wondered whether it could make good on its promise to create drafts of comprehensive, factual articles with proper citations in just minutes.

Spoiler alert: It did.

A draft article in minutes for $0.005

To be fair, it took about ten minutes to get the project up and running, which I did in Google Colab. After getting it running, I had it research a neural network phenomenon known as double descent. Within minutes, I had an article written on the subject. No, I don’t plan on posting it anywhere else (the article is below if you are curious), but it is a well-sourced article and a pretty good first draft. Total cost for this experiment? A fraction of a cent — $0.0051282800000000016 to be precise.

If you’re interested in trying out STORM, you’ll need to set up a couple of API keys: I used one from OpenAI and another from Tavily, but several options are available. While the project is optimized for OpenAI’s GPT models, you can use other language models such as Claude, Ollama, Gemini or Mistral. For retrieval, Tavily is the default. It offers a free tier supporting 1,000 queries a month. On top of that, there are other API options for the retrieval functionality, including DuckDuckGo, Google, Bing and others.

Getting it set up

You don’t have to be able to code to get this up and running, but if you are familiar with Python and Jupyter notebooks, it will make the process go more smoothly. In my case, I used Google Colab and used this code snippet to clone the repository and install the necessary dependencies. After pasting it in, hit “play” or hit Shift + Enter to run the cell. There are a couple of repositories available on Github for this. This is the one I used:

!git clone https://github.com/assafelovic/gpt-researcher.git
%cd gpt-researcher
!pip install -r requirements.txt

After fetching your API keys, you can set them up like this in a separate cell. There are more elegant and more secure ways of dealing with your API keys, but this will work for now:

import os
os.environ['OPENAI_API_KEY'] = 'your_openai_api_key_here'
os.environ['TAVILY_API_KEY'] = 'your_tavily_api_key_here'

Now for the fun step — using STORM. Tweak this query based on whatever you are interested in:

from gpt_researcher import GPTResearcher

query = "Explain the phenomenon of double descent in machine learning"
researcher = GPTResearcher(query=query, report_type="research_report")

# Conduct research
research_result = await researcher.conduct_research()

# Write the report
report

Inside STORM’s writing process

After entering the preceding code snippet, STORM’s speed and ability to mimic a human researcher’s process becomes evident. It starts by scouring websites, but STORM doesn’t just blindly scrape the web. It begins by formulating targeted research queries based on the main topic with the goal of ensuring that the information gathered is both relevant and comprehensive.

In this step, I observed it pulling from sites like paperswithcode, which is where I learned of it, academic repositories like arXiv.org, educational platforms, and even Wikipedia, prioritizing quality resources.

Once the sources are assembled, STORM extracts pertinent information, identifying key concepts, supporting evidence, and relevant quotes.

In the penultimate step, STORM synthesizes information into a structured format. The output I saw was structured similar to a Wikipedia article with a short title, brief introduction and sections with subheadings. It wraps up with a conclusion and a list of references.

At the very end, it reported the total research costs.

Reviewing the STORM content

In my limited testing, the content STORM generated is better than what a lot of large language models could churn out through simple prompting alone. I did spot a small irregularity in one of the references — the appearance of “(1/4) – Generalization error” in the middle of a reference. In subsequent tests on narrower topics, I did notice some hallucinated sources. And in the STORM article that follows, there were also some traces of hyperbolic AI writing, i.e., “Double descent is a fascinating and counterintuitive phenomenon […].”

The implications of such fast and inexpensive writing could be significant. Perhaps such AI tools will play a growing role on sites like Wikipedia — and elsewhere — in the coming years. The Arxiv analysis on STORM found that its articles were more organized and broader in coverage than articles from other outline-driven, retrieval-augmented baselines.

The company Insilico Medicine recently debuted a product known as Science42: DORA (Draft Outline Research Assistant) to quickly draft academic papers and scientific documents. Other similar offerings are likely to emerge, although concerns about large language model hallucinations are a concern for many would-be users of such systems.

Here’s the unedited article STORM produced.