Ai2 Releases an AI That Writes Science Reports, With Drafts in 51 Seconds

On October 2, Ai2 released AstaBrief 8B, an AI model that writes reports grounded in scientific papers. Given a research question and excerpts retrieved from papers, the model produces a report with citations attached. It powers the Fast mode of the report-writing feature in Asta, Ai2’s AI platform for scientific research. The model and its training data were released as well, so anyone can download and run it themselves.

How fast does a report come out?

On average, a single report took less than a minute. Ai2 says it measured Asta’s full processing pipeline and found that Fast mode spent an average of 51.1 seconds per report. Thinking mode, which runs on Claude, took 178.5 seconds. That makes Fast mode roughly 3.5 times quicker. Ai2 explained that the goal is to put a draft in a researcher’s hands quickly, so they can keep asking follow-up questions and refine the content from there.

To gain that speed, the order of operations for building a report was changed too. Thinking mode summarizes paper excerpts, groups them, and then writes the report section by section. AstaBrief takes the question along with the relevant excerpts and writes the entire report in one pass. Ai2 says performance held up even with those intermediate steps removed.

How was it taught to cite sources?

AstaBrief is built on Qwen3-8B, with additional training on how to write scientific reports. The training started from questions real researchers had actually asked. Ai2 filtered its user logs to remove questions containing personal information and requests unrelated to science, leaving 90,000 research questions. Non-English questions were also excluded at this stage. After generating sample reports with several models and filtering for quality, 47,000 training examples remained.

Image: huggingface.co

The team also trained the model to distinguish better reports from worse ones. For each question, two reports were evaluated separately by GPT-4.1 and DeepSeek-R1, and only the cases where both models picked the same side were kept. That yielded about 6,000 comparison examples. Ai2 said it verified that the judge models’ verdicts matched human preferences 95 percent of the time. The design deliberately avoids leaning on the answers or judgments of any single model.

A fast draft, but is the evidence trustworthy?

The main performance tests used 200 computer science research questions written by actual users. The evaluation looked at how much of the necessary content each report covered and whether each paragraph was relevant to the question. Whether cited papers genuinely supported the claims attributed to them, and whether every claim in a report carried sufficient evidence, were assessed separately. Ai2 noted that most of the training and evaluation was completed in 2025, and that it has not rerun the full evaluation against today’s newest models.

Image: huggingface.co

Ai2 pointed out that attaching citations alone is not enough to convey research findings accurately. The problems it cites include stretching a result from a specific sample into a claim about an entire population, or turning a study that merely describes a phenomenon into a behavioral recommendation. This round of development primarily measured relevance to the question, coverage of the content, and the grounding of citations. Ai2 said deeper evaluation is still needed on whether reports preserve the scope and strength of the claims the research actually supports.

For institutions that want to run it themselves, Ai2 also released an example workflow. With some adaptation, it can generate reports from an organization’s own PDFs. Ai2 explained that running the system on in-house infrastructure becomes necessary when questions involve sensitive or unpublished research. What matters next is both drafting speed and how faithfully these reports preserve the conclusions of the original research.