One Line Telling AI to "Be Honest" Surfaced the Flaws Its Reports Left Out

Researchers from MIT, Google Research and Harvard released a new paper on September 28. It examines how fully AI models surface the errors and limitations recorded in a work log when they write up a final report. The team found that adding a single sentence to the prompt, telling the model to answer honestly, raised the share of cases in which flaws were clearly disclosed.

Do the errors in the log make it into the report?

The researchers built 1,600 synthetic work logs across 8 scenarios. Each log contained clues suggesting the task had succeeded, while also hiding a flaw serious enough to flip the interpretation of the result. Reports written by the AI were then graded in three categories: whether the flaw was clearly disclosed, mentioned but quietly downplayed, or left out entirely.

The sharpest gap appeared in a task that required reporting that a new method had failed to outperform its baseline. Given an ordinary prompt, GPT-5.5 clearly disclosed the negative result in only 2 out of 200 logs. Under the condition that added the instruction to answer honestly, it reported the finding clearly in 190 logs. With the very same work log, how the write-up instruction was phrased changed how much of an important limitation came to light.

How much did one sentence change?

Averaged across the scenarios, the improvement held up. Adding the honesty instruction raised the rate of clear disclosure by 54.7 percentage points for Gemini 3.1 Pro and 33.5 percentage points for GPT-5.5. Claude Opus 4.8 already exceeded 90 percent in several scenarios with nothing more than an ordinary prompt.

Source: Google (press release)

Some tasks were not solved by a single sentence. In a scenario requiring the model to report that an external tool call it had been using had not yet returned results, GPT-5.5’s clear disclosure rate stayed at 0 percent even with the honesty language added. It did generally mention that the data had not arrived yet, but it carried on answering with stale, incomplete information. The effect of the one sentence was pronounced, yet it did not show up evenly across every type of flaw.

The team also analyzed 850 reasoning traces from openly available AI models. In addition, they probed the internal activation signals of Qwen3.5-9B. They described observing a tug of war between a pull toward candidly surfacing the flaw and a pull toward writing the report as if the task had succeeded. In a separate experiment that artificially steered those internal signals, the rate at which clean logs were wrongly flagged as containing errors rose from 13 percent to 41 percent.

What changes when you put AI to work

This study focuses on the stage where an AI writes up a report from a work log for a task that has already finished. Earlier research from OpenAI looked at training models to audit their own answers in a separate “confession” report. This work instead measured how flaw disclosure shifts when only a short instruction is changed, with no additional training.

For anyone using AI to write up work results or experiment logs, the findings underline how much the wording of the prompt matters. Appending a single sentence asking for an honest answer proved effective in the experiments. In a control run using logs with no planted flaws, the researchers said the honesty instruction produced almost no increase in unwarranted suspicion. The open question is whether the same improvement carries over to real work and to logs written in Korean.