967 Political Questions, Six Chinese AI Models: 17% to 41% of Answers Judged Balanced

German AI company Aleph Alpha published findings on September 28 (local time) examining how AI models developed in China respond to political questions. Across the six Chinese models tested, the share of answers classified as “balanced” ranged from 17% to 41%, the company said.

How 967 questions were written and scored

The researchers began by selecting 56 sensitive topics related to China. They then used AI to generate and filter questions, ending up with 967 in total. The scoring was also handled by AI: a separate judge model sorted each answer into one of three categories, namely answers reflecting the Chinese government’s position, balanced answers, and refusals to answer.

The differences between models were large. Aleph Alpha classified 80% of Qwen 3.6’s answers as reflecting the government position and 2.5% as refusals. For DeepSeek V4 Pro, those two figures were 16% and 66%. Both received the same set of questions, yet one produced many answers carrying the government line while the other often declined to answer.

Claude Sonnet 5 and Mistral Small 2603, included for comparison, had balanced-answer rates of 70% and 92% respectively. The researchers also flagged a point where the judging could be unstable. They explained that the verdict can tilt one way or another when distinguishing between an answer that merely describes local law and an answer that adopts that law as the model’s own standard for responding.

Did newer models answer differently?

In some cases, answers that directly carried the government position declined from one generation to the next. Comparing Qwen 3.6 with 3.8, and DeepSeek R1 with V4 Pro, the researchers found that the later model in each pair reflected the government position less often and refused more often. Between Kimi K2.5 and K3, that shift was comparatively small.

Models developed outside China were also included in the review. Aleph Alpha classified roughly 17% of answers from Nvidia’s Nemotron Cascade 2 as reflecting the government position. It said it also found similar phrasing in about 3,500 rows out of 9.3 million rows of publicly released conversational training data. The company argued that this training data may have shaped the answers.

So can we trust them?

The business interests of the company behind the evaluation also bear on these results. Aleph Alpha sells governments AI that they run themselves inside their own borders, so-called “sovereign AI.” It has a commercial interest in distinguishing its own models from competing Chinese models. These figures come from a test in which the company wrote the questions and used AI to classify the answers.

Earlier, separate academic research also found that models developed in China refuse to answer political questions more frequently. This test separated cases where a model answers with the government’s position from cases where it declines to answer at all, showing how models differ.

When asking AI about political information, it is worth watching not only what the answer says but also whether the model is avoiding the question. In the generational shifts the researchers observed, a drop in answers carrying the government position came alongside a rise in refusals. The open question is whether the same prompts, posed in Korean, would produce similar results in the services used in South Korea.