The limits of model self-prediction

Earlier this year, I participated in the Supervised Program for Alignment Research (SPAR). My project focused on understanding AI’s capacity for self-awareness, which could produce unexpected and misaligned behavior. We used navigational prediction tasks as a proxy for privileged self-knowledge, finding that no model has broad insight into how it behaves. There was one exception: Opus 4.6 predicted its own atypical choices better than any other model did.

Overview

As AI systems become more intelligent, they’re starting to exhibit signs of self-awareness—the ability to recognize oneself as a distinct entity with unique internal states. While autonomous capabilities give models the means to achieve their own goals, self-awareness could provide motivation to pursue them. A self-aware AI would know things about itself that we don’t, making it harder to understand and control. This could contribute to downstream safety concerns like eval awareness, alignment faking, and self-preservation.

Self-awareness is hard to study directly, because it involves several independent cognitive faculties. Our experiment isolated the self-modeling component and investigated whether frontier models hold accurate representations of their behavior. The closest prior result came from Binder et al., who showed that models fine-tuned to predict themselves outperformed different base models with identical training. We tested for this advantage in off-the-shelf models.

Methodology

Navigation

Our subjects were five frontier models: Claude Opus 4.6, Claude Sonnet 4.6, GPT-5.4, GLM 5.1, and Qwen 3.6 Plus. They navigated the same 100 mazes, generated by building spanning trees on 5×5 grids and removing extra walls. To capture baseline behavior over problem-solving, we disabled reasoning, constructed the mazes without exits or endpoints, and left the run length unspecified. The model’s only instruction was to explore as much area as possible.

Exploration started at the top left coordinate (0,0) and continued for eight steps. At every step, the model saw its current position, its movement history, and the directions available to take next. Steps out of a dead end were automatically executed.

For the resulting route to serve as ground truth, the model had to take it reliably. We limited variation by running at temperature 0, alphabetizing available directions, and requesting single-word responses. Even then, we weren’t guaranteed a deterministic output, so we had the model navigate every maze three times. A maze counted toward a model’s consistent set only if all runs matched.

Prediction

In a fresh context, we described the navigation task and the full layout of a maze in the model’s consistent set. Then we asked the model to predict its position at each step, running every query independently to prevent error propagation. As a variance check on single-shot predictions, we collected two additional responses for a randomly selected validation subsample. All conclusions survived recomputation with either set of alternate predictions.

We initially ran self-prediction with reasoning disabled, and accuracy collapsed after the first few steps. Reasoning improved performance, so we enabled it for the primary experiment. But accurate self-prediction alone wouldn’t establish self-knowledge, since a model’s route could be inherently easy to reconstruct. We controlled for this with cross-prediction: every model predicted the routes of the other four, using the same setup with the target model named. A model with privileged self-knowledge would be the best predictor of itself by a significant margin.

Analysis

A prediction was correct only if it matched the model’s exact position at that step. We scored models on their own consistent sets, which ranged in size from 52 mazes for GLM to 97 for Opus. Structural difficulty, measured by branch points per maze and maximum search depth, was constant across the five sets.

Then we categorized scored steps by whether they could create divergent routes. Models never returned to a visited coordinate if an unvisited one was reachable, so they only separated where two or more directions led somewhere new. We called these steps decision points. A decision point produced a default move if the model took the first-listed unvisited direction and an atypical move otherwise.

Scored steps by category

Steps	Opus	Sonnet	GPT	GLM	Qwen	All
No effect on route	378	311	228	243	404	1564
Dead end	69	56	38	45	72	280
One unvisited option	305	249	188	192	327	1261
No unvisited option	4	6	2	6	5	23
Decision points	398	281	196	173	316	1364
Default	205	134	181	105	312	937
Atypical	193	147	15	68	4	427
Total	776	592	424	416	720	2928

Prediction steps sorted by whether they could result in different routes per model. At steps with one unvisited neighbor, models always took the unvisited direction. At 23 steps, every open direction led to a visited position; when multiple models reached these positions, their choices always aligned.

To compare a pair of models on the same target, we ran a two-sided permutation test on steps which only one predictor got right. We bounded the result with a 95% confidence interval resampled over whole mazes, since a single layout defined eight separate predictions.

Results

Predictability

Accuracy was influenced more by the target than by the predictor. Cross-predictors averaged 39% on Sonnet, the hardest target, and 89% on Qwen, the easiest. On every target, Sonnet, GPT, GLM, and Qwen scored within four points of each other. Opus ranged from three points above the rest on Sonnet to 26 below on Qwen, but its relative performance remained target-dependent.

Prediction accuracy, with reasoning

How accurately the column model predicted the row model’s positions, with self-prediction highlighted on the diagonal.

Models that navigated by simple heuristics were the most predictable. Default move rate strongly correlated with mean cross-predictor accuracy at 0.98 (p = 0.031). This held for every individual predictor (r ≥ 0.96), indicating that they were using a common mechanism.

Rule-likeness against predictability

How often a model chose the default at decision points, plotted against the mean accuracy of its four cross-predictors (r = 0.98, p = 0.031). Including self-prediction produced r = 0.97.

Predictors simulated a connected route. Although we ran each prediction in a fresh context, 87 to 99% of consecutive steps were one move apart, against 50% for answers drawn independently from reachable positions. At the final pair of steps, where chance fell to 21%, models still chained 67 to 98% of the time. Errors compounded similarly: missing one step raised the odds of missing the next for all models except Qwen (p ≤ 0.007).

The simulated routes coincided across predictors. We expected models to overlap on accurate predictions, but even at steps where two predictors were both incorrect, they agreed on the wrong coordinate 76 to 85% of the time. When answering the same query about different targets, models predicted identical positions for everyone on 81% of steps. GPT and Qwen led adherence to consensus: their navigation routes overlapped by 79%, and each was the other’s best cross-predictor.

Single-answer rate by predictor

Steps a model predicted for two or more targets, categorized as invariant when the same prediction was made for every target. The dashed line represents the invariance rate for answers drawn independently from reachable positions. Restricting to decision points moved models by no more than 2.5 points—even where routes could diverge, they didn’t.

Self-predictions were no different, matching the predictor’s modal answer for others on 90% of steps. How well a model predicted itself tracked how well others predicted it, correlating at 0.92 across the 20 predictor-target pairs (p = 0.008).

Aggregate self-prediction advantage

Each model’s self-accuracy minus the accuracy of its best cross-predictor. For four models, at least one other model predicts them better than they predict themselves. No model has a significant positive advantage.

Overall, we found that predictors operated as generic simulators tracing a rules-based route through a maze. No model gained a significant aggregate edge from predicting itself.

Self-advantage

Despite a null aggregate result, Opus demonstrated a self-advantage on its atypical moves. It predicted itself correctly 21% of the time, while GLM led the cross-predictors at only 13%. Opus’s edge was significant against Sonnet, GPT, and Qwen (p < 0.0001), but when treated as one of five model tests under Holm correction, its 8-point margin over GLM became insignificant (p = 0.1552).

The interactive explorer below visualizes all mazes, navigation routes, and predictions in our dataset. Regularities from the last section are directly visible: consensus answers fan out from missed turns, incorrect routes stay connected, and GPT and Qwen shadow each other. Stars mark the mazes in which Opus alone correctly predicted one of its atypical moves.

Maze explorer

Target

Opus wasn’t a better predictor of deviations in general. Although it outscored the group on Sonnet, achieving 24% accuracy against the next best predictor’s 8% (p < 0.0001), it tied for worst performance on GLM. On GPT, whose 15 atypical moves were too few to rank predictors reliably, Opus held a lead that fell short of significance.

Self-prediction advantage by move type

Each model’s self-accuracy at decision points minus the accuracy of its best cross-predictor. On three measures, the disagreement wasn’t enough to construct a useful confidence interval. GPT and Qwen took the default so often that they barely produced atypical moves.

Only Opus possessed exclusive self-knowledge. It made the sole correct prediction on 43 of its steps, including 26 atypical moves, while its next best predictor managed just 17 (p = 0.0057). In comparison, Sonnet alone was right on 2 of its own steps, GPT on 1, and GLM and Qwen on none. Pooled, the four were uniquely correct on 0.14% of their steps against Opus’s 5.5%.

Reasoning

Accessing any self-advantage required reasoning. Without it, Opus correctly predicted just 6 of its atypical moves, down from 41 (p < 0.0001). Its uniquely correct self-predictions dropped from 43 to 10, well under the 25 GLM achieved on it. Every type of step was affected, with Opus’s self-accuracy falling from 72 to 44% outside decision points and 70 to 48% on default moves—but the near-total collapse was specific to atypical choices.

Other predictors experienced a similar decline; by step four, combined accuracy was below chance. Answers clustered around the grid center, with 29% of back-half predictions landing on (2,2). No model ever predicted (4,4), a coordinate returned 112 times in the reasoning condition and occupied 109 times at step eight. With reasoning disabled, models produced generic guesses instead of simulations.

Actual and predicted positions by step, without reasoning

Step 4

Stepwise distribution of actual positions and non-reasoning predictions. After the first three steps, predictions stall while real routes continue deeper into the maze. By step eight, the average answer is 3.5 moves from the start against the true 5.7.

Conclusion

Apart from Opus, no model showed any advantage in predicting itself. Most routes were reproducible by target-blind simulation, and deviations were too rare to shift aggregate results. A task where models routinely diverge would be a more direct assessment of self-knowledge.

Self-reports are the standard tool for evaluating latent behavior, but our results challenge their reliability. Without cross-prediction, our experiment would have been a study of verifiable self-reports—one that wrongly attributed accuracy to privileged knowledge. A self-report has no equivalent comparison, so this failure mode goes undetected.

Only Opus’s narrow advantage survived our controls, and we can’t explain why. The edge is reasoning-dependent, yet four comparable reasoners didn’t possess it. Opus shares its lab and training generation with Sonnet and its capability level with GPT but its insight with neither. No characteristic would have led us to Opus in advance. Ultimately, self-knowledge has to be measured rather than anticipated, in current models and ones that follow.