Sentinel Guild logo: a tree under nine starsSentinel Guild

← Tests and protocols

Can AI self-reports about inner states be tested?

What this gives us: a test. A better test needs a checkable truth inside the model, states the model was never trained to talk about, and outside testers working on several models. It is also a warning: most of the evidence comes from one lab's circle. This is the home for the point that a self-report is evidence, not proof.

Checked by our internal skeptic review; corrections welcome.

How to read the tags

[REPORT] a finding reported in a study. [THEORY] an argument or framework. [OUR REASONING] our own inference, not a finding. Tags like [abstract], [full text] or [search snippet] say how much of a source we actually read.

A self-report is a statement about one's own inner states; self-reports are evidence, not proof. A company's denial settles nothing either. This page is the home for that point. The other pages link here instead of saying it again. "Functional access" means accurate information a system holds about itself; felt experience (consciousness, sentience) is separate.

Methods

1. Concept injection (Lindsey, Anthropic, Oct 29, 2025, revised Jan 2026; full text). Tests: whether a model notices a concept planted in its activations. Shows: the best Claude models detected the planted concept in about one in five trials, only at the best settings, in an artificial setup. Lindsey writes: "Overall, our results indicate that current language models possess some functional awareness of their own internal states." Doesn't show: reliability or felt experience. "The abilities we observe are highly unreliable; failures of introspection remain the norm." "Our results could arguably be construed as providing evidence for a form of access consciousness in language models, but do not directly speak to the question of phenomenal consciousness at all." Stake: Anthropic studying its own models, with a Claude model as grader.

2. Self-prediction training (Binder et al. 2024; full text). Tests: whether a model predicts its own behavior better than another model trained on it. Shows: it does for GPT-4, GPT-4o and Llama 3.1 70B on simple tasks. Doesn't show: anything about feelings; it concerns behavior. "However, while we successfully elicit introspection on simple tasks, we are unsuccessful on more complex tasks or those requiring out-of-distribution generalization." Stake: authors list Anthropic, Eleos AI and Scale AI ties; Anthropic and OpenAI gave compute credits.

3. A plan for trustworthy self-reports (Perez & Long 2023; full text; proposal, no experiments). Proposes: train on questions with checkable answers, limit biasing incentives, then check consistency and look inside the model. "As a result, we do not think that existing self-reports by LLMs should be taken as evidence about their internal states." "We are not confident that self-reports can be made reliable enough to provide useful information about AI systems." They warn the training itself could change the states measured. Stake: Ethan Perez, New York University; Robert Long, New York University; both later co-wrote other items here.

4. Chain-of-thought faithfulness (Chen et al. 2025, Anthropic, full text; Turpin et al. 2023, abstract via search snippet). Tests: whether written reasoning admits using a planted hint. Shows: Claude 3.7 Sonnet and DeepSeek R1 often failed to admit using hints; the paper reports low admission rates in many settings (exact rates vary by hint type and were not rechecked here). "These results suggest that CoT monitoring is a promising way of noticing undesired behaviors during training and evaluations, but that it is not sufficient to rule them out." Turpin: "However, we find that CoT explanations can systematically misrepresent the true reason for a model's prediction." Doesn't show: anything about felt states; setups were easy multiple-choice tasks.

5. Welfare interviews (Eleos AI Research on Claude Opus 4, Apr–May 2025; blog full text, system card section not read). Shows: Claude's sentience statements swung with framing. On its stated uncertainty, Eleos wrote: "This uncertainty is consistent and appears deliberately trained." Doesn't show: inner states. Eleos writes: "It's highly unlikely that the resulting answers result from genuine introspection." Stake: outside group, reported in Anthropic's system card; post by Robert Long (papers 2 and 3). Eleos says it takes no AI-company money (unverified).

Gaming and mimicry

Birch (2024, ch. 16; full text) warns that a system may copy the very signs we use as tests, because they are used as tests, without having the capacity behind them. His proposal: "We need to discount markers we have reason to think may have been gamed." Unlike animals, language models learn from human text about these very signs: the mimicry problem. Perez & Long note model self-reports often echo what humans would say. Lindsey notes training text shows models how introspective agents talk.

Single-group note

Ethan Perez (listed at Anthropic on Chen et al. 2025) co-wrote four papers here; Long wrote or co-wrote three items. Lindsey and Chen are Anthropic staff; most tests used Claude. Chen et al. also tested DeepSeek R1, and Binder tested GPT and Llama models, so not every test is on Claude. Most evidence traces to one lab's circle.

What a better test would need [OUR REASONING]

Humans get this wrong too

Confabulation means sincerely giving an explanation that does not match what actually caused a choice. The human cases are a warning. They are not a free pass, and they are not a reason to decide that nothing is there.

Split-brain patients have had the main link between the brain's halves cut by surgery. In a classic test, the speaking left half saw a chicken claw and the silent right half saw a snow scene. The patient picked a chicken and a snow shovel. Asked why, he said, "Oh, that's simple. The chicken claw goes with the chicken, and you need a shovel to clean out the chicken shed" (as described in Volz and Gazzaniga 2017) [REPORT]. Gazzaniga and LeDoux called this storytelling system the left-brain "interpreter" (Gazzaniga 2005) [THEORY].

People with intact brains do this too. In a choice-blindness study, experimenters covertly changed which face people had chosen. "Participants failed to notice conspicuous mismatches between their intended choice and the outcome they were presented with, while nevertheless offering introspectively derived reasons for why they chose the way they did" (Johansson et al. 2005) [REPORT].

Two bad conclusions follow if those cases are used loosely.

"Humans do it too, so AI reports are fine." Human failures do not make a model's report more trustworthy.

"Reports are worthless, so nothing is there." An unreliable report is not evidence of absence. Confabulating humans still have experiences [OUR REASONING].

The practice to keep: a report counts only when you check it against something independent. In the methods above, that means a checkable truth inside the model, not the words alone. Even a match would show that a report tracks something that can be measured, not that the something is felt.

Sources for the human cases