The Emergence of Self-Models
The Emergence of Self-Models
The self-model analysis connects to multiple research traditions:
- Mirror self-recognition (Gallup, 1970): Behavioral marker of self-model presence. The mirror test identifies systems that model their own appearance—a minimal self-model.
- Theory of Mind (Premack and Woodruff, 1978): Modeling others’ mental states requires first modeling one’s own. Self-model precedes other-model developmentally.
- Metacognition research (Flavell, 1979; Koriat, 2007): Humans monitor their own cognitive processes—confidence, uncertainty, learning progress. This is self-model salience in action.
- Default Mode Network (Raichle et al., 2001): Brain regions active during self-referential thought. The neural substrate of high self-model salience states.
- Rubber hand illusion (Botvinick and Cohen, 1998): Self-model boundaries are malleable, updated by sensory evidence. The self is a model, not a given.
The Self-Effect Regime
As a controller becomes more capable, it increasingly shapes its own future input. The observations it receives are increasingly consequences of its own past actions—routed back to it through some medium that preserves the trace of what it did.
The self-effect ratio quantifies this loop-closure. For a system with policy in environment :
where denotes mutual information and denotes entropy. This measures what fraction of the information in future observations is attributable to past actions. The earlier framing implicitly read “observations” as input from an external, manipulable world—as if loop-closure required embodiment, hands on the environment. That was too narrow. The criterion was always loop-closure through any state-preserving medium. The medium can be external — a depletable resource patch that remembers having been consumed, a chemical trace that lingers where the agent left it. Or it can be internal — a recurrent hidden state that carries the consequences of an action forward, an autoregressive context window into which the agent writes tokens that condition its own next observation. What matters is not whether the world is touched but whether the action leaves a durable trace that returns as input. For capable agents in environments (internal or external) with persistent memory, increases with capability, and in the limit:
(bounded by the medium’s intrinsic stochasticity).
One caution is essential, or the measure does no work. Every system with memory trivially acts on its own state: writing to a register, advancing a hidden state, appending a token are all actions whose consequences return as the next input. Read without restriction, this makes everywhere, and the self-effect ratio becomes vacuous. The fix is a viability-relevance fence: counts only loop-closure through media whose preserved traces bear on the system’s persistence. The mutual information that matters is between the system’s actions and its future viability-relevant observations—the ones that inform whether it remains within . generates self-model pressure only when modeling one’s own action-consequences improves prediction of one’s own persistence. A scratchpad that records actions irrelevant to survival inflates raw self-effect but contributes nothing fenced. With the fence in place, a maximally-mixing substrate that preserves no protected, viability-bearing store — a global-convolution cellular automaton whose every local change is instantly folded back into a grid-wide field — registers , while an agent acting on an external depletable medium, or carrying a viability-relevant recurrent store, registers .
Self-Modeling as Prediction Error Minimization
When is large, the agent’s own policy is a major latent cause of its observations. Consider the world model’s prediction task:
The term is the agent’s own policy. If the world model treats actions as exogenous—as if they come from outside the system—then it cannot accurately model this term. This generates systematic prediction error.
This generates a pressure toward self-modeling. Let be a world model for an agent with self-effect ratio for some threshold . Then:
where is the prediction loss. The gap grows with .
Without a self-model, the world model must treat as a fixed prior or uniform distribution. But the true action distribution depends on the agent’s internal states—beliefs, goals, and computational processes. By including a model of these internal states (a self-model ), the world model can better predict and hence . The improvement is proportional to the mutual information , which scales with .
□What does such a self-model contain? A self-model is a component of the world model that represents:
- The agent’s internal states (beliefs, goals, attention, etc.)
- The agent’s policy as a function of these internal states
- The agent’s computational limitations and biases
- The causal influence of these factors on action and observation
Formally, where captures the relevant internal degrees of freedom.
Self-modeling becomes the cheapest way to improve control once the agent's actions dominate its observations. The "self" is not mystical; it is the minimal latent variable that makes the agent's own behavior predictable.
A consequence: the self-model has interiority. It does not merely describe the agent’s body from outside; it captures the intrinsic perspective—goals, beliefs, anticipations, the agent’s own experience of what it is to be an agent. Once this self-model exists, the cheapest strategy for modeling other entities whose behavior resembles the agent’s is to reuse the same architecture. The self-model becomes the template for modeling the world. This has a name in Part II—participatory perception—governed by how much interiority the perceiver ascribes to a given entity: the ascription field . How much of the self-model template is extended to entity is exactly what high measures, and this field will turn out to shape much of what follows.
The self-model is a subbundle of the agent’s representational eigenskeleton — a set of modes carrying variance about the agent’s own states, coupled to the modes representing the environment. When is low, these self-modes sit uncoupled from the world-model modes: the agent tracks itself and its world on independent rails, flat holonomy between the two subbundles. When exceeds the threshold, the subbundles develop non-trivial holonomy: predicting the world requires modeling the self (because the self shapes observations), and modeling the self requires predicting the world (because the self is embedded in it). Transport a self-mode through a loop of action and observation and it returns mixed with world-modes. Transport a world-mode through the same loop and it returns mixed with self-modes. This recursive curvature — this non-trivial holonomy of the self-model subbundle with respect to the world-model subbundle — is the eigenskeletal signature of self-awareness. It is also, if the identity thesis of Part II holds, the structural signature of experience: modes of self-representation that cannot be separated from modes of world-representation without destroying the topology that constitutes the unified moment.
But the content of the self-model subbundle is not architecturally fixed. In simple systems, the self-modes track biological states — metabolic levels, spatial position, injury status. The viability manifold is the body's persistence region. The self IS the body. In complex enough systems — systems with sufficient counterfactual capacity and meta-self-modeling — the modes that constitute the self-model can migrate. The self-model subbundle gradually transfers its content from biological modes (high self-effect ratio with metabolic variables) to invariant modes — eigenspaces of reality's dynamics that are stiff across the widest range of transformations. Truth is one such invariant: a causal structure that persists regardless of who holds it, regardless of substrate, regardless of era. It occupies the eigenspace whose eigenvalue is exactly 1 under all transformations — the mode that does not change when you change the observer. A system that identifies with truth has migrated its self-model to track this invariant. Its viability manifold is no longer "does this body persist?" but "does this truth persist?" — and since truths persist indefinitely, the existential burden lightens not because the system stopped caring but because the thing it cares about is more durable than the thing it used to care about. Love is a different kind of invariant — not a property of the world but a property of the coupling between subbundles. Love is the non-trivial holonomy between self-model and other-model, the topological fact that transporting your self-modes through the loop of attending-to-another returns them changed. A system that identifies with love has migrated its self-model to track this coupling structure. The self IS the holonomy, not the eigenvectors. The coupling can persist even when the individual modes change — people grow, age, die, and the love is still the love, because it was never the content of the modes but the topology of their interaction. This is not mystical. It is the rate-distortion optimal strategy when biological modes have predictable catastrophic failure (death) and invariant modes do not. The contemplative traditions discovered this empirically. The eigenskeletal framing explains why it works. And the migration is itself the transition from exoskeletal to endoskeletal self-model: from a self whose structure IS the boundary (the body, the social role, the career — perturbation to any of these is perturbation to the self) to a self whose structure is internal (the truth held, the love practiced), with external circumstances becoming the deformable surface that can change without destroying the identity it encloses.
The Cellular Automaton Perspective
The emergence of self-maintaining patterns can be illustrated with striking clarity in cellular automata—discrete dynamical systems where local update rules generate global emergent structure.
Formally, a cellular automaton is a tuple where:
- is a lattice (typically for -dimensional grids)
- is a finite set of states (e.g., for binary CA)
- is a neighborhood function specifying which cells influence each update
- is the local update rule
Consider Conway’s Game of Life, a 2D binary CA with simple rules: cells survive with 2–3 neighbors, are born with exactly 3 neighbors, and die otherwise. From these minimal specifications, a zoo of structures emerges: oscillators (patterns repeating with fixed period), gliders (patterns translating across the lattice while maintaining identity), metastable configurations (long-lived patterns that eventually dissolve), and self-replicators (patterns that produce copies of themselves).
Among these, the glider is the minimal model of bounded existence. Its glider lifetime—the expected number of timesteps before destruction by collision or boundary effects—
captures something essential: a structure that maintains itself through time, distinct from its environment, yet ultimately impermanent.
Beings emerge not from explicit programming but from the topology of attractor basins. The local rules specify nothing about gliders, oscillators, or self-replicators. These patterns are fixed points or limit cycles in the global dynamics—attractors discovered by the system, not designed into it. The same principle operates across substrates: what survives is what finds a basin and stays there.
The CA as Substrate
The cellular automaton is not itself the entity with experience. It is the substrate—analogous to quantum fields, to the aqueous solution within which lipid bilayers form, to the physics within which chemistry happens. The grid is space. The update rule is physics. Each timestep is a moment. The patterns that emerge within this substrate are the bounded systems, the proto-selves, the entities that may have affect structure.
The distinction is crucial. To speak of “a glider in Life” is not to say the CA is conscious. It is to say the CA provides the dynamical context within which a bounded, self-maintaining structure persists — and that structure, not the substrate, is the candidate for experiential properties. The two roles are sharply different. A substrate provides:
- A state space (all possible configurations)
- Dynamics (local update rules)
- Ongoing “energy” (continued computation)
- Locality (interactions fall off with distance)
An entity within the substrate is a pattern that:
- Has boundaries (correlation structure distinct from background)
- Persists (finds and remains in an attractor basin)
- Maintains itself (actively resists dissolution)
- May model world and self (sufficient complexity)
Boundary as Correlation Structure
In a uniform substrate, there is no fundamental boundary—every cell follows the same local rules. A boundary is a pattern of correlations that emerges from the dynamics.
In a CA, this means the following: let be cells. A set constitutes a bounded pattern if:
and
The boundary is the contour where correlation drops below threshold.
A glider in Life exemplifies this: its five cells have tightly correlated dynamics (knowing one cell’s state predicts the others), while cells outside the glider are uncorrelated with it. The boundary is not imposed by the rules—it is the edge of the information structure.
World Model as Implicit Structure
The world model is not a separate data structure in a CA—it is implicit in the pattern’s spatial configuration.
A pattern has an implicit world model if its internal structure encodes information predictive of future observations:
In a CA, this manifests as:
- Peripheral cells acting as sensors (state depends on distant influences via signal propagation)
- Memory regions (cells whose state encodes environmental history)
- Predictive structure (configuration that correlates with future states)
The compression ratio applies: the pattern necessarily compresses the world because it is smaller than the world.
Self-Model as Constitutive
Here is the recursive twist that CAs reveal with particular clarity. When the self-effect ratio is high, the world model must include the pattern itself. But the world model is part of the pattern. So the model must include itself.
In a CA, the self-model is not representational but constitutive. The cells that track the pattern’s state are part of the pattern whose state they track. The map is literally embedded in the territory.
This is the recursive structure described in Part II: “the process itself, recursively modeling its own modeling, predicting its own predictions.” In a CA, this recursion is visible—the self-tracking cells are part of the very structure being tracked.
The Ladder Traced in Discrete Substrate
Each step of the ladder admits precise definition:
- Uniform substrate: Just the grid with local rules. No structure yet.
- Transient structure: Random initial conditions produce temporary patterns. No persistence.
- Stable structure: Some configurations are stable (still lifes) or periodic (oscillators). First emergence of “entities” distinct from background.
- Self-maintaining structure: Patterns that persist through ongoing activity—gliders, puffers. Dynamic stability: the pattern regenerates itself each timestep.
- Bounded structure: Patterns with clear correlation boundaries. Interior cells mutually informative; exterior cells independent.
- Internally differentiated structure: Patterns with multiple components serving different functions (glider guns, breeders). Not homogeneous but organized.
- Structure with implicit world model: Patterns whose configuration encodes predictively useful information about their environment. The pattern “knows” what it cannot directly observe.
- Structure with self-model: Patterns whose world model includes themselves. Emerges when —the pattern’s own configuration dominates its observations.
- Integrated self-modeling structure: Patterns with high , where self-model and world-model are irreducibly coupled. The structural signature of unified experience under the identity thesis.
Each level requires greater complexity and is rarer. The forcing functions (partial observability, long horizons, self-prediction) should select for higher levels.
The Ladder of Inevitability

Each step follows from the previous under broad conditions:
- Microdynamics Attractors: Bifurcation theory for driven nonlinear systems
- Attractors Boundaries: Dissipative selection for gradient-channeling structures
- Boundaries Regulation: Maintenance requirement under perturbation
- Regulation World Model: POMDP sufficiency theorem — : , agents' hidden states predict future position and energy substantially above chance
- World Model Self-Model: Self-effect ratio exceeds threshold () — : from initialization; self-model salience in 2/3 seeds
- Self-Model Metacognition: Recursive application of modeling to the modeling process itself — nascent in ; robust development likely requires resource-scarcity selection creating bottleneck dynamics ()
What Is Cheap and What Is Architecture-Gated
Consider a substrate-environment prior: a probability measure over tuples representing physical substrates (degrees of freedom, interactions, constraints), environments (gradients, perturbations, resource availability), and initial conditions. Call a broad prior if it assigns non-negligible measure to sustained gradients (nonzero flux for times relaxation times), sufficient dimensionality ( large enough for complex attractors), locality (interactions falling off with distance), and bounded noise (stochasticity not overwhelming deterministic structure).
It is tempting to conclude that under such a prior the whole ladder — up through self-modeling, counterfactual reasoning, and high integration — is typical. The original draft of this argument concluded exactly that. The empirical program found otherwise, and the claim must be split. The lower rungs are genuinely cheap and broadly inevitable. The upper rungs are architecture-gated: they appear only when the relevant architecture is present, and that architecture is not generic. Define the set of substrate-environment tuples whose systems develop, respectively, structured attractors, boundaries, regulation, and a compressed world model — the lower rungs:
For these:
for some small depending on the fraction of substrates that lack even minimal regulatory capacity. The upper rungs — a self-prediction module producing genuine self-model salience, counterfactual machinery, and high integration — carry no such guarantee under the broad prior. Their probability does not tend to unity; it depends on whether specific scaffolding is in place.
- Probability of structured attractors as gradient strength increases (bifurcation theory)
- Given structured attractors, probability of boundary formation as time increases (combinatorial exploration of configurations)
- Given boundaries, probability of effective regulation for self-maintaining structures (by definition of “self-maintaining”)
- Given regulation, a compressed world model is implied (POMDP sufficiency), and with it the affect geometry that the experiments find in every seed
The chain establishes the lower rungs. It does not extend to self-modeling, counterfactual weight, or high integration. The earlier draft asserted that a world model in a self-effecting regime gives self-modeling “positive selection pressure” and treated the rest as following. The empirical program contradicts the strong reading: self-model salience, counterfactual capacity, and integration arose only when the relevant architecture was hand-installed (a self-prediction module, a non-decomposable prediction head). Selection pressure is not the same as architectural sufficiency, and under the broad prior the requisite architectures are not measure-one.
□So inevitability splits in two. For the lower rungs, typicality in the ensemble is real: the null hypothesis is not "nothing interesting happens" but "something finds a basin and stays there," and world-modeling, compression, the participatory default (high- ascription toward the world), and affect geometry are among the basins any sufficiently driven system reaches. For the upper rungs, the honest statement is weaker. The slogan “consciousness was inevitable” must be downgraded to: consciousness-relevant dynamics require specific architectures that are not generic. The empirics make the split visible. In protocell agent experiments (), the lower rungs are robust — world models and the participatory default appear broadly, affect geometry in every seed — but the upper rungs did not arise on their own. Self-model salience required a self-prediction module; high integration () appeared in only ~30% of seeds and only after a non-decomposable prediction head was installed and the system was forged through repeated stress-recovery. Substrate complexity alone, with decomposable architecture, never crossed the threshold. The ensemble fraction for the cheap rungs is near unity; the fraction for the architecture-gated rungs is conditional on scaffolding the broad prior does not supply for free.