Integration Measures

Integration Measures

Let’s define precise measures of integration that will play a central role in the phenomenological analysis.

The first is transfer entropy, which captures directed causal influence between components. The transfer entropy from process $X$ to process $Y$ measures the information that $X$ provides about the future of $Y$ beyond what $Y$’s own past provides:

The deepest measure is integrated information ($\Phi$). Following IIT, the integrated information of a system in state $\state$ is the extent to which the system’s causal structure exceeds the sum of its parts:

where the minimum is over all bipartitions of the system, and $D$ is an appropriate divergence (typically Earth Mover’s distance in IIT 4.0).

In practice, computing $\Phi$ exactly is intractable. Three proxies make it operational:

1. Transfer entropy density—average transfer entropy across all directed pairs:
   $$\bar{\text{TE}} = \frac{1}{n(n-1)} \sum_{i \neq j} \text{TE}_{i \to j}$$
2. Partition prediction loss—the cost of factoring the model:
   $$\Delta_P = \mathcal{L}_{\text{pred}}[\text{partitioned model}] - \mathcal{L}_{\text{pred}}[\text{full model}]$$
3. Synergy—the information that components provide jointly beyond their individual contributions:
   $$\text{Syn}(X_1, …, X_k \to Y) = \MI(X_1, …, X_k; Y) - \sum_i \MI(X_i; Y | X_{-i})$$

A complementary measure captures the system’s representational breadth rather than its causal coupling. The effective rank of a system with state covariance matrix $C$ measures how many dimensions it actually uses:

where $\lambda_i$ are the eigenvalues of $C$. This is bounded by $1 \leq \effrank \leq \rank(C)$, with $\effrank = 1$ when all variance is in one dimension (maximally concentrated) and $\effrank = \rank(C)$ when variance is uniformly distributed across all active dimensions.