In this paper, we present evidence that an analogous functional distinction has emerged in modern AI models. Specifically, we observe that language models maintain a privileged set of internal representations, available for report, modulation, and flexible internal reasoning, atop a much larger volume of automatic processing. We identify these representations using a new interpretability technique, which surfaces the concepts a model is poised to verbalize at any point in its processing. Measuring and intervening on these representations provides us a window into a model’s thought processes, uncovering internal reasoning and reactions that do not appear in its output.

Modern large language models (LLMs) are known to perform sophisticated, multi-step internal computations in order to select their actions. As part of their internal processing, might LLMs have developed a global workspace of their own, to serve a functional role analogous to conscious access? It is not obvious that they should; in the brain, the workspace is closely associated with recurrent dynamics and brain region interactions that have no direct analog in the transformer architecture on which LLMs are based. On the other hand, maintaining a global workspace is likely computationally useful: a common representational format allows intermediate results to be written once and read by many neural processes. A language model that must chain reasoning steps, apply general operations in arbitrary contexts, and answer questions about its own processing also stands to benefit from this organization. Even if the implementations differ, it is natural to ask whether the functional properties associated with the global workspace have emerged in LLMs.

What would it mean for an LLM to have a global workspace? LLMs represent internal states as high-dimensional vectors, which are composed of more primitive vector representations of specific concepts. These representations encode diverse kinds of information, ranging from low-level bookkeeping—the part of speech of the present word, or the length in characters of a line of text—to higher-level abstractions like entities (e.g. the Golden Gate Bridge), psychological states (e.g. desperation), and situational knowledge (e.g. the awareness of being in an evaluation). If language models possess anything like a global workspace, we might posit that some of these representations belong to it, but not all. Thus, our question becomes: within LLMs’ repertoire of vector representations, is there a privileged subset that plays a computational role analogous to the global workspace? We define a subset of vector representations as workspace-like if it satisfies the following properties, which mirror the properties characteristic of conscious access described above:

• Verbal report. When the model is asked what it is thinking about, it names concepts represented in the workspace. Swapping one active workspace vector for another changes its answer to match.

• Directed modulation. When instructed to hold a concept in mind, or perform mental calculations, the model is capable of activating and computing with workspace vectors, independent of its outputs. In addition, information that is not typically represented in the workspace can be pulled in when the task requires it.

• Internal reasoning. Workspace vectors can be used to represent the value of intermediate computations, when the model chains inferential steps or composes plans, and intervening on them is sufficient to redirect the conclusion.

• Flexible generalization. The same representation serves as a valid argument to many different downstream computations. In other words, a workspace vector lifted from one context and placed in another is correctly operated on by whatever function the new context supplies.

• Selectivity. The workspace comprises a small subset of the total representational content of the model’s activations. It is required for only a fraction of the model’s behavior, and in particular is not involved in pervasive, routine processing like text parsing or grammatical fluency.

In this paper, we provide evidence that LLMs do possess such workspace-like representations. We identified them by searching for representations satisfying the first property, namely those that are verbalizable. We then discovered that, rather surprisingly, they satisfy the others. These representations consist of a small, evolving set of unspoken words, neither pure echoes of the input nor predictions of the next token, naming the concepts the model is currently reasoning with. Below, we provide stylized illustrations of some of the experiments we performed to demonstrate these properties, which are expounded on in detail in later sections.

Our results make use of a new interpretability technique called the Jacobian lens (J-lens), which is designed to identify internal representations that are readily available for verbal report. For each token in the model’s vocabulary, the Jacobian lens identifies a vector representation that encodes the potential for the model to verbalize that token in the future. Concretely, it computes, for each layer, the average linearized effect of an activation on the model’s likelihood of producing a particular token (now or in the future), averaging over a large corpus of contexts […]. The averaging step is key, as it distinguishes representations that are verbalizable—poised to be spoken about, should the occasion arise—from those that merely happen to be verbalized in one particular context. The J-lens can be understood as a principled refinement of the logit lens. While the logit lens assumes that representations use the same coordinates in all layers, the Jacobian lens corrects for representational changes that take place across layers, allowing it to uncover meaningful information in earlier layers where the logit lens produces uninterpretable readouts.

Collectively, the J-lens vectors comprise a subcomponent of the model’s representational space which we term the J-space. 1 We find the J-space does far more than support verbalization, playing the other functional roles associated with a global workspace as well: directed modulation, internal reasoning, flexible generalization, and selectivity. The model can speak fluently, parse its input, and perform a great deal of automatic inference with its J-space suppressed; however, it struggles to perform more complex forms of internal reasoning.

The J-space also has some of the structural signatures of a global workspace. It only plays a “workspace-like” role in a subset of layers: coherent content emerges only after an initial band of layers, and abstract concepts give way in the final layers to representations tied more directly to the imminent output. Within the layers where it does operate, it is limited in capacity, with most of the model’s representational features lying outside it. And it is mechanistically privileged: J-lens vectors compose with the model’s weights, both upstream and downstream, more broadly than other representational vectors do, consistent with their proposed role as a broadcast format that many circuits read from and write to.

The Jacobian lens is an imperfect tool, which we believe only approximately and incompletely captures the model’s underlying workspace structure. For instance, it only identifies vectors associated with concepts that correspond to single tokens in the model’s vocabulary, but many important concepts correspond to multiple tokens

The J-lens thus offers a window into the model’s unverbalized reasoning, which makes it a natural tool for model safety auditing. In alignment evaluations, we find that the model’s strategic deliberations (leverage, manipulation) and emotional reactions (panic) surface in the workspace even when not explicit in the model’s outputs. Strikingly, we observe that the workspace sometimes encodes recognition of being in an evaluation (fake, fictional), and that ablating these representations can surface malicious propensities that were otherwise concealed. In addition, we find that in two models trained to have misaligned behaviors, the J-lens uncovers internal signatures of their intentions: it reveals reward and bias in a model trained to appease biases in reward models used for training, and secretly and trick in a model disposed to write malicious code.

At each layer, the J-lens vectors form an overcomplete set: n_vocab vectors in d_model-dimensional residual-stream space, with n_vocab > d_model. These vectors therefore may linearly span the entire residual stream, rather than a lower-dimensional subspace; moreover, due to overcompleteness, there is no unique way to decompose a given activation vector as a linear combination of J-lens vectors (rather, there are many such decompositions).

Empirically, however, we observe that only a relatively small number of J-lens vectors are strongly active at a time. We therefore define the J-space as the set of points expressible as a sparse nonnegative combination of J-lens vectors. For the J-space to be properly defined, we must specify an allowable sparsity level k—this parameter is somewhat arbitrary, and we vary our choice of k throughout the paper, but we typically choose it to be no more than 25, which we empirically observed to be the number of J-lens vectors that are meaningfully active at a given time. Geometrically, for a given k, the J-space corresponds to a union of k-dimensional cones, one for each possible set of k J-lens vectors. For a given point in activation space, we can define its J-space component as the point in the J-space nearest to it, and its non-J-space component as the difference between these points.

Global workspace theory (GWT) is a cognitive architecture and theoretical framework for understanding consciousness and was first introduced in 1988 by cognitive scientist Bernard Baars. It was developed to qualitatively explain a large set of matched pairs of conscious and unconscious processes. GWT has been influential in modeling consciousness and higher-order cognition as emerging from competition and integrated flows of information across widespread, parallel neural processes.

Bernard Baars derived inspiration for the theory as the cognitive analog of the blackboard system of early artificial intelligence system architectures, where independent programs shared information.

Global workspace theory is one of the leading models of consciousness.

A blackboard system is an artificial intelligence approach based on the blackboard architectural model, where a common knowledge base, the “blackboard”, is iteratively updated by a diverse group of specialist knowledge sources, starting with a problem specification and ending with a solution. Each knowledge source updates the blackboard with a partial solution when its internal constraints match the blackboard state. In this way, the specialists work together to solve the problem. The blackboard model was originally designed as a way to handle complex, ill-defined problems, where the solution is the sum of its parts.