A category is something a system does

What if categorization isn’t the brain assigning a label after recognition, but an ongoing process shaped by past experience that guides perception and action as new sensory signals arrive?

This inversion of traditional thinking sits at the center of a fascinating new neuroscience paper, titled Categorization is “baked” into the brain, a Perspective by Lisa Feldman Barrett and Earl K. Miller in Nature Reviews Neuroscience.¹ Conor Feehly’s Quanta article ↗ provides a useful analogy for the proposal: the nervous system is less like a filing cabinet that stores fixed categories and more like a prediction engine that continually reconstructs them.

In Barrett and Miller’s account, categorization begins before incoming sensory signals have been fully processed. Predictions based on prior experience shape how those signals are processed. They help determine which details count as equivalent, which differences matter, and which actions the organism prepares.

The framework is based on neuroscience methodologies (it doesn’t directly mention HDC, Vector Symbolic Architectures (VSA), hypervectors, binding, or associative memory). But it raises some fascinating questions for us at HDC Labs:

• Should a category prototype be one durable class summary, or something reconstructed for the current situation?
• Should a lens filter results after retrieval, or reshape the effective geometry in which retrieval happens?
• Can a system decide how much computation and learning an event deserves according to its predicted value and cost?

This post summarizes the paper’s claims, highlights the HDC connection, and describes an experiment we are pursuing to test whether HDC can implement the framework’s requirements.

Categories before labels

The Quanta article² illustrates the idea with an analogy. Say an animal experiences a sensation, like a scratch on the leg. In a familiar, safe environment, the sensation may be ignored. When the animal is moving around in an environment covered in tall grass, the body is already primed to sense danger, so the same sensory evidence may be categorized as a possible bite from an insect or a snake. The signal has not changed, but its meaning for action has.

The traditional thinking as described by Barrett and Miller explains this in the forward direction. Sensory systems detect features, combine them into a representation, compare that representation with category knowledge in memory, and finally assign a class. In their paper, they present an alternative that reverses this causal emphasis.

Instead of retrieving a known category, the brain uses memories of prior situations to help construct a possible category first. The brain predicts (rather than retrieves) likely bodily consequences and actions. Incoming signals are interpreted inside this already active context.¹

This makes categorization an operating principle that runs throughout processing rather than after it finishes. A category is a temporary event in which different inputs are treated as equivalent for the present situation at hand. It draws on prior experience, while preparing a way of seeing and regulating the next action.

The bow-tie of information compression

In the feedforward direction, sensory signals travel deeper into the brain and are whittled down: many signals converge onto fewer neurons, fine physical details are lost, and the remaining information becomes more abstract. This narrowing forms one half of a conceptual “bow-tie”, converging toward regions that combine information from the senses, memory, and body.² This idea of compression during concept learning and increasing abstraction across several brain systems is supported by substantial prior work.³

Feedback forms the other half of the bow tie, traveling in the opposite direction. The compressed states expand into increasingly specific predictions about expected sensory signals and possible actions. As new sensory signals arrive, they are compared with what the brain already expects. Signals that fit can be suppressed or explained; residual mismatches continue forward and can become teaching signals.

The bow-tie image is useful, but it should not be mistaken for one literal central bottleneck. The paper describes a recurrent, branching gradient, and its use of the term “limbic system” includes cortical areas. The framework spans the whole brain and body. It makes the interesting observation that cognition happens in a much broader region than the cortical command center.¹

The authors also describe the notion of mixed selectivity: the response of a neuron can depend on combinations of stimulus, task, action, and current network state.⁴ This supports the broader point that a signal’s effective meaning is not fixed independently of its ensemble context. It does not, by itself, identify one unique algorithm for constructing that context.

A category is an equivalence for a purpose

Compression alone cannot tell us which details are safe to discard. A category becomes useful only when treating different inputs as equivalent serves a purpose.

Barrett and Miller answer in terms of “situated action” and allostasis. Put simply, allostasis is how the body anticipates what it will need and adjusts its energy use in advance. Categorization supports that preparation: the same object can call for different actions as the body’s needs change. The categorization of an apple can be done by grouping its features such as shape and color, but it can also be as abstract as “food now,” “save for later,” “throw away,” or “object to hand to someone.” Which distinctions matter depends on the current goal, environment, learned history, and bodily state.¹

The apple example reveals how broadly Barrett and Miller’s framework differs from a fixed-prototype model. The two accounts disagree about where category knowledge comes from, when past experience shapes sensory processing, what counts as similar, and what categorization produces:

Question	Simplified fixed-prototype account	Barrett and Miller’s account
What supplies knowledge?	A durable class summary or stored exemplars	Past experience reconstructed for the present situation
When does past experience shape processing?	During comparison or after feature extraction	Before and during feedforward processing
What defines similarity?	Relatively stable feature resemblance	Functional equivalence for the current goal
What is produced?	A class assignment	A situated perception, action, and body-regulation event

The framework is particularly strong because it links categorization to a reason for categorizing. An organism has limited energy, time, and processing capacity. It cannot preserve every distinction in the sensory stream. A category is useful when it keeps the distinctions that matter for an anticipated future and compresses the rest.

For HDC, this raises a practical design challenge: how can a system preserve what it has learned while changing which distinctions matter from one situation to another?

From neuroscience to HDC

HDC gives us explicit mechanisms for distributed representation, associative lookup, prototype construction, compositional binding, and incremental updates. Those mechanisms let us explore that challenge through operations we can build and test.

Durable memory, transient prototype

Barrett and Miller describe categories as temporary constructions shaped by the current situation. An HDC prototype is a different kind of object: a mathematical representation, often built by bundling the hypervectors of encoded examples.⁵ We are not claiming that the two are equivalent. Instead, the contrast motivates an HDC adaptation: keep long-lived memory stable, but construct the prototype needed for the current situation.

Conceptually, a situated category could be constructed as:

$$\mathbf{h}_{c\mid L_t} = \operatorname{normalize} \left( \bigoplus_{i\in\mathcal{M}_c} w_i(L_t)\,\mathbf{h}_{i} \right),$$

where $\mathcal{M}_c$ is durable memory associated with category $c$, $L_t$ is the current lens, and $w_i(L_t)$ selects or weights the remembered exemplars that matter in the present situation. The result $\mathbf{h}_{c\mid L_t}$ does not replace durable memory. It is a temporary construction built from that memory.

This is an HDC Labs adaptation opportunity, not an algorithm in Barrett and Miller’s paper. Its value is that it makes a distinction the framework requires: memory must persist across situations, while categories can change from moment to moment.

A lens changes effective topology

We have previously thought of an HDC lens as a way to take a particular view of a larger hyperspace: geospatial relations, threats, ownership, or another relevant facet. The paper pushes this idea further. If context shapes feedforward processing from the outset, a lens cannot be only a predicate applied after retrieval. It must participate in constructing what counts as nearby.

A candidate lens state might combine goals, task, environment, history, and resource condition:

$$\mathbf{h}_{L,t} = \mathbf{h}_{\text{goal},t} \oplus \mathbf{h}_{\text{task},t} \oplus \mathbf{h}_{\text{environment},t} \oplus \mathbf{h}_{\text{history},t} \oplus \mathbf{h}_{\text{resource},t}.$$

The bundling symbol is conceptual here. It does not imply that an unweighted superposition is the right implementation. The bigger idea is that the lens changes the effective neighborhood graph induced by the encoder, similarity function, memory, and cleanup threshold⁶ :

$$\mathcal{N}_{L_t}(\mathbf{h}_{q}) = \operatorname{TopK}_{\mathbf{h}_{i}\in\mathcal{M}} s_{L_t}(\mathbf{h}_{q},\mathbf{h}_{i}).$$

Changing $L_t$ can change which memories are neighbors even when the durable hypervectors remain fixed. A system might gate features before encoding, select a component subspace, change a cleanup rule, construct a situated prototype, or route the query through a context-indexed associative memory. “Changing topology” means changing the effective retrieval structure for the situation rather than physically rewriting one global space.

There is an important algebraic trap hiding in this idea. With bipolar element-wise binding, applying the same lens to both a query and a prototype preserves their dot product:

$$(\mathbf{h}_{q}\otimes\mathbf{h}_{L})\cdot (\mathbf{h}_{p}\otimes\mathbf{h}_{L}) = \mathbf{h}_{q}\cdot\mathbf{h}_{p}.$$

The reversible transform does not re-rank the prototypes. A computational lens must act asymmetrically or change state through selection, weighting, encoding, prototype construction, memory routing, or prediction.

The same issue appears in our review of hardware-aware HDC . A device transfer function can reshape perceived geometry unintentionally; a lens would reshape task-relevant geometry intentionally. Both force the same question: which invariant must survive? A classifier may need the same winner and a useful margin.

A relational memory may need separation between unrelated roles and a reliable inverse. A lens that improves situated classification while destroying unbinding has not solved the whole architecture.

Predict cheaply; process the valuable residual

The framework treats unpredicted feedforward activity as an important learning signal. In HDC, a simple conceptual mismatch score could be:

$$e_t = 1-\cos\!\left( \mathbf{h}_{\text{observed},t}, \mathbf{h}_{\text{predicted},t} \right).$$

But surprise alone is not enough. A flickering shadow can be unexpected and irrelevant; a small change in battery temperature can be expected and consequential. A system would need to combine residual magnitude with uncertainty, urgency, predicted value, and the cost of investigating or learning.

The distinction suggests a different objective from simply making every operation cheap. Predict familiar input through a low-cost path, then purchase precision where the expected consequence justifies it. Expensive graph traversal, a higher-fidelity sensor pass, a larger model call, or a durable memory update becomes a controlled allocation rather than the default response to every event.

From allostasis to system design

From an HDC perspective, a lens is a hypervector that represents the system’s current goal and operating conditions. By changing which stored patterns are most relevant and which observations or actions the system predicts, the lens sits at the center of the loop below:

This loop changes six architectural assumptions.

1. Context becomes endogenous state. A lens includes goals, history, environment, commitments, and resource condition, not only a user-selected filter.
2. Concepts operate at multiple timescales. Durable memories support situated prototypes and momentary exemplars rather than one mutable hypervector doing every job.
3. Inference becomes bidirectional. Bottom-up encoding is paired with top-down prediction.
4. Learning becomes selective. Residuals are weighted by anticipated consequence before they trigger updates.
5. Action closes the loop. A category matters because it changes what the system prepares to do, inspect, or ignore.
6. Compute becomes a regulated resource. Events can take a cheap associative path or be escalated according to predicted value and cost.

Graphs and hypervectors could play complementary roles inside this loop. A graph can retain causal, temporal, and relational specificity. A hypervector can carry compressed equivalence, support approximate retrieval, and summarize the active state. This is a proposed division of labor, not a completed architecture or a result established by the neuroscience paper.

Where in-memory computation could matter

HDC repeatedly compares large distributed patterns with stored prototypes. Moving those representations back and forth between memory and a central processor can dominate the cost that the simple arithmetic was supposed to save. In-memory HDC systems have demonstrated encoding and associative computation using memristive crossbars and phase-change memory devices.⁷ More recent work has implemented hardware-algorithm co-design for HDC on a multicore analogue memristive system-on-chip.⁸

In such an architecture, computational memory could provide the cheap local path: compare an observation with expected or situated prototypes where those memories are stored, then escalate only a value-weighted residual.

Hardware introduces a caveat. Drift, device variation, write endurance, analogue-to-digital conversion, and peripheral circuits can bend geometry or erase an apparent energy advantage. In-memory computation matters because it can reduce movement and parallelize associative work, not because specialized hardware makes an architecture automatically efficient.

Experiments we would pursue next

Future work should pursue a bounded experiment comparing conventional fixed categories with context-conditioned predictive categorization under ambiguous or noisy input, changing goals, and changing resource conditions.

We want to learn whether an HDC system can reinterpret the same evidence appropriately when the situation changes, allocate more computation and learning to consequential residuals, and do so without destabilizing durable memory. The experiment follows directly from the paper’s invitation to compare its neural-context hypothesis with prototype, exemplar, and network models.¹

The public research question therefore has three parts:

• Reinterpretation: Does the same evidence support the appropriate category when goals or resource conditions change?
• Selectivity: Does the system spend deeper computation and learning on consequential residuals?
• Stability: Can it adapt without erasing durable category memory?

Our goal is to report the methods, model choices, evaluation, and results through experimentation, rather than treating the architectural analogy as a result that’s set in stone.

From recognizing patterns to deciding what to do

What drew us to Barrett and Miller’s account was the connection between recognizing the world and preparing to act in it. In their proposal, categorization helps an organism anticipate its needs, so the same evidence can call for different responses. For us, the interesting HDC question became how a representation could help a system decide what to do next.

Following that idea into HDC changes how we think about the surrounding architecture. Durable memories could supply prototypes rebuilt for the present goal, while a lens shapes which memories are relevant during retrieval. Familiar input might need only a local associative match; a consequential mismatch might justify graph reasoning, a richer model, or learning. The system’s available time and energy would help govern those choices.

HDC gives us a concrete way to explore these ideas: build the representations, observe how they shape retrieval and action, and refine how the system learns from experience. Our aim is to connect stable memory with flexible behavior and deliberate use of computation. A neuroscience perspective prompted us to see those capabilities as parts of one design problem: how can a system use what it knows to prepare for what it needs to do?

Footnotes

1. Lisa Feldman Barrett and Earl K. Miller, “Categorization is ‘baked’ into the brain” ↗ , Nature Reviews Neuroscience 27 (2026): 435-456. Open author copy ↗ . ↩ ↩² ↩³ ↩⁴ ↩⁵

2. Conor Feehly, “A New Framework for How the Brain Compresses Our Noisy World” ↗ , Quanta Magazine, August 24, 2026. ↩ ↩²

3. Maria L. Mack, Alison R. Preston, and Bradley C. Love, “Ventromedial prefrontal cortex compression during concept learning” ↗ , Nature Communications 11 (2020): 46. ↩

4. Kay M. Tye et al., “Mixed selectivity: cellular computations for complexity” ↗ , Neuron 112 (2024): 2289-2303. ↩

5. Denis Kleyko et al., “Classification and Recall With Binary Hyperdimensional Computing: Tradeoffs in Choice of Density and Mapping Characteristics” ↗ , IEEE Transactions on Neural Networks and Learning Systems 29 (2018): 5880-5898. ↩

6. In HDC, cleanup replaces a noisy or approximate hypervector with its closest known match in associative memory. A cleanup threshold sets how similar the match must be before it is accepted. ↩

7. Geethan Karunaratne et al., “In-memory hyperdimensional computing” ↗ , Nature Electronics 3 (2020): 327-337. ↩

8. Yi Huang et al., “Hyperdimensional in-memory computing with analogue memristive crossbar arrays” ↗ , Nature Communications 17 (2026): 9162. ↩