What is a Hypervector?

What is a hypervector?

In the previous post, we represented the information of a person, Maya, with one high-dimensional representation (what we’ll call a hypervector), and used part of that profile as a retrieval cue. In this post, we’ll unpack what a hypervector actually is.

In HDC, a hypervector is a vector (an array of numbers) that contains a very large number of dimensions, typically in the tens of thousands. The most common choice is 10,000 dimensions, which we can write as:

$$\mathbf{h} = [h_1, h_2, \ldots, h_D], \qquad D = 10{,}000$$

$D$ is the number of dimensions, and each $h_i$ is one component of the hypervector.

A hypervector isn’t a new mathematical object beyond a vector. What makes it useful is the geometry that appears when we use many dimensions and spread a representation across them.

---

What’s inside a hypervector?

The components of a hypervector are numbers, but their range and type vary by implementation. Common types include:

1. Binary: $\mathbf{h} \in \{0,1\}^{D}$
2. Bipolar: $\mathbf{h} \in \{-1,+1\}^{D}$
3. Integer: $\mathbf{h} \in \mathbb{Z}^{D}$
4. Real-valued: $\mathbf{h} \in \mathbb{R}^{D}$

The number representation affects the shape of the space, how HDC operations work, and which similarity measures make sense. Binary hypervectors are often compared with Hamming distance, while bipolar and real-valued hypervectors can use measures such as cosine similarity.

Why so many dimensions?

The exact measure depends on the representation, but high dimensionality gives independently generated base hypervectors a reliable, unrelated baseline. Two random binary hypervectors tend to differ in about half their components.

For the cosine view, base hypervectors are usually sampled from a distribution centered on the zero vector. In bipolar space, $+1$ and $-1$ balance in expectation, so the generating distribution’s centroid coincides with the coordinate origin. We draw each arrow from that shared origin: its endpoint defines a hypervector, and the angle between arrows gives their cosine similarity. For bipolar hypervectors (or real-valued hypervectors sampled uniformly in direction), that similarity tends toward zero:

$$\operatorname{sim}(\mathbf{a}, \mathbf{b}) \approx 0$$

As $D$ grows, random comparisons concentrate more tightly around these baselines. Atomic hypervectors are therefore reliably distinguishable rather than accidentally similar.

This leaves room to assign many starting symbols their own base hypervectors, then introduce meaningful relationships through the way we compose them.

For readers used to embedding-based retrieval, high dimensionality isn’t just a larger coordinate count. It matters in three practical ways:

1. Many distinct patterns can coexist. Independently generated base hypervectors begin nearly orthogonal, making accidental matches unlikely.
2. Meaningful similarity can stand out. Deliberately related or composed hypervectors can rise above the near-zero similarity of unrelated base hypervectors.
3. Composition stays fixed-width. HDC operations produce another hypervector in the same space rather than a representation that grows with every new component.

The entire pattern carries the representation

In an ordinary feature vector, a coordinate might have an explicit interpretation: the first value could be age, while another could be height or temperature. A hypervector is usually different. No single coordinate contains “Maya,” “hiking,” or “interest”; the pattern across the whole hypervector carries the representation.

This is its distributed, or holographic, property. The next post explores what that enables when part of a hypervector is missing, altered, or used as a retrieval cue.

Different objects, one shared space

The Maya hypervector example combined several hypervectors: roles, values, bound facts, a bundled profile, and a query. Even though they represented different things, they all had the same dimensionality:

$$\mathbf{r}_{\text{name}}, \mathbf{v}_{\text{Maya}}, \mathbf{f}_{\text{name}}, \mathbf{h}_{\text{Maya}} \in \mathcal{V}^{D}$$

$\mathcal{V}$ represents the chosen value set, such as $\{-1,+1\}$ or $\mathbb{R}$.

Keeping every representation in the same space is a practical feature of HDC. We can combine simple symbols into structured facts and larger memories without changing the basic shape of the object.

Where similarity comes from

It’s tempting to imagine that a hypervector for hiking naturally sits close to one for walking. That relationship exists only if the representation was constructed to preserve it.

Independently generated base hypervectors start nearly orthogonal, so the base hypervectors for hiking and walking contain no semantic relationship. If we want to preserve that relationship, we can use a lower-dimensional text embedding model that has learned from language that the terms are related. An encoder can then map that information into hypervector space and combine it with roles, relationships, or other fields.

A useful working definition

We can now summarize the qualities of a hypervector more precisely:

Quality	What it means
High-dimensional	It contains thousands of components, most often 10,000
Fixed-width	Operations can produce new representations in the same space
Distributed	The entire pattern carries the representation
Distinct and independent	Independently generated starting hypervectors are typically near-orthogonal, leaving room for more nuanced degrees of similarity
Composable	Simple hypervectors can be combined to produce larger structured representations, encoding domain knowledge

Together, these qualities make hypervectors useful building blocks without prescribing one encoding, data type, operation, or application.

The next post focuses on one of those qualities: what happens when information is distributed across the whole hypervector, and why a recognizable signal can survive partial loss or noise.