The Holographic Nature of Hypervectors

What does “holographic” mean in Hyperdimensional Computing?

An HDC representation is holographic, meaning that information isn’t stored in isolated components. It’s distributed across the entire hypervector, so the overall pattern carries the meaning. No single coordinate holds the full information for the name “Maya” or her interest, “hiking.”

Pentti Kanerva¹ calls this independence from position. Unlike traditional data storage, where losing a single bit can render information unreadable, a hypervector can remain recognizable even when some components are missing or altered.

If you’re used to structured data with rows and fields, this can feel a little backwards. We can no longer point to one coordinate and say, “the name of a person lives here.” What we gain is a representation that we can compare with others as a whole. And instead of requiring an exact match, we can ask whether a noisy pattern is still close to something we know.

---

A visual analogy: why size matters

A great way to understand this concept is by imagining two images: one with a large number of pixels and one with a small number. If we randomly damage the same fraction of pixels in each image, the effect on the overall picture is very different.

• If you have a 100-pixel image and you randomly damage 20% of it, the distortion would be highly noticeable.
• Now do the same to an image with 10,000 pixels. An identical 20% is lost, yet the overall structure remains nearly unchanged to the viewer.

The same amount of data loss (20%) was applied in both scenarios in the image above. The main difference is how much room each picture had to absorb it.

In a similar fashion, hypervectors operate in ultra-high-dimensional spaces (typically 10,000 dimensions or more), so they have a lot more room to absorb this kind of damage. A hypervector with $D$ components can be written as:

$$\mathbf{h} = [h_1, h_2, \ldots, h_D]$$

If $k$ of those components are altered or damaged, the fraction of the hypervector that changed is simply $\frac{k}{D}$. How resilient the representation is to such damage depends on the value of $D$. Losing 20% of a 100-pixel image leaves only 80 pixels to represent the concept, whereas losing 20% of a 10,000-dimensional hypervector still leaves 8,000 components intact.

Because the meaning lives in the whole pattern rather than in any single position, the remaining components still point in more or less the same direction. A corrupted hypervector drifts only a little from where it started rather than losing a fact outright, so the representation degrades gracefully instead of failing completely.

What the holographic property enables

Real-world inputs are messy. Sensor traces are noisy, hardware glitches introduce errors, and a query usually mentions only part of what we’re looking for. Because a hypervector tolerates all three, HDC opens up applications that are awkward for systems built around exact matching:

• Fault-tolerant AI: Autonomous driving, robotics, and healthcare all contend with imperfect inputs, from noisy sensors to incomplete records. Holographic hypervectors preserve the broader pattern when part of that input goes missing.
• Associative memory: A system can recall a complex pattern from a partial cue and update what it knows as new examples arrive, much like human memory.
• Neuromorphic computing: Distributed representations and simple operations suit highly parallel, brain-inspired hardware. The area is still experimental, but it points to ways of learning and inference beyond a conventional processor.

All three rest on the same two properties: a damaged hypervector stays recognizable, and a partial cue is enough to retrieve it. The sections below work through where both come from.

Maya as a holistic record

We saw bundling in the first post, where Maya’s name and interests were combined into one profile hypervector:

$$\mathbf{h}_{\text{Maya}} = \mathbf{f}_{\text{name}} \oplus \mathbf{f}_{\text{hiking}} \oplus \mathbf{f}_{\text{music}}$$

Here, $\oplus$ means “bundle these hypervectors together.” The exact bundling operation depends on the type of hypervectors being used.

The result is still one 10,000-dimensional hypervector. It doesn’t have a name section followed by hiking and music sections. All three facts are spread over the same pattern.

Why a partial match is enough

Bundling loses information by design. When Maya’s three facts are superposed into a single hypervector, the result is partially dissimilar to any one of them, because each one is diluted by the others. Bundle $n$ facts together and the similarity between the bundle and any one fact inside it works out to roughly $1/\sqrt{n}$.

So Maya’s profile is only about $0.58$ similar to the hiking fact it contains.² In a real-world dataset, it’s common to superpose a large number of facts (e.g., 50). Even in that scenario, the similarity is $0.14$, which is still far higher than the similarity between two unrelated hypervectors, which is close to zero.

That gap (due to the extremely high dimensionality) is what makes associative search practical in HDC. Take the hiking fact on its own, use it as a query, and every profile containing it emerges with a similarity score higher than every other profile that doesn’t.

Conclusions

Here are the key takeaways:

1. A hypervector holds its meaning everywhere, all at once.
2. It also holds just enough similarity to remain close to the components that formed it.
3. The holographic properties of hypervectors let us tolerate damage even when a significant chunk of the representation is missing or altered: the entire pattern is still recognizable, and a partial cue can help retrieve the whole record.

When weighing up HDC as a tool for a given problem, the useful question to ask is whether the signal we care about will stay above a threshold once every relevant fact, role, and value has been bound and bundled into the holistic representation. The decision ultimately comes down to the way the hypervector is constructed, and there are several ways to do so.

There’s a lot more coming from us at HDC Labs along these lines, so stay tuned for more insights!

Sources

• Pentti Kanerva, “Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors” (2009) ↗

Footnotes

1. In his 2009 paper, “Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors” ↗ , Kanerva brings several earlier distributed-representation models into one HDC framework. The terms independence from position and holistic representation come from this paper. ↩

2. The $1/\sqrt{n}$ figure assumes the bundled facts are near-orthogonal and that bundling is plain summation. Binarizing the result by majority sign gives exactly $0.5$ for three facts rather than $0.58$, which doesn’t change the argument. It does set a ceiling, though: since similarity keeps fading as $1/\sqrt{n}$, a record packed with enough facts will eventually sink close enough to the floor that an unrelated record beats it by chance. ↩