What can we recover from a bundled hypervector?

In our earlier post ↗ on HRR and MAP algebras, we demonstrated mathematically how binding can be reversed. But that opened up a question about the other key operation in HDC encoding: bundling. What can we recover after several facts have been bundled together? This shifts the question from reversing an operation to recognizing the facts represented in its result.

Binding represents the association between a role and a value with a hypervector typically dissimilar to either input. Bundling brings several hypervectors together through superposition, preserving similarity to its inputs.

A single bundled hypervector can therefore act as a searchable representation of an entire record: a query can specify a related interest or a combination of attributes and retrieve it by similarity, without requiring exact field-value matches. Let’s see how this works in practice.

Start with two records

We’ll use a small dataset from our experiment repository ↗ : two people, Maya and Nina, each described by an age, an eye color, and three interests.

Each field gives us a role and a value to bind together. Age and eye color occur once per person, while interests reuse the same role three times. The names are lookup identifiers and are not bound into the representation.

Build the encoded representation

We want to represent all five of Maya’s facts in one hypervector, $\mathbf{h}_{\mathrm{Maya}}$. The pipeline has three stages: an embedding model represents the values, MAP projection converts those embeddings into bipolar hypervectors, and HDC encoding binds the values to roles and bundles the facts.

Use an embedding model to capture semantic similarity

Maya has an interest in jazz, but a search query might ask for lounge music, which isn’t in the source data. If we assign each interest phrase an independent random hypervector, we can distinguish the phrases, but their encodings won’t reflect their semantic relationship.

We’ll use nomic-embed-text to generate semantic embeddings of each value, including age and eye color as well as interests. As we explained in our earlier post Where does meaning in a hypervector come from?, the embedding model already learned semantic similarities for us, so we don’t need to discover meaning again in hyperspace.

Transfer learned similarity into hyperspace

To use those embeddings with HDC’s algebra, we project up to the higher-dimensional hyperspace¹ using a fixed random transformation. We then convert each component to $+1$ or $-1$ according to its sign, giving us bipolar Multiply-Add-Permute (MAP) hypervectors.

Bind and bundle the values into one hypervector

We use independent random bipolar hypervectors for age, eye color, and interest, reusing the same roles for Maya and Nina. We’ll write $\mathbf{h}_{\mathrm{doc}}(x)$ for an encoded stored value and $\mathbf{h}_{\mathrm{query}}(x)$ for an encoded search query.

Binding is element-wise multiplication, denoted by $\otimes$. We use it to combine each value with its role, producing one hypervector per fact.

Bundling is element-wise addition: we add the first components of those five hypervectors, then the second components, and so on. Each sum becomes the corresponding component of $\mathbf{h}_{\mathrm{Maya}}$. Written in terms of the five facts:

Nina’s hypervector, $\mathbf{h}_{\mathrm{Nina}}$, follows the same structure with her values. Every fact spans all $D=4096$ dimensions used in this experiment. Adding them changes the component values, but the bundle still has $D$ components.

We retain the raw sum without signing the bundle back to $\pm1$. This preserves the component magnitudes, allowing us to subtract a known fact’s exact binding later. Recovery starts with less information: we know which attribute we’re asking about, but want to identify its value by similarity.

Retrieve a record by attribute

Suppose we’re only interested in finding people with brown eyes. In a database, we’d normally ask a structured query and apply a metadata filter such as WHERE eye_color = 'brown'.

We’re interested in the HDC version of the same request: can a similarity query containing only the eye-color role and the value “brown eyes” retrieve Maya’s record?

The query phrase “brown eyes” is encoded and bound to the eye-color role:

We then calculate the query’s cosine similarity with $\mathbf{h}_{\mathrm{Maya}}$ and $\mathbf{h}_{\mathrm{Nina}}$, ranking them from highest to lowest score.

The result is a ranking by similarity to the requested attribute, rather than the exact field-value matches returned by our SQL filter. A record can therefore be retrieved for a semantically related value even when its stored text differs from the query.

Experiments

We use Maya and Nina’s five-fact records with a text embedding model (nomic-embed-text) and a 4,096-dimensional MAP projection. The choice of 4,096 dimensions is arbitrary (we didn’t run an experimental sweep of different dimensions in this study). Across five experiments, we change the query or subtract a contribution from Maya’s bundle to test semantic retrieval, query preferences, and exact removal.

We also measure how projection and signing change semantic similarity, then separate retrieval scores into contributions from individual facts. This lets us distinguish changes introduced by MAP projection from interference introduced by superposition.

E1: Check whether projection preserves semantic similarity

The source data for Maya records her interest as jazz. A query for lounge music can only help us find stored jazz if their encodings retain a useful notion of semantic similarity between these terms.

We use jazz as the exact-wording baseline, then compare it with the related queries music, lounge music, and R&B. This lets us ask whether conversion to MAP hypervectors preserves the distinction between an exact-wording query and a related phrase.

The first group compares jazz encoded as a query with jazz encoded as a stored value. These encodings differ because we use Nomic’s query and document prefixes,² so their similarity is already below 1 before projection.

Each group of bars in the chart below follows the same query through three stages: native embeddings, linear projection, and signing (a process we explained in detail in our earlier post, Where does meaning in a hypervector come from?). We can then check whether the relative ordering of the four queries survives those transformations, before binding or bundling introduces any other effects.

For lounge music compared with stored jazz, linear projection leaves their cosine similarity largely unchanged. The larger change occurs during signing: keeping only the sign of each component discards its magnitude, so the bipolar hypervectors retain the relationship with a lower similarity score.

E2: Retrieve a record with a phrase we never stored

We can now use the encoded query to ask: during retrieval, can the phrase lounge music rank Maya ahead of Nina (because only Maya has an interest that’s semantically similar to it)?

The query is encoded as follows:

We used LanceDB to store each person’s hypervector alongside their person_id, so we get back the projected columns we ask for, ordered by cosine distance. The returned ID lets us look up the original record to retrieve the full source data, including the full list of interests.

result = (
    table.search(unit(query).to(torch.float32).tolist())
    .distance_type("cosine")
    .bypass_vector_index()
    .limit(table.count_rows())
    .to_arrow()
)
rows = pl.from_arrow(result).to_dicts()
scores_by_id = {row["person_id"]: 1.0 - row["_distance"] for row in rows}

We test the exact stored interest, a related phrase, and an unrelated phrase:

As expected, Maya ranks first for lounge music. The exact wording jazz separates the records more strongly, even though its query encoding differs from the stored encoding. Moving to a related phrase broadens the matching evidence; it doesn’t guarantee equally clear discrimination.

The unrelated aircraft maintenance cue still gives both people scores near 0.30 and selects Nina.

E3: Explain which facts contribute to the retrieval score

What drives Maya’s lounge music score: jazz, her other interests, or interference from age and eye color? To explain the retrieval score, we can unbind the interest role and examine the values contributing to the match.³ In HDC literature, this is called a role probe.

Comparing this role probe with the query value gives the same cosine score as the bound query’s search against Maya’s record. The experiment checks this equivalence across all 26 probes, with agreement within $10^{-12}$.

We then split Maya’s score into contributions from each fact, using the same normalization for all five so they add up to the original score.

Jazz clearly contributes the most to the score, with tea and climbing also adding some of the evidence. Age and eye color remain in the interest probe: their interference is the crosstalk we introduced earlier, now measured to have little effect on this score.⁴

The additive structure of bundling lets us explain retrieval scores through individual source facts, unlike traditional embedding retrieval.

E4: Refine the query with positive and negative evidence

Both records contain a tea hypervector bound to the interest role and bundled with the other facts. To search for that shared interest, we encode tea as a query hypervector and bind it to the same role:

Suppose we now want to ask a query that favors tea drinkers but penalizes any evidence for the cue “brown eyes”. We can express that preference using the same additive structure of bundling:

For each record, its dot-product score with the brown-eye query is subtracted from its score with the tea query, before cosine normalization. A stronger brown-eye match therefore subtracts more from the score. We expect this to penalize Maya more than Nina.

Subtracting the brown-eye query lowers Maya’s score more than Nina’s, moving Nina into first place. Maya still scores above zero because the refined query also matches other facts in her record.

This connects back to associative search over connected data ↗ . A graph might represent an interest as an edge; our HDC encoding represents it as a role-value binding. Similarity can propose a person from a partial cue, and the source record supplies exact properties and relationships for filtering and traversal.

E5: Test whether subtraction recovers the remaining bundle

Does subtracting the jazz contribution from Maya’s bundle produce the same hypervector as encoding her remaining facts from scratch? We compare every coordinate, then repeat the subtraction with query-encoded jazz and lounge music to test whether a similar contribution can substitute for the exact one.

We retained the raw component sums. Signing the bundle back to $\pm1$ would turn both $+5$ and $+1$ into $+1$, for example, losing the magnitudes needed for exact subtraction. With those magnitudes intact, subtracting the exact contribution for jazz should leave the sum of everything else. This is the prediction we test with $\mathbf{h}_{\mathrm{orange}}$, shown in orange bars in the chart below:

But lounge music is a similar cue that also closely matched Maya during retrieval. Does that similarity make it a valid substitute when subtracting? We test this with $\mathbf{h}_{\mathrm{green}}$, shown in green bars in the chart below:

We compare both results with Maya’s original bundle (blue) using four queries. Each group of bars shows the scores for one query. In particular, let’s focus on the middle cases, music and lounge music, from the chart:

Semantic similarity lets us use similar phrases for retrieval, but those same phrases do not work well when subtracting from a bundle. The green lounge-music bar shows that subtracting this query leaves the resulting bundle with almost zero similarity to that same query.

The orange bars show that subtracting jazz from the full hypervector still lets us match the remaining evidence from tea and climbing. In E3, both interests contributed to the lounge music score alongside jazz. Subtracting jazz leaves those interests in the bundle.

Conclusions

Our experiments showed what we can recover from a bundled record:

• Preserve semantic relationships: The tested phrases had different cosine similarity scores in text embedding space and HDC space, but retained their relative similarity ordering.
• Retrieve from a related cue: Lounge music ranked Maya first without an exact match to jazz. An unrelated query also produced a clear winner, so this showed how ranking alone didn’t establish relevance, but similarity guarantees a non-null result.
• Explain a score: Jazz, tea, and climbing all contributed to the lounge-music match. We could clearly measure and explain those contributions using the encoded source facts.
• Express query preferences: Subtracting evidence for brown eyes moved Nina above Maya in the tea search. It changed the ranking, without enforcing an exact exclusion.
• Distinguish removal from retrieval: The exact jazz contribution could be subtracted from the additive bundle. A related phrase could retrieve the record, but couldn’t substitute for that contribution when removing it.

You can reproduce these results using the code and instructions in the experiment repository ↗ .

When working with HDC, it’s natural to ask “Can bundling be reversed like binding?” This study shows why the more useful question for bundling is less about exact reversibility and more about recoverability: can we reliably recognize facts represented in the bundle? We don’t need to reconstruct every input to retrieve a relevant record or examine evidence for an attribute.

What we would study next

As more and more bound facts are added to a bundle, their contributions become harder to distinguish. The next question is: what happens to individual answer recovery and whole-record similarity as we add more facts?

Our next study will vary the number of facts and hypervector dimensions in MAP, measuring how accurately we can read back one answer and how well whole response patterns retain their similarity. The aim is to learn what a bundle preserves as it grows, rather than assume a universal number of facts that every bundle can hold. Stay tuned!