How Adiyan remembers, and how retrieval actually works

Chapter 01

Why RAG exists

A language model is a fixed function. Its weights encode what it saw during training and nothing else. Three consequences follow, and retrieval-augmented generation exists because of all three at once.

• It cannot know your data. Someone uploads a price list, a contract, a 300-page book. None of it was in any model's training set.
• It cannot know recent data. Weights freeze at a cutoff date. Retraining a model to add one document is absurd economics.
• It cannot cite. A model answering from its weights cannot tell you which document the answer came from. For anything auditable, that alone disqualifies it.

The two obvious alternatives, and why they lose

Fine-tuning bakes knowledge into weights. It is the wrong tool for facts: expensive, slow, requires retraining per update, offers no provenance, and is notoriously unreliable at making a model reproduce one exact string such as a reference number. Fine-tuning changes behaviour and style well. It teaches facts badly.

Putting everything in the prompt works until it doesn't. Adiyan's knowledge base holds 1,601 chunks. At a modest 200 tokens each that is roughly 320,000 tokens per request, against a local model running a 16,000-token context window. Cost and latency scale with whatever you stuff in, and answer quality degrades well before the hard limit is reached.

Every RAG system is the same seven steps. The rest of this document is each step, in order.

Chapter 02

What a dimension is

Everything in this document rests on one idea, so it is worth building it from nothing.

Start with one number

Suppose you want to describe a cup of coffee to a computer. You pick a property you can measure — temperature — and write down one number: 85. That single number is one dimension. A dimension is just one measured property; one slot in a list.

One number is thin. Add a second property, volume in millilitres, and now you have two numbers: [85, 250]. Add sweetness on a scale of 0 to 10 and you have three: [85, 250, 3].

That list of three numbers is a three-dimensional vector. Nothing more mysterious than that. Dimension is the word for "how many numbers are in the list", and each position in the list always means the same property.

Why a list of numbers is useful: it becomes a position

Two numbers can be drawn as a point on a page — the first is how far across, the second how far up. Three numbers place a point in a room: across, up, and how far back.

Once things are points, "similar" becomes "close together". Two coffees measuring [85, 250, 3] and [83, 255, 3] sit almost on top of each other. A cold, huge, very sweet one at [4, 900, 9] sits far away. You have turned a judgement about similarity into arithmetic about distance — and arithmetic is something a computer can do a million times a second.

Beyond three numbers you can no longer picture it. That is a limitation of human visual imagination, not of the maths. The formula for distance between two points works identically for 3 numbers or 768; it just has more terms. Nobody visualises 768 dimensions. Everybody computes with them.

A worked example, small enough to check by hand

Imagine a tiny model that describes any word using exactly three properties, each scored 0 to 1:

Now measure how aligned two of these lists are. Multiply them position by position, add the results, then divide by both lengths so that only direction counts (Chapter 3 explains why that division matters):

cheetah · leopard = (0.90×0.90) + (0.10×0.10) + (0.95×0.80)
                  = 0.81 + 0.01 + 0.76  =  1.58
 ‖cheetah‖ = √(0.90² + 0.10² + 0.95²) = 1.312
 ‖leopard‖ = √(0.90² + 0.10² + 0.80²) = 1.208

 similarity = 1.58 / (1.312 × 1.208) = 0.996   ← nearly identical

cheetah · pizza   = (0.90×0.00) + (0.10×0.95) + (0.95×0.05)
                  = 0.00 + 0.095 + 0.0475 =  0.14
 similarity = 0.14 / (1.312 × 0.951) = 0.114   ← barely related

0.996 versus 0.114. The computer did not read anything or know what a cheetah is. It multiplied and divided nine numbers, and the answer matched human intuition — because the numbers were assigned well.

The leap: the model invents the properties itself

In the toy above, a human chose "animal-ness", "food-ness", "speed-ness". A real embedding model does that part itself. During training it learns which properties are worth measuring so that texts used in similar ways end up with similar numbers.

Two consequences of that, both important:

• Nobody can say what dimension 412 means. The properties are learned, not designed. They do not correspond to words a human would use. Individually they are uninterpretable; collectively they position meaning very precisely.
• Similarity of meaning, not spelling. "What does the trip cost?" and "How much for the tour?" share almost no letters and land close together. "bank of the river" and "bank account" share a whole word and land far apart.

Adiyan uses nomic-embed-text. It takes any text up to 8,192 tokens and returns exactly 768 numbers — the same idea as the toy, with 768 learned properties instead of 3 hand-picked ones.

# mesh/memory/memory_index.py — the actual construction
from llama_index.embeddings.ollama import OllamaEmbedding

self.embed_model = OllamaEmbedding(
    model_name='nomic-embed-text',
    base_url=ollama_url,          # http://localhost:11434
)

One object serves both halves of the system, and that is a hard requirement: the question and the documents must be embedded by the same model. Two different models produce two unrelated coordinate systems, and distances measured between them are meaningless.

Two numbers worth being able to recite

• Storage per vector. 768 numbers × 4 bytes each (a float32) = 3,072 bytes, about 3 KB. Adiyan's 1,601 chunks are therefore roughly 4.9 MB of raw vectors. A million chunks would be about 3 GB — the scale at which compression stops being optional.
• Why 768 and not more. More dimensions means more capacity to distinguish concepts, paid for in storage, memory and compute. 768 is a common middle. nomic-embed-text is additionally trained so its vectors can be truncated to 512, 256 or even 64 numbers with gradual rather than catastrophic quality loss. Adiyan uses all 768.

Chapter 03

Measuring closeness

Chapter 2 turned text into points. "Find relevant text" is now "find nearby points" — but near by which measure? Three candidates matter.

Cosine similarity is the standard for text, and it is what every Adiyan collection uses. The reason is length-independence: a one-sentence answer and a three-paragraph answer on the same topic point in nearly the same direction but have very different magnitudes. Cosine throws the magnitude away and compares direction only — which is to say, meaning only.

# Read live from the running Qdrant instance
adiyan_knowledge_base       size=768  distance=Cosine  points=1601
adiyan_book_pages           size=768  distance=Cosine  points=1306
adiyan_conversation_memory  size=768  distance=Cosine  points=177
adiyan_coaching_memory      size=768  distance=Cosine  points=68

The formula, which you already used by hand in Chapter 2:

cos(θ) = (a · b) / (‖a‖ ‖b‖)

The numerator is the dot product; the denominator divides out both lengths. A corollary that catches people out: if vectors are already normalised to length 1, cosine similarity and dot product are the same number — and dot product is cheaper to compute. Some systems normalise at write time for exactly that reason.

Chapter 04

What a vector database is

You now have points and a way to measure closeness. A vector database is simply a database whose primary job is storing those points and answering the question "what is nearest to this?"

The clearest way to understand it is against the database you already know.

The fundamental difference: you query by example, not by value

A relational database answers questions where you already know what you are looking for:

SELECT * FROM documents WHERE title = 'Refund Policy';
SELECT * FROM orders   WHERE total > 500 AND created_at > '2026-01-01';

Every one of those is an exact test: equal to, greater than, contains this substring. A row either passes or it doesn't. There is no notion of a row being a bit like your query — and critically, if the document is called "Cancellation Terms" rather than "Refund Policy", the first query returns nothing at all, even though it is exactly the document you wanted.

A vector database answers a different shape of question entirely:

# "Here is a point. Give me the 4 stored points closest to it."
client.query_points(
    collection_name='adiyan_knowledge_base',
    query=[0.021, -0.118, 0.334, ...],   # 768 numbers: the question, embedded
    limit=4,
)

You do not tell it what to match. You hand it an example — the question itself, turned into a point — and it returns whatever is nearest, ranked, each with a score. "Cancellation Terms" comes back for a query about refunds because the two sit near each other in meaning-space, even with no shared words.

That last row is the one that bites newcomers, and it is why Chapter 3's score floor exists. SQL returning zero rows is informative — nothing matched. A vector search returning four results tells you nothing by itself, because it would have returned four results regardless. The scores, not the count, carry the information.

Qdrant's data model: three nouns

The payload is where Adiyan keeps source_filename, chunk_index, visibility and owner_identity. It is what makes filtered vector search possible, and therefore what makes access control possible at all.

Actually querying it

Two levels. Directly against the Qdrant client, with a filter:

from qdrant_client import QdrantClient
from qdrant_client.models import Filter, FieldCondition, MatchValue

client = QdrantClient(url='http://localhost:6339')

hits = client.query_points(
    collection_name='adiyan_knowledge_base',
    query=question_vector,                 # 768 floats
    limit=4,
    query_filter=Filter(must=[             # the SQL-like part
        FieldCondition(key='visibility', match=MatchValue(value='global'))
    ]),
).points

for h in hits:
    print(h.score, h.payload['source_filename'])

Or through LlamaIndex, which handles embedding the query string for you — this is what Adiyan actually calls:

# mesh/memory/memory_index.py
retriever = self.kb_index.as_retriever(similarity_top_k=4, filters=filters)
nodes = retriever.retrieve('what does the refund policy say')
# the string is embedded, searched, and matching chunks returned — one call

Do you always need one?

Honestly, no. With 1,601 vectors you could hold a NumPy array in memory and compare against all of them on every query. That is exact brute-force search: perfectly accurate, trivially simple, and fast enough at this scale.

You reach for a real vector database when you need some combination of: persistence across restarts, more vectors than fit comfortably in memory, concurrent writes while serving reads, payload filtering, and search that stays fast as the collection grows. That last one is Chapter 5.

Worth stating plainly: vector and relational databases are complementary, not rivals. Adiyan runs three stores at once — Qdrant for vectors, MongoDB for agent config and registered clients, SQLite for the raw-file index that maps a filename to bytes on disk. Postgres with the pgvector extension can also do both in one engine, which is a reasonable choice when you'd rather not run a second database.

Chapter 05

How it searches fast

Brute force is O(n): ten million vectors means ten million comparisons per question. Making that sublinear requires giving something up.

The trade: approximate nearest neighbour

Stop guaranteeing the exact nearest neighbours; accept the almost certainly nearest ones. The quality measure is recall — of the true top-k, what fraction did the index actually find. Production systems typically tune for 0.95–0.99 recall and take an orders-of-magnitude speedup in exchange.

HNSW, the structure Qdrant uses

Hierarchical Navigable Small World graphs are the dominant approach. Think of an express train network layered over a local one:

Adiyan does not override these; its collections were created by LlamaIndex's QdrantVectorStore with Qdrant's defaults, which at 1,601 points is entirely sensible. Past roughly m=64, ef_construct=512 the returns diminish sharply. If you need more recall in production, ef is the knob to reach for first — it costs latency per query but requires no reindexing.

Chapter 06

Chunking

You cannot embed a 300-page book as one vector. 768 numbers cannot represent 300 pages of distinct meaning; averaging it all together yields a point that is near nothing in particular. So documents get split into chunks, each embedded separately.

The chunk is both the unit of retrieval and the unit of context, and that dual role creates the central tension:

The strategies

• Fixed-size — split every N characters. Trivial and fast; cuts mid-sentence.
• Sentence / recursive — respect sentence and paragraph boundaries, packing up to a size cap. The benchmark-validated default, and what Adiyan uses.
• Structural — split on the document's own markup: markdown headings, HTML sections. Excellent when that structure is genuine.
• Semantic — embed sentences, cut where consecutive similarity drops. Better recall in several benchmarks, but roughly 14× slower to index and prone to producing tiny fragments unless a minimum size is enforced.

Overlap repeats the tail of one chunk at the head of the next, so a fact straddling a boundary survives intact somewhere. Adiyan carries 100 characters on an 800-character chunk — 12.5%. An honest caveat from 2026 benchmark work: overlap is not universally beneficial, and at least one systematic analysis found no measurable gain in its setup while indexing cost rose.

# mesh/memory/memory_index.py — the two-stage split
def _split_text(self, markdown: str, safe_name: str) -> List[str]:
    if not safe_name.lower().endswith(('.md', '.markdown')):
        return self._splitter.split_text(markdown)

    nodes = self._markdown_splitter.get_nodes_from_documents([Document(text=markdown)])
    chunks = []
    for node in nodes:
        chunks.extend(self._splitter.split_text(node.get_content()))  # size cap still applies
    return chunks

The file-extension check is a deliberate judgement: Docling-converted PDFs also emit markdown headings, but those come from extraction heuristics rather than an author's real structure, so they are not trusted as split points. Only genuinely authored markdown takes the structural path.

Chapter 07

Ingestion

Before anything can be chunked it must become text. That is parsing, and it is where most real systems quietly lose data.

Adiyan uses Docling as its general parser — PDFs, images and office formats in, markdown out. One format gets special handling, for a measured reason.

The slide-deck path

Docling's PowerPoint backend does not OCR pictures embedded in slides. For decks exported from design tools — where essentially every slide is one big picture — that means extracting nothing at all. So Adiyan runs a two-tier strategy per slide:

1. Walk the shape tree with python-pptx, recursing into groups, collecting text frames and table cells.
2. If a slide yielded no text, pull out its pictures and run each through Docling as a standalone image to OCR it.
3. If that still yields nothing, log a warning naming the slide — deliberately not silent, so anyone investigating a poor analysis can discover which slide was dropped.

The write path

Once text exists, ingestion is mechanical. Note the delete-before-insert: LlamaIndex's insert() only adds, so re-uploading a document without deleting first leaves stale chunks alongside the new ones with colliding chunk_index values.

# mesh/memory/memory_index.py — ingest_document(), condensed
# 1. clear any existing chunks for this exact key (re-upload safety)
self._qdrant_client.delete(
    collection_name=KB_COLLECTION_NAME,
    points_selector=Filter(must=[FieldCondition(
        key='source_filename', match=MatchValue(value=source_key))]),
)

# 2. chunk, then insert one LlamaIndex Document per chunk
chunks = self._split_text(markdown, safe_name)
for i, chunk in enumerate(chunks):
    self.kb_index.insert(Document(
        text=chunk,
        metadata={
            'source_filename': source_key,     # "<username>/<filename>"
            'chunk_index': i,                  # ordering, for reconstruction
            'ingested_at': timestamp,
            'owner_identity': owner_identity,  # access control
            'visibility': visibility,          # 'private' | 'global'
        },
    ))

That single insert() is doing three things: embedding the chunk via the configured model, generating a point id, and upserting vector plus payload into Qdrant.

Metadata is a design decision, not bookkeeping

Adiyan additionally keeps the original bytes on disk plus a SQLite index row. Vectors can answer "what does it say"; they cannot answer "send me that file back". Different jobs, different storage.

Chapter 08

Retrieval

Retrieval is: embed the question with the same model, ask the index for the k nearest points, return their text. All the engineering lives in the qualifiers.

Choosing k

Adiyan uses KB_DEFAULT_TOP_K = 4 across the whole knowledge base and DOC_SEARCH_DEFAULT_TOP_K = 5 once the search is already narrowed to a single document. The asymmetry is deliberate: with the space already constrained to one document, extra candidates cost little and raise the odds the right passage is included. Too low and the answer may never reach the prompt; too high and you pay tokens while diluting it with noise.

Filtering as an access boundary

This is the most interesting retrieval code in Adiyan. One shared collection holds documents belonging to different people; the filter is what keeps them apart.

# mesh/memory/memory_index.py
def _scope_filters(requester_id: Optional[str]) -> MetadataFilters:
    return MetadataFilters(
        condition=FilterCondition.OR,
        filters=[
            MetadataFilter(key='visibility', value='global',
                           operator=FilterOperator.EQ),
            MetadataFilter(key='owner_identity', value=requester_id or '',
                           operator=FilterOperator.EQ),
        ],
    )

def retrieve_knowledge_base(self, query, top_k=KB_DEFAULT_TOP_K,
                            requester_id=None, is_owner=False):
    filters = None if is_owner else _scope_filters(requester_id)
    retriever = self.kb_index.as_retriever(similarity_top_k=top_k, filters=filters)
    return [n.node.get_content() for n in retriever.retrieve(query)]

Read as policy: a chunk is visible if it is marked global, or if you own it — unless you are the owner, who bypasses filtering entirely. Because Qdrant pre-filters during traversal, the k results returned are k permitted results rather than k results subsequently censored.

Every read path needs the same rule

A subtle point worth raising unprompted: filtering search is not enough. Adiyan also exposes "read this document by exact filename" and "send me the original file". If those paths skip the rule, knowing or guessing a filename defeats the entire boundary. So all six read paths — retrieve_knowledge_base, search_within_document, find_source_document, get_document_text, get_document and list_documents — take the same requester_id and is_owner pair.

Chapter 09

Generation

Retrieval returns strings. Generation places them in a prompt alongside the question, with instructions constraining the model to use them. The entire game is grounding: the answer must derive from retrieved context, not from the model's own parametric memory.

Adiyan assembles context in a fixed three-part order, so the model always sees consistent structure regardless of which parts exist:

# mesh/orchestrator/skills/handle_message.py
context_blocks = [b for b in (memory_context, history) if b]
if context_blocks:
    augmented_text = '\n\n'.join(context_blocks) + f'\n\nNew message: {text}'
else:
    augmented_text = text

Three sources with three different lifetimes: long-term facts (semantic, from mem0), recent turns (an in-process rolling window, relevance-filtered), and the message itself.

The instruction doing the work

Write a short, natural WhatsApp reply based only on what is in this
result - do not invent anything not present in it.

And the reasoning agent carries a strict grounding flag which, when enabled, appends:

...only tool-verified evidence counts as a basis for an answer.
Never state a specific real-world detail (an event, date, price,
availability) unless a tool actually returned it.

Context rot

More context is not monotonically better. Research through 2026 identifies a quality cliff — one analysis places it near 2,500 tokens — beyond which extra retrieved material degrades answers rather than improving them. That is the counter-argument to "just raise k".

Chapter 10

Beyond naive RAG

Everything so far is "naive RAG": embed, search once, generate. Four upgrades matter. Adiyan implements two.

1 · Hybrid search not in Adiyan

Dense vectors are weak on exact tokens — product codes, part numbers, surnames. Sparse lexical retrieval (BM25) is excellent at precisely that and weak at paraphrase. Hybrid runs both and fuses the rankings, usually with Reciprocal Rank Fusion. Qdrant supports sparse vectors natively. This is the most defensible "what would you add next" answer for Adiyan.

2 · Re-ranking a variant, in Adiyan

Retrieve a wide net, then re-score each candidate with a slower but more accurate model — typically a cross-encoder, which reads query and document together instead of comparing pre-computed vectors — and keep the best few.

Adiyan has no cross-encoder, but it does re-rank, on a different axis: recency. Conversation memory over-fetches and reweights by age before truncating.

# mesh/memory/mem0_backend.py
RECENCY_HALF_LIFE_DAYS = 14.0

def _recency_weight(timestamp_str: str) -> float:
    age_days = (datetime.now(timezone.utc) - ts).total_seconds() / 86400
    return 0.5 ** (max(age_days, 0) / RECENCY_HALF_LIFE_DAYS)

# over-fetch 3x, then: final score = similarity × recency_weight
result = memory.search(query, filters={'user_id': contact_name},
                       top_k=max(top_k * 3, 10))

3 · Query transformation not in Adiyan

Rewrite the question before embedding it: resolve pronouns, split multi-part questions, or generate a hypothetical answer and embed that instead (HyDE), on the theory that answers embed closer to answers than questions do.

4 · Agentic RAG core of Adiyan

Naive RAG retrieves exactly once, before generating. Agentic RAG lets a reasoning loop decide whether to retrieve, what to retrieve, and whether to retrieve again after seeing results. Adiyan's Analysis Agent is a ReAct loop capped at 10 steps, with these tools:

The cost is real. One pricing question in testing consumed eight sequential local-model calls across roughly two minutes, and wandered through an irrelevant memory lookup before finding the right document on its second search. Agentic RAG buys multi-hop capability and pays in latency and variance.

The scratchpad pattern

A naive agent loop appends every tool result to a growing conversation and eventually overflows its context window. Adiyan instead maintains a typed scratchpad — findings, documents_checked, open_questions — and after each tool call runs a separate compaction step that folds the new observation into an updated scratchpad, discarding the raw text. The decide-step never sees raw history. That is what lets a 10-step investigation fit inside a 16k context window.

Chapter 11

Evaluation

"It seems to work" is not an evaluation. RAG has two failure surfaces requiring separate measurement, because a perfect generator cannot rescue bad retrieval, and good retrieval does not guarantee a faithful answer.

These four are the RAGAS core; the framework also aggregates them into a single mean and adds extensions such as noise sensitivity and response groundedness. Faithfulness is the hallucination metric — computed by decomposing the answer into atomic claims and checking each against the retrieved context.

Adiyan's own measurement has been case-based rather than framework-based: a fixed eval case with a recorded similarity score, re-measured after the chunking change — 0.496 to 0.665. That is a legitimate regression test for retrieval, and notably it caught a problem no amount of prompt tuning would have fixed.

Chapter 12 · Capstone

Adiyan, end to end

Adiyan is a mesh of agents behind a WhatsApp interface. Its retrieval subsystem is not one pipeline but two distinct memory engines with different content, write paths and retrieval semantics — sharing one Qdrant instance and one embedding model.

The two engines, contrasted

The distinction is the architecturally interesting part: a knowledge base is append-and-retrieve, whereas a memory needs consolidation — deciding whether a new statement adds a fact, updates one, or contradicts one. Adiyan deliberately did not hand-roll the second; it delegated to mem0 and then compensated for that library's known non-deletion behaviour with recency re-ranking.

Full stack inventory

Chapter 13

Incidents & fixes

These are the parts worth telling as stories, because each names a general principle and carries a measured number.

1 · Chunk dilution suppressed a fact that was present

Symptom: a fact sitting verbatim in the index scored 0.496 and was rejected by the 0.55 floor. Cause: heading-blind splitting merged five unrelated sections into one ~3,000-character chunk, averaging the embedding across all of them. Fix: markdown-structural splitting for authored files, re-measured at 0.665. Principle: when a present fact isn't retrieved, suspect chunk composition before touching thresholds.

2 · Unscoped search leaked private data

Symptom: one person's private identity document surfaced in a different requester's results. Cause: retrieval had no notion of ownership; the collection was shared and unfiltered. Fix: visibility and owner_identity payload fields, an OR pre-filter on every read path, an owner bypass, and a migration tagging all 1,601 pre-existing chunks as owner-private. Principle: multi-tenant vector search needs a filter as a security boundary, applied to every read path rather than just the search one.

3 · A stale fact outranked its own correction

Symptom: a corrected preference lost to the statement it superseded, 0.725 against 0.781. Cause: the memory store keeps corrections as new entries without deleting originals — and a correction phrased "crimson, not teal" contains the superseded value as a literal token, which drags its embedding back toward the stale fact, because dense embeddings do not represent negation (cos("teal", "not teal") = 0.86; an unrelated word scores 0.39). Fix: over-fetch threefold and multiply similarity by an exponential recency weight. Principle: pure similarity has no concept of time, truth, or negation. Anything mutable needs a temporal signal in the ranking.

4 · Silent extraction produced fabricated analysis

Symptom: a slide deck was confidently "analysed" from nothing. Cause: the PowerPoint parser does not OCR embedded images, so image-only slides extracted zero text with no error raised. Fix: per-slide two-tier extraction with explicit warning logs on total failure. Principle: a parse returning empty is not a no-op — it is a corrupted index that fails later, elsewhere, as confident prose.

5 · A permission gap silently disabled retrieval for every client

Symptom: a registered user's question produced no reply whatsoever. Cause: the default permission tier granted knowledge-base search but never the general-reasoning fallback, so every message routed there failed authorisation and was dropped silently. Fix: grant the missing capability. Principle: a retrieval subsystem is only as available as the authorisation layer in front of it, and silence is the worst possible error surface.

Chapter 14

Ready reckoner

Short answers to the questions this system gets asked most, each anchored to something real that happened while building it.

Appendix

References

Fundamentals in Chapters 2–6 and 11 were checked against these sources. Everything in Chapters 7–10 and 12–13 comes from Adiyan's own source, verified against its live Qdrant instance and installed package versions.

• nomic-ai/nomic-embed-text-v1.5 — model card · dimensions, Matryoshka truncation, 8192-token context
• Nomic Embed Matryoshka · variable-dimension embeddings
• Qdrant — HNSW indexing fundamentals · graph structure and traversal
• Qdrant — indexing documentation · m and ef_construct defaults
• Tuning Qdrant HNSW: ef, m, exact search · recall and latency trade-offs
• Ragas — available metrics · the four core metrics and extensions
• RAG evaluation metrics — Confident AI · faithfulness as claim decomposition
• RAG chunking strategies: the 2026 benchmark guide · recursive baseline, semantic chunking cost
• Best chunking strategies for RAG (2026) · strategy comparison
• LlamaIndex — Nomic embedding integration