The problem: models do not know your data
A language model was trained on public text from the internet. It does not know your organisation's policies, your project documentation, your client correspondence, or your internal procedures. If you want the model to work with your specific knowledge, you must give it access. There are two fundamental approaches: you can embed your documents into a vector index and retrieve relevant passages based on semantic similarity, or you can maintain your documents as searchable text and retrieve relevant sections through keyword and pattern matching.
Semantic retrieval (vector-based)
In semantic retrieval, your documents are split into chunks and each chunk is converted into a numerical representation (an embedding) that captures its meaning. These embeddings are stored in a vector database. When you ask a question, your question is also converted into an embedding, and the system finds the document chunks whose embeddings are most similar to your question's embedding. These chunks are then inserted into the context window alongside your prompt, so the model can answer based on your actual data. This approach works well for fuzzy, conceptual questions: "what is our policy on remote work?" finds relevant passages even if they use different words than your question.
Structured text retrieval (search-based)
In structured text retrieval, your documents remain as plain text files, and a search tool finds relevant passages using keyword matching, regular expressions, or file-path patterns. This is the approach used by code-editing environments that index your entire codebase and let an agent search through it with precise queries. It works well for exact, specific lookups: "find every mention of clause 7.3 in the contract folder" or "find all files that reference the client name Müller". The advantage is precision and transparency. You can see exactly what was found and why. The disadvantage is that it requires the searcher (you or the agent) to know roughly what to look for.
Combining both approaches
The most effective environments use both strategies. Semantic retrieval handles broad, exploratory questions where you do not know the exact wording. Structured retrieval handles precise, targeted lookups where you know what you are looking for. A well-designed environment indexes your data both ways: a vector index for semantic search and a text index for keyword search. The agent, or the workflow, decides which retrieval method to use based on the nature of the query.
What gets indexed matters
The quality of retrieval depends entirely on what you put into the index. If your documents are poorly structured, full of duplicates, or missing metadata, retrieval will return noise. If your documents are well-organised, with clear headings, consistent naming, and relevant metadata, retrieval will return precisely what the model needs. Indexing is not a technical afterthought; it is the foundation of every effective AI workflow. The time you spend curating and structuring your data is the single best investment you can make in the quality of your AI-assisted work.
Examples
Semantic vs. keyword retrieval
You search your policy documents for information about parental leave. A keyword search for "parental leave" finds documents that use that exact phrase. A semantic search finds those documents plus a section titled "family-related absence entitlements" that covers the same topic using different words. Both results are useful; neither alone is complete.