Embeddings and vector search for AI exams
An embedding is a list of numbers, produced by a model, that represents the meaning of a piece of text, an image or other content, so that similar items end up close together. Vector search embeds the query with the same model and returns the stored items nearest to it. The exam decision rule: use vector search when the user’s words differ from the document’s words, use keyword search when exact tokens matter (part numbers, names, codes), and use hybrid search when both kinds of query occur. This guide goes layer by layer, from the numbers up to the index.
Layer 1: the embedding
An embedding model maps content to a fixed-length vector, for example a few hundred to a few thousand numbers. The length (the dimensions) is set by the model, and the index must be created with the same number. Microsoft’s Azure AI Search documentation describes the effect: “dog” and “canine” are linguistically different but conceptually close, and “dog” in English and “hund” in German can match when the model is multilingual. Databricks AI Search (the new name of Databricks Vector Search) describes embeddings the same way, as numerical representations of the semantic content of data.
Two rules follow, and exam questions use both:
- Same model for documents and queries. Vectors from different models live in different spaces and cannot be compared. Changing the embedding model means re-embedding everything.
- Embeddings are generated, not searched for. The model call is a separate step, either in an ingestion pipeline (integrated vectorization in Azure AI Search, a Delta Sync index in Databricks) or in your own code.
Layer 2: measuring “close”
The index needs a similarity metric. Azure AI Search supports cosine, dotProduct and euclidean, plus hamming for binary vectors.
| Metric | Measures | Note |
|---|---|---|
| Cosine | The angle between two vectors, ignoring length | Metric used by Azure OpenAI embedding models, so choose cosine with them |
| Dot product | Angle and length together | Identical to cosine for normalized vectors, slightly faster |
| Euclidean (L2) | Straight-line distance | Databricks AI Search uses L2 and says to normalize embeddings if you want cosine behaviour |
A small calculation shows the difference. Take a = (1, 2, 2) and b = (2, 4, 4), which points the same way but is twice as long. Cosine similarity is 1.0 (identical direction), Euclidean distance is 3.0 and the dot product is 18. Take c = (2, 1, -2): cosine with a is 0.0 (perpendicular, unrelated). Cosine says a and b are the same meaning, Euclidean says they are 3 apart. For normalized vectors all three metrics give the same ranking, which is why exam answers often say “normalize, then it does not matter”.
Azure AI Search does not return the raw cosine value as @search.score. It returns 1 / (1 + cosine distance), a transformed score that decreases as similarity gets worse.
Layer 3: finding neighbours fast
Comparing the query with every stored vector gives the exact answer, called exhaustive KNN (k nearest neighbours). It is correct and grows linearly with data, so Microsoft recommends it for small or medium data sets, or when precision matters more than speed, and for building a ground-truth set to measure how good an approximate method is.
Approximate nearest neighbour (ANN) algorithms trade a little accuracy for large speed gains. Both Azure AI Search and Databricks use HNSW (Hierarchical Navigable Small World), a layered graph that the query walks from coarse to fine. HNSW keeps its data points in memory, which counts against the vector index quota in Azure AI Search. Tunable parameters include m (neighbours per node), efConstruction (candidates considered while building) and efSearch (queue size while querying). A field indexed for HNSW can still be queried exhaustively with "exhaustive": true, but a field indexed only for exhaustive KNN cannot use HNSW.
The exam shorthand: ANN is fast and approximate, KNN is slow and exact, and recall is the share of the true nearest neighbours that ANN actually returns.
Layer 4: hybrid search and rank fusion
Hybrid search sends one request with both a keyword query (scored by BM25) and a vector query. They run in parallel and a single ranked list is produced by Reciprocal Rank Fusion (RRF): each item scores the sum of 1 / (k + rank) over the lists it appears in. Databricks sets the constant k to 60 in its hybrid mode and then normalises the scores. A semantic ranker can then re-score the top results; in Azure AI Search this is optional, and Microsoft reports that, in its benchmark testing, hybrid retrieval with the semantic ranker gave significant relevance gains.
Worked example with k = 60. The keyword search ranks documents d3, d1, d2; the vector search ranks d1, d2, d3.
| Document | Keyword rank | Vector rank | RRF score |
|---|---|---|---|
| d1 | 2 | 1 | 1/62 + 1/61 = 0.03252 |
| d3 | 1 | 3 | 1/61 + 1/63 = 0.03227 |
| d2 | 3 | 2 | 1/63 + 1/62 = 0.03200 |
d1 wins by being high on both lists, even though d3 was first on keywords. RRF uses ranks, not raw scores, so it does not matter that BM25 and cosine scores are on different scales.
Filters combine with vector queries. Azure AI Search can apply a metadata filter before or after the vector query (preFilter, postFilter, or the preview strictPostFilter), and this is how multi-tenant or per-category retrieval is done in RAG. Access control still has to be designed: a filter on a group field is the usual way to keep users from retrieving chunks they should not see, similar in spirit to row-level security.
Which exams test this
| Exam | How it appears | On this site |
|---|---|---|
| DP-800 | Implement AI capabilities in database solutions (25–30%): generating and maintaining embeddings, choosing what to chunk and embed, the vector type and vector indexes, ANN versus KNN, index types and metrics, full-text versus vector versus hybrid search, and reciprocal rank fusion. Applies to the outline from October 19, 2026; the change log lists only minor changes to these areas | AI capabilities |
| AI-200 | Develop AI solutions by using Azure data management services (25–30%): storing embeddings and running vector similarity search in Cosmos DB for NoSQL and in PostgreSQL with pgvector, tuning pgvector indexes, and vector indexing in Azure Managed Redis | Data management |
| AI-103 | Implement information extraction solutions (10–15%): configuring semantic, hybrid and vector search for grounding; Plan and manage an Azure AI solution (25–30%): choosing the Foundry service for vector search and the retrieval and indexing method | Information extraction |
| AI-300 | Optimize generative AI systems and model performance (10–15%): tuning similarity thresholds, selecting and fine-tuning embedding models, and hybrid search that combines semantic and keyword retrieval | GenAI optimization |
| Databricks GenAI Engineer | Application Development (30%): choosing an embedding model’s context length; Assembling and Deploying Applications (22%): creating, querying and configuring a Mosaic AI Vector Search (now called Databricks AI Search) index | Design and data preparation |
| AIF-C01 | Fundamentals of GenAI (24%): embeddings, vectors and chunking as core terms; Applications of Foundation Models (28%): AWS services that store embeddings in vector databases (OpenSearch Service, Aurora, Neptune, RDS for PostgreSQL) | Foundation model applications |
| MLA-C02 | Data Preparation for ML and AI (28%): configuring scalable vector databases and using embedding models to turn text and images into vectors; ML Model and FM Development (24%): optimizing retrieval components and embedding models | Data preparation |
| CY0-001 | Basic AI Concepts Related to Cybersecurity (17%): objective 1.2 names RAG with vector storage and embeddings under data security for AI. A minor topic, seen from the security angle | CY0-001 basic AI concepts related to cybersecurity explained |
How questions are built around vector search
The scenario names the query type and the failure. “Users search with different wording than the documents” points to vector search. “Engineers type an exact code like FLT-2291 and get nothing relevant” points to keyword or hybrid search, since vector search blurs exact tokens. “Both kinds of queries” means hybrid. “Results are slightly worse after switching to an approximate index” is the ANN recall trade-off, fixed by raising search parameters or using exhaustive search. “Search stopped working after we changed the embedding model” means the index was not rebuilt. Wrong chunking is another distractor: before changing the index type, check that chunks are neither multi-topic nor cut mid-sentence, as covered in RAG explained and fine-tuning versus RAG versus prompting.
Question 1. A parts catalogue search must handle both exact part numbers such as FLT-2291 and natural-language questions such as why does the filter clog in cold weather. Which retrieval design fits best?
- A. Keyword search only
- B. Vector search only, with a larger embedding model
- C. Hybrid search combining keyword and vector queries
- D. Exhaustive KNN on a vector field only
Show answer
Answer: C
Hybrid search runs keyword and vector queries together and fuses the results with reciprocal rank fusion, so exact tokens and paraphrased questions are both covered. Vector search alone can blur exact codes. Keyword search alone misses paraphrases. A larger embedding model does not make a vector search exact on identifiers.
Question 2. A team indexed one million document chunks with embeddings from model X, and queries are embedded with model Y that has the same number of dimensions. Results look random. What is the cause?
- A. HNSW is too approximate and should be replaced by Euclidean distance
- B. The query and document vectors come from different embedding models
- C. Cosine similarity cannot be used with one million vectors
- D. Hybrid search was not enabled
Show answer
Answer: B
Documents and queries must be embedded by the same model, because each model defines its own vector space; equal dimension counts do not make the spaces comparable. Re-embed the content or the queries with one model. Switching from cosine to Euclidean cannot fix mismatched spaces, HNSW recall loss would be small rather than random, and adding BM25 does not repair the vector side.
Go deeper
- Vector search overview (Azure AI Search, Microsoft Learn)
- Vector relevance and ranking (Azure AI Search, Microsoft Learn)
- Hybrid search overview (Azure AI Search, Microsoft Learn)
- Databricks AI Search, formerly Vector Search (Databricks documentation)
On this site: AI agents explained for what consumes retrieval results, LLM evaluation for measuring retrieval quality, and how to study for certification exams.