Databricks GenAI application development

Application Development is worth 30% of the Databricks Generative AI Engineer Associate exam — the largest section, around 14 of the 45 scored questions. This is where you build the thing: the chain, the retrieval, the prompts and the behaviour when something goes wrong.

The RAG pattern, end to end

Almost everything in this section is a variation on one pipeline:

A user asks a question.
The question is embedded and matched against the index.
The top matching chunks are retrieved.
Those chunks are inserted into a prompt alongside the question and instructions.
The model answers from that context.
The response is returned, often with sources.

Know each step well enough to say what goes wrong when it fails, because that is how questions are framed.

Retrieval

How many chunks to retrieve. Too few and the answer lacks context; too many and you dilute the prompt and pay for tokens that add nothing.
Relevance versus recall — getting the right content versus getting all the possibly-right content.
Filtering by metadata, so retrieval respects source, date or permission.
Handling the case where nothing relevant is found. An application that hallucinates rather than saying it does not know is a design failure.

Prompt construction

System instructions for standing behaviour: tone, language, scope, and what to do when context is insufficient.
Inserting retrieved context clearly, so the model can tell content from instruction.
Asking for citations where sources matter.
Managing the context window — retrieved content, conversation history and instructions all compete for the same budget.

The instruction worth having in every RAG system prompt: answer only from the provided context, and say so if the context does not contain the answer. It is the cheapest hallucination control available.

LLM chains

Multi-step pipelines where one step’s output feeds the next:

Retrieval then generation, the basic chain.
Query rewriting before retrieval, so a conversational question becomes a good search query.
Multi-step reasoning, where a task is decomposed.
Post-processing, such as formatting or validating output.

Chains introduce a failure surface: a step that silently returns nothing will produce a confident, wrong answer downstream.

Tools and agents

Tool or function calling, so the model can do something rather than only answer.
Deciding which tools an application should expose.
Controls around consequential actions — anything irreversible needs a human approval step.
Handling tool failures gracefully.

Handling failure

Questions frequently describe something going wrong. Know the standard responses:

Failure	Response
Retrieval returns nothing relevant	Say so; do not let the model invent an answer
Retrieved context exceeds the window	Reduce chunks retrieved, or summarise
A tool call errors	Handle it in the chain; do not pass the error to the model as fact
The model ignores instructions	Strengthen the system prompt; separate data from instructions
Answers are fluent but wrong	Look at retrieval first, not the model

Sample questions

Question 1. A RAG assistant confidently answers questions its document corpus does not cover. What is the most appropriate fix?

A. Switch to a larger, more capable model
B. Retrieve twenty chunks instead of five
C. Instruct the model to answer only from the provided context and to say when it cannot
D. Increase the temperature so answers are more varied

Show answer

Answer: C

Instructing the model to answer only from the provided context and to state when the context is insufficient directly addresses answering beyond the corpus. A larger model is just as capable of inventing an answer, more retrieved chunks does not help when nothing relevant exists, and raising temperature increases variability.

Question 2. In a multi-turn conversation, users ask follow-up questions like 'and what about the second one?'. Retrieval returns poor results for these. What should the chain add?

A. A query rewriting step that makes the follow-up self-contained before retrieval
B. A larger document index
C. A lower temperature setting on the model
D. Retrieval of more chunks per query

Show answer

Answer: A

A follow-up phrased conversationally makes a poor standalone search query, so rewriting it into a self-contained query using conversation history before retrieval is the standard fix. A larger index does not help a bad query, lowering temperature affects generation, and more chunks returns more irrelevant content.

Question 3. An agent built on the platform can issue account credits as a tool action. What control belongs in the design?

A. Log every credit issued for later review
B. Automatically retry the action if it fails
C. A stronger system prompt telling the agent to be careful
D. A human approval step before the credit action executes

Show answer

Answer: D

An irreversible financial action requires human approval before it executes, which is the standard control for consequential agent actions. Logging records the action afterwards, retrying repeats it, and a stronger prompt does not enforce anything.

What to practise

Build a chain with query rewriting, retrieval and generation, then break each step in turn: return no chunks, return irrelevant chunks, and remove the “answer only from context” instruction. Watching how each failure manifests in the final answer teaches this section faster than any reading, and it is exactly the diagnostic reasoning the questions test.