Databricks DE Professional: ingestion, quality, modelling
Data Ingestion & Acquisition (7%), Data Transformation, Cleansing, and Quality (10%) and Data Modelling (6%) are 23% of the Databricks Data Engineer Professional exam between them — around 14 of the 59 scored questions.
Note how small ingestion has become. On the Associate it is 21%; here it is 7%. The Professional assumes you can load data and asks harder questions about what happens next.
Ingestion and acquisition (7%)
About four questions, pitched above Associate level:
- Incremental patterns as the default, not the optimisation.
- Handling source schema changes deliberately rather than failing or silently accepting.
- Reprocessing safely — being able to rebuild a target from source without duplicating.
- Acquiring from varied sources: files, streams, databases, external systems.
- Handling out-of-order and duplicate arrivals at source.
Transformation, cleansing and quality (10%)
The step up from the Associate is enforcement. It is not enough to clean data; you must validate that it is clean and act when it is not.
Cleansing:
- Deduplication on a business key, and choosing which record survives.
- Null handling as a deliberate decision with documented consequences.
- Standardising formats, types and reference values.
- Late-arriving data — records for a period already processed.
Quality:
- Defining expectations: this column is never null, this value is within range, this key is unique.
- Deciding what happens on violation — fail the pipeline, quarantine the rows, or warn and continue. Each is right in different circumstances, and the exam expects you to match the response to the stakes.
- Recording quality metrics over time, so degradation is visible.
The scenario to recognise: a pipeline that succeeds while producing wrong data. Job success proves the code ran. Only quality validation proves the output is right.
Data modelling (6%)
About four questions, and the most classical content on the exam:
- Dimensional modelling on the lakehouse: facts and dimensions, and choosing the grain of a fact table.
- Slowly changing dimensions, particularly the distinction between:
- Type 1 — overwrite; history is lost.
- Type 2 — add a new row with validity dates; history is preserved, so historical facts join to the values current at the time.
- Normalisation versus denormalisation, and why analytical models often denormalise.
- How the medallion layers relate to a dimensional model — gold is usually where it lives.
Type 2 dimensions are reliably tested. Any scenario requiring point-in-time correctness — “the price as it was when the order was placed” — is a type 2 question.
Sample questions
Question 1. A nightly pipeline reports success every run, but a finance metric has been understated for three weeks. What control was missing?
- A. Cluster resource monitoring
- B. Data quality expectations validated against the output, with alerting on violations
- C. Longer log retention on the job
- D. A larger cluster to avoid resource contention
Show answer
Answer: B
Job success confirms only that the code executed, so data quality expectations validated against the output are what would have caught an incorrect metric. Cluster monitoring, longer retention and a bigger cluster address unrelated concerns.
Question 2. A customer's assigned sales region changes. Historical orders must continue to report under the region that applied at the time of the order. Which dimension design is required?
- A. Type 1 slowly changing dimension
- B. Store the region directly on each order row and never update it
- C. Delete and recreate the dimension each night
- D. Type 2 slowly changing dimension with validity dates
Show answer
Answer: D
A type 2 slowly changing dimension inserts a new row with validity dates on change, so historical facts join to the values current when they occurred. Type 1 overwrites and loses history, storing the region on the fact table denormalises without solving dimension management, and deleting history is the opposite of the requirement.
Question 3. Records occasionally arrive two days late for a period that has already been aggregated into the gold layer. What approach handles this correctly?
- A. Reprocess the affected periods and merge the corrected aggregates
- B. Discard records arriving after the period has been processed
- C. Append late records directly to the existing aggregate
- D. Delay all processing by three days so nothing is late
Show answer
Answer: A
Reprocessing the affected period and merging the corrected results keeps aggregates accurate without duplicating, which is what late-arriving data requires. Discarding late records loses data, appending them to the aggregate double-counts, and delaying all processing harms freshness for everything.
What to practise
Define three quality expectations on a table you own and deliberately violate each, choosing a different response for each violation — fail, quarantine, warn. Then build a type 2 dimension and run a point-in-time join against it.
Those two exercises cover the most examined ideas across all three sections.