Databricks Data Engineer: governance and troubleshooting
Governance and Security (15%) and Troubleshooting, Monitoring, and Optimization (10%) are 25% of the Databricks Data Engineer Associate exam between them — around 12 of the 45 scored questions.
Governance and security (15%)
Almost entirely Unity Catalog.
The object hierarchy. Catalog → schema → table or volume. Permissions are granted at each level and inherit downward, which is why granting broadly at catalog level is usually the wrong answer.
Access control. Granting and revoking privileges to users and groups, and applying least privilege. Groups over individuals — a scenario about managing access for a team wants a group.
Fine-grained control. Restricting access to specific columns or rows rather than whole tables. The recurring scenario: an analyst needs a table but must not see one sensitive column. The answer is column-level control or masking, never a separate copy of the table — copies drift and multiply the governance problem.
Lineage. What a table was built from and what depends on it. The answer whenever a question asks about impact analysis before a change, or tracing where data came from.
Sensitive data handling — identifying it, masking it, and controlling who can see it.
Troubleshooting, monitoring and optimisation (10%)
About five questions, and they reward having debugged something real.
Diagnosing failures. Reading a job run’s history to find which task failed and what the error says. Distinguishing a transient failure from a systematic one — the first justifies a retry, the second does not.
Common performance problems:
| Problem | Symptom | Remedy |
|---|---|---|
| Small files | Reads slow, overhead dominates | Compact into fewer, larger files |
| Data skew | A few tasks take far longer than the rest | Redistribute the skewed key |
| Unnecessary shuffles | Large data movement between stages | Restructure joins and aggregations |
| Over-partitioning | Many tiny partitions | Partition on lower-cardinality columns |
The small file problem is the most examined. It arises naturally from frequent incremental writes, and compaction is the remedy.
Cost awareness. Right-sizing clusters, terminating idle compute, and not solving a data layout problem by buying more machines. When a question offers “use a larger cluster” alongside a structural fix, the structural fix is almost always correct.
Monitoring. Job success and failure over time, runtime trends, and knowing when something has quietly degraded.
Sample questions
Question 1. A team of eight analysts needs read access to twelve tables in one schema, and membership will change as people join and leave. What is the appropriate approach?
- A. Grant select to each analyst individually on each table
- B. Grant select to a group on the schema, and manage group membership
- C. Grant select to all users at catalog level
- D. Create copies of the twelve tables in an analyst-owned schema
Show answer
Answer: B
Granting to a group and managing membership keeps permissions stable as people change, which is exactly the stated condition. Granting to each user individually creates ongoing maintenance, granting at catalog level over-permissions beyond the twelve tables, and copies of tables duplicate data and drift.
Question 2. One stage of a job consistently has a few tasks running far longer than the rest, while most finish quickly. What does this indicate?
- A. Data skew — some keys hold far more data than others
- B. The small file problem
- C. An undersized cluster
- D. A missing index on the table
Show answer
Answer: A
A small number of long-running tasks alongside many fast ones is the signature of data skew, where one key holds disproportionate data. Small files slow reads broadly rather than concentrating in a few tasks, an undersized cluster slows everything evenly, and a missing index is not a Delta concept here.
Question 3. Before dropping a column from a widely used silver table, an engineer must find out what downstream tables and dashboards depend on it. Which capability provides this?
- A. Delta Lake time travel
- B. OPTIMIZE on the table
- C. Unity Catalog lineage
- D. Job run history
Show answer
Answer: C
Unity Catalog lineage shows what a table was built from and what consumes it, which is what impact analysis before a change requires. Time travel reads previous versions, OPTIMIZE compacts files, and job run history shows executions rather than dependencies.
What to practise
Grant a permission to a group, then restrict a single column and try to read it as a limited user. Open the lineage view for a table you built and trace it back to source.
Then create the small file problem deliberately — write in a loop — and time a query before and after compaction. That contrast teaches the optimisation section faster than reading about it, and it connects directly to the ingestion section, where small files originate.