Databricks ML Professional: ML Ops
ML Ops is worth 44% of the Databricks Machine Learning Professional exam — around 26 of the 59 scored questions, tied with model development as the largest section.
It has no counterpart on the ML Associate, and it is the single clearest reason the Professional expects a year of experience rather than six months. Most of this material describes problems that only appear after a model has been live for months.
The founding idea
A machine learning system degrades on its own.
Conventional software does what it did yesterday until someone changes it. A model’s quality depends on the world matching its training data, and the world moves. That makes monitoring and retraining structural requirements rather than good practice.
A corollary the exam returns to repeatedly: infrastructure monitoring tells you nothing about model quality. A degraded model is up, fast, and answering — just wrong. Green dashboards prove nothing.
Monitoring
Three layers, and the exam distinguishes them:
| Layer | Answers |
|---|---|
| Infrastructure | Is the service healthy? |
| Pipeline | Did the job run, and how long did it take? |
| Model | Are the predictions still any good? |
For the model layer, monitor:
- Input distributions — have the features shifted?
- Prediction distributions — is the model’s output mix changing?
- Outcome quality, where ground truth eventually arrives, though often with a delay.
- Segment performance, since degradation often starts in one group.
None of this works without a baseline captured at training time. You cannot detect a shift from normal without a record of normal.
Drift
The distinction most likely to appear:
Data drift — the input distribution moves. Your customers change, a source system starts formatting a field differently, a new region is added.
Concept drift — the relationship between inputs and the correct answer changes. Inputs may look identical, but what they imply has shifted: behaviour changes, fraud patterns adapt, a market moves.
Scenario language to watch for: “feature distributions are unchanged but accuracy has fallen” is concept drift. “the input data now looks different” is data drift.
Both are gradual. Anything that degrades immediately on deployment is not drift — it is more likely leakage or a training/serving mismatch.
Retraining
- Scheduled retraining on a fixed cadence: simple, predictable, sometimes wasteful.
- Triggered retraining when monitoring detects degradation: responsive, but requires monitoring you trust.
- What data to retrain on — a rolling window, or everything.
The control candidates most often miss: a retrained model must be validated against the incumbent before promotion. Newer is not automatically better. An automated pipeline that promotes every retrained model without comparison will eventually promote a worse one.
Versioning and lifecycle
- The model registry as the record of what exists and what is live.
- Promotion through stages, with approval where appropriate.
- Rollback — being able to return to the previous version quickly.
- Logging which version served each prediction, which is what makes a historical prediction explainable months later.
- Governance in Unity Catalog: ownership, access and lineage.
Automation
The full loop: monitor → detect → retrain → validate → promote → monitor. Automating it without losing the validation gate is the professional-level skill.
Sample questions
Question 1. A model's input feature distributions have shifted noticeably over six months, though accuracy has not yet fallen. What is occurring, and what is the appropriate response?
- A. Data drift; investigate and prepare to retrain before accuracy degrades
- B. Concept drift; no action is required while accuracy holds
- C. Normal variation; no monitoring response is warranted
- D. An infrastructure problem; scale the serving endpoint
Show answer
Answer: A
Shifting input distributions is data drift, and the response is to investigate and prepare to retrain before quality degrades rather than waiting for it to. Concept drift involves a changed input-to-outcome relationship, no action ignores a leading indicator, and scaling the endpoint addresses capacity rather than data.
Question 2. An organisation monitors endpoint latency, error rate and cluster health. A model produced materially wrong predictions for seven weeks before anyone noticed. What was missing?
- A. More granular latency percentiles
- B. Longer log retention on the endpoint
- C. A larger serving endpoint
- D. Model quality monitoring — prediction and input distributions against a training baseline
Show answer
Answer: D
Infrastructure and endpoint monitoring cannot detect a model that is healthy but wrong, so model-quality monitoring against a baseline is the missing layer. More latency detail, longer log retention and a larger endpoint all address the layer that was already working.
Question 3. An automated pipeline retrains nightly and promotes the new model to production each time. Occasionally production quality drops sharply for a day. What should be added?
- A. Retrain twice daily so bad models are replaced faster
- B. A validation gate comparing the new model against the incumbent before promotion
- C. A larger cluster for the retraining job
- D. Remove automation and retrain manually
Show answer
Answer: B
A validation gate comparing the retrained model against the incumbent on a consistent holdout prevents a worse model reaching production. Retraining more often increases exposure to the problem, a larger cluster affects training speed, and removing automation discards the benefit instead of adding the missing control.
What to practise
Take a model you have in production, or simulate one, and answer four questions in writing:
- How would I know it has degraded?
- How would I tell data drift from concept drift?
- What stops a worse retrained model reaching production?
- Which version served the prediction made last Tuesday?
If any answer is “someone would notice eventually”, you have found precisely what this section tests — and it is 44% of the exam.