Databricks ML Professional cheat sheet

Last-minute reference for the Databricks Machine Learning Professional exam. Half of it is operations.

Exam facts

Requirement	Answer
Roll back within minutes	Registry retains the previous version + tested rollback
Labels arrive 30 days late	Proxy monitoring now, true metrics on arrival
Validate on live traffic without risking all users	Canary or shadow deployment
Automated retraining promoted a worse model	Promotion gate vs incumbent on a held-out set
Reconstruct a prediction six months later	Model version, served features and configuration for that request
Two teams break each other’s pipelines	Schema and semantics contract enforced in CI
Cost jumped after promotion	Serving instance type, min replicas, autoscaling bounds
Prevent unreviewed changes shipping	Code review and tests as mandatory gates

Feature available at training but three days late in production → cannot serve as evaluated
Predictions diverge from validation → training-serving skew; one shared definition
Monthly rolling window retrain → parameterise the window, record the resolved data version
Strong aggregate, fails one segment → evaluate per segment
Two teams report different accuracy → different data, splits or metric definitions
Ensemble gains 0.3% at triple cost → decide explicitly against business value; default simpler
Reproducibility needs seeds, library versions, data version and non-deterministic ops together

Sub-100ms with feature-store lookups → online feature lookup latency is the risk
Nightly 80M batch plus low-latency single requests → separate serving paths, same registered model version

Input distribution change versus a change in the input-to-target relationship. The second is only visible once labels arrive.

No passing score is published. Calibrate on a weighted practice test.