Spark Developer Associate cheat sheet
Last-minute reference for the Databricks Associate Developer for Apache Spark exam.
Exam facts
| Questions | 45 scored |
| Duration | 90 minutes |
| Fee | 200 USD |
| Language | Python |
| Passing score | Not published by Databricks |
Section weights
| Section | Weight |
|---|---|
| DataFrame / DataSet API applications | 30% |
| Spark architecture and components | 20% |
| Using Spark SQL | 20% |
| Troubleshooting and tuning | 10% |
| Structured Streaming | 10% |
| Spark Connect | 5% |
| Pandas API on Spark | 5% |
Transformations vs actions
Transformations are lazy — select, filter, withColumn, join. They build the plan.
Actions trigger execution — count, collect, show, write.
collect() brings every row to the driver and can exhaust its memory.
Architecture
- Driver — hosts the SparkSession, plans the job, coordinates tasks
- Executors — do the work
- Stage boundary — wherever a shuffle is required
- Adaptive query execution — fixes bad static estimates at runtime: partition counts, join strategy, skew
Joins
| Join | Returns |
|---|---|
| Inner | Only matching keys |
| Left outer | All left rows, nulls where unmatched |
| Left semi | Left rows that have a match, left columns only |
| Left anti | Left rows with no match |
| Broadcast | Small side sent to every executor; removes the shuffle |
Optimisations that happen for you
- Predicate pushdown — filters applied at the source
- Column pruning — only selected columns read from columnar formats
- Partition pruning — whole partitions skipped by a partition-column filter
Troubleshooting
- Three tasks dominate a stage → data skew
- Heavy spill during shuffle → tune shuffle partition count so partitions fit in memory
- Repeated expensive recomputation across actions → persist or cache
- Row-at-a-time Python UDF slow → built-in expressions
Streaming
- Resume without reprocessing → checkpoint location
- Late events, bounded state → watermark on the event time column
Other
- Spark Connect — client sends a logical plan over a connection instead of running in the driver process
- Pandas API on Spark — pandas syntax, distributed execution
Night-before checklist
- Which operations are actions
- Where a stage boundary comes from
- The five join types
- Check ID and proctoring rules — see exam day
Take the 20-question practice test.