Free Spark Developer Associate sample questions
Try these before reading the answers. The mix follows the weighting, so the DataFrame API, architecture and Spark SQL appear most.
Question 1. Which of these transformations causes a shuffle?
- A. filter
- B. select
- C. groupBy
- D. withColumn
Show answer
Answer: C
groupBy requires records with the same key to be co-located, so data moves across the cluster, making it a wide transformation. filter, select and withColumn each operate within a partition without moving data and are narrow transformations.
Question 2. A job joins a 3 TB table with a 50 MB lookup table and spends most of its runtime shuffling. What is the most effective optimisation?
- A. Increase the number of executors
- B. Broadcast the 50 MB lookup table
- C. Repartition the 3 TB table on the join key
- D. Cache the 3 TB table before joining
Show answer
Answer: B
Broadcasting the small table sends a copy to every executor, eliminating the shuffle of the large table, which is what dominates runtime. A larger cluster shuffles the same volume at greater cost, repartitioning the large table adds a shuffle, and caching consumes memory without removing the join shuffle.
Question 3. A developer writes five chained transformations and then calls count(). When does Spark execute the work?
- A. As each transformation is written
- B. When the DataFrame is first created
- C. When the Spark session is closed
- D. When count() is called, since transformations are lazy
Show answer
Answer: D
Spark transformations are lazy and build a plan, and execution is triggered only when an action such as count() runs. The work does not happen as each transformation is written, nor at DataFrame creation, and it is not deferred until the session closes.
Question 4. A transformation applies a simple uppercase operation to every row using a Python UDF. Performance is poor on a large DataFrame. What is the best change?
- A. Replace the UDF with the built-in upper() function
- B. Increase the cluster size
- C. Cache the DataFrame before applying the UDF
- D. Repartition into more partitions first
Show answer
Answer: A
A Python UDF serialises data between the JVM and Python for every row, which is the source of the slowdown, and a built-in Spark SQL function performs the same work natively without that overhead. A larger cluster pays more for the same inefficiency, caching does not remove per-row cost, and repartitioning multiplies overhead.
Question 5. In a Structured Streaming application, what is the purpose of checkpointing?
- A. Caching results to make queries faster
- B. Compressing the output to reduce storage
- C. Recording progress and state so the stream can recover after a failure
- D. Scheduling when each micro-batch runs
Show answer
Answer: C
Checkpointing records processing progress and state so a stream can recover and resume without reprocessing or losing data after a failure or restart. It does not cache results for speed, does not compress output, and is not a scheduling mechanism.
How did you do?
Notice how many turn on one idea: does this move data across the cluster? Shuffles explain stage boundaries, join strategy, skew and most performance questions.
If the architecture questions felt least certain, start there — it is 20% of the exam and it makes the other sections comprehensible. See week 1 of the study plan.