Delta Lake and Spark practice questions
Fifteen free Delta Lake and Spark practice questions with explained answers, written for anyone preparing for a Databricks certification. Delta Lake is the open table format that adds a transaction log, ACID transactions and time travel on top of Parquet files in cloud storage, and Spark is the engine that reads and writes it. Both are tested in the Databricks Data Engineer Associate and the Spark Developer Associate exams. All questions are original, use Databricks SQL and PySpark, and assume Unity Catalog where tables are mentioned.
How to use these questions
Answer each question before opening the explanation, and commit to one letter. Underline the constraint word in the stem, such as “without copying data”, “without a full shuffle” or “default”, because it usually separates two similar commands. Read the explanation even when the answer was right, since each one also says why the other three options fail. The background is in the Delta Lake explained topic guide.
Warm-up: tables and Spark basics
Question 1. An engineer lists the storage folder of a Delta table and finds Parquet data files plus a folder named _delta_log. What does _delta_log hold?
- A. A cache of recent query results
- B. A backup copy of every Parquet file
- C. The ordered record of commits that defines each table version
- D. Error logs from failed jobs
Show answer
Answer: C
The _delta_log folder holds the ordered transaction log: JSON commit files (with periodic Parquet checkpoints) that record which data files were added or removed in each version, which is what gives Delta its ACID guarantees and time travel. It is not a cache of query results, not a copy of the data, and not the place where failed jobs write their error logs.
Question 2. In Unity Catalog, a team runs DROP TABLE on an external Delta table. What happens?
- A. The table registration is removed and the data files remain in storage
- B. The table registration and all data files are deleted at once
- C. The data files are moved to a trash schema in the catalog
- D. The command fails until the files are deleted manually
Show answer
Answer: A
For an external table, Unity Catalog removes only the table metadata; the data files stay in the external storage location. Dropping a managed table is what removes the underlying data. The files are not deleted immediately, they are not moved to a recycle schema, and the command does not fail simply because the table is external.
Question 3. Which line of PySpark makes Spark actually execute the job, given that df is a DataFrame read from a Delta table?
- A. df.filter(df.amount > 100)
- B. df.withColumn(’tax’, df.amount * 0.2)
- C. df.select(‘id’, ‘amount’)
- D. df.count()
Show answer
Answer: D
count() is an action: it needs a result, so Spark builds the plan and runs it. filter, withColumn and select are lazy transformations that only extend the logical plan. Reading the table with spark.read is also lazy beyond metadata and schema inference. Spark runs nothing for those lines until an action is called.
Question 4. A streaming job appends small batches every minute to a Delta table, and the table now has tens of thousands of tiny files that slow queries. Which command addresses this?
- A. VACUUM sales
- B. OPTIMIZE sales
- C. ANALYZE TABLE sales COMPUTE STATISTICS
- D. DESCRIBE HISTORY sales
Show answer
Answer: B
OPTIMIZE compacts small files into larger ones, which reduces file listing and open overhead. VACUUM deletes files that are no longer referenced by the table and does not merge live files. ANALYZE TABLE computes statistics and does not rewrite data. DESCRIBE HISTORY only reports the table’s operations.
Question 5. Each night a table of customers must be updated from a staging table: existing customers get new values and unseen customers are added. Which statement does this in one atomic operation?
- A. INSERT INTO customers SELECT * FROM staging
- B. INSERT OVERWRITE customers SELECT * FROM staging
- C. MERGE INTO customers USING staging ON the key, with matched update and not matched insert
- D. COPY INTO customers FROM staging
Show answer
Answer: C
MERGE INTO with WHEN MATCHED THEN UPDATE and WHEN NOT MATCHED THEN INSERT performs an upsert atomically. INSERT INTO would duplicate existing customers. INSERT OVERWRITE replaces the entire table with the staging rows, dropping customers not in staging. COPY INTO loads files from a storage path, not rows from another table.
Scenarios: ingestion, recovery and cloning
Question 6. A bad UPDATE ran at version 14 of a Delta table, and the table must go back to its state at version 13 in place. History is within the retention period. Which command does this?
- A. RESTORE TABLE orders TO VERSION AS OF 13
- B. SELECT * FROM orders VERSION AS OF 13
- C. ROLLBACK TABLE orders TO 13
- D. VACUUM orders RETAIN 13 HOURS
Show answer
Answer: A
RESTORE TABLE … TO VERSION AS OF 13 rolls the table back to that version by writing a new commit, as long as the needed data files still exist. SELECT with VERSION AS OF only reads the old version and changes nothing. There is no ROLLBACK TABLE command for Delta. VACUUM … RETAIN deletes old files and would make recovery harder, not do it.
Question 7. A folder in cloud storage receives a few hundred new CSV files a day. A job runs the same COPY INTO statement into a Delta table each morning. What does the second run load by default?
- A. All files in the folder again, creating duplicates
- B. Only the files not loaded by earlier runs
- C. Nothing, because the folder was already read once
- D. All files, after truncating the target table
Show answer
Answer: B
COPY INTO is idempotent: it tracks the files it already loaded and skips them, so only the new files are ingested. It does not reload everything by default, that needs the FORCE option. It does not truncate the table first, and it does not fail merely because the target table already has data.
Question 8. Files arrive continuously in cloud storage in the millions, and the schema gains new columns from time to time. The pipeline must ingest incrementally and handle schema evolution. Which ingestion method fits best?
- A. COPY INTO with FORCE set to true
- B. spark.read of the whole folder on each run
- C. A notebook that lists files with dbutils and loops over them
- D. Auto Loader with a checkpoint and schema evolution
Show answer
Answer: D
Auto Loader (the cloudFiles source with a checkpoint) discovers new files incrementally at large scale and supports schema inference and evolution. COPY INTO is designed for smaller file counts, in the thousands. A full spark.read of the folder rereads everything every run. Manually listing files with dbutils does not scale or track state.
Question 9. A DataFrame with an extra column, coupon, is appended to a Delta table that has no such column. No schema options are set. What happens, and how can the new column be accepted on purpose?
- A. The column is silently dropped; set mergeSchema to false to keep it
- B. The column is added automatically; no setting is needed
- C. The write fails; add the column with ALTER TABLE or write with mergeSchema set to true
- D. The write fails; the only way is to drop and recreate the table
Show answer
Answer: C
Delta enforces the table schema on write, so the append fails with a schema mismatch error. To evolve the schema deliberately, add the column with ALTER TABLE or write with the mergeSchema option set to true. The extra column is not dropped silently and not added silently, and overwriteSchema is meant for replacing the schema on an overwrite, not for a plain append.
Question 10. A team needs a dev copy of a 5 TB production Delta table for testing within seconds, without copying the data files, and changes to the copy must not affect production. What should they use?
- A. CREATE TABLE dev_orders SHALLOW CLONE prod_orders
- B. CREATE TABLE dev_orders DEEP CLONE prod_orders
- C. CREATE VIEW dev_orders AS SELECT * FROM prod_orders
- D. CREATE TABLE dev_orders AS SELECT * FROM prod_orders
Show answer
Answer: A
A SHALLOW CLONE copies only the table metadata and references the source data files, so it is fast and cheap; writes to the clone go to its own new files and leave production untouched. DEEP CLONE copies all data, which is slow and costs storage. A view is not a table and cannot be changed independently. CREATE TABLE AS SELECT also copies all 5 TB.
Edge cases and traps
Question 11. VACUUM was run on a Delta table with default settings. A week and a half later, someone queries the table with VERSION AS OF a version written 10 days ago and the query fails with a missing files error. Why?
- A. Time travel is disabled in Unity Catalog
- B. VACUUM deletes the transaction log after 7 days
- C. VACUUM compacted the old version into the new one
- D. VACUUM removed unreferenced files older than the 7-day retention, so old versions can no longer be read
Show answer
Answer: D
By default VACUUM removes data files that are no longer referenced by the current table version and are older than the 7-day retention threshold, so the files needed by a 10-day-old version are gone, even though the log entries still exist. The log is not deleted at 7 days (log retention is 30 days by default), time travel is not disabled by Unity Catalog, and VACUUM does not compact files.
Question 12. A downstream team needs row-level inserts, updates and deletes from a Delta table since version 20, including the previous value of changed rows. What is needed?
- A. Run DESCRIBE HISTORY and parse the operation metrics
- B. Enable change data feed and read it with table_changes
- C. Query the table twice with VERSION AS OF and compare
- D. Run VACUUM with RETAIN 0 HOURS
Show answer
Answer: B
Change data feed must be enabled on the table (delta.enableChangeDataFeed = true) and the changes are then read with table_changes or the readChangeFeed option; it records changes made after it was enabled, including pre- and post-images of updates. DESCRIBE HISTORY lists operations, not row-level changes. A VERSION AS OF query returns a full snapshot, so the consumer would have to diff snapshots. VACUUM removes files and records nothing.
Question 13. Which PySpark operation forces Spark to shuffle data across the cluster?
- A. df.filter(df.country == ‘NL’)
- B. df.select(‘country’, ‘amount’)
- C. df.groupBy(‘country’).count()
- D. df.withColumn(‘flag’, df.amount > 0)
Show answer
Answer: C
groupBy(…).count() is a wide transformation: rows with the same key must be brought to the same partition, which requires a shuffle. filter, select and withColumn are narrow transformations that work on each partition independently and need no data exchange.
Question 14. A 500 GB fact DataFrame is joined to a 40 MB dimension DataFrame on a key. With default settings Spark uses a sort-merge join and shuffles both sides. What is the most direct way to avoid shuffling the large side?
- A. Wrap the dimension DataFrame in broadcast() in the join
- B. Repartition the fact DataFrame by the join key first
- C. Cache the fact DataFrame
- D. Call coalesce(1) on the fact DataFrame
Show answer
Answer: A
Marking the small side with the broadcast() hint sends a copy to every executor so the large side is joined in place, with no shuffle of the 500 GB. The default broadcast threshold is 10 MB, which is why the 40 MB table was not broadcast automatically. Repartitioning the large side is itself a shuffle. Caching does not change the join strategy, and coalesce(1) on the fact table would funnel it all to one task.
Question 15. After a selective filter, a DataFrame still has 2,000 partitions but only a few thousand rows, and writing it creates 2,000 tiny files. The partition count must be reduced without a full shuffle. What should be used?
- A. df.repartition(8)
- B. df.coalesce(8)
- C. df.cache()
- D. Raise spark.sql.shuffle.partitions to 4000
Show answer
Answer: B
coalesce(n) reduces the number of partitions by merging existing ones, avoiding a full shuffle. repartition(n) can also lower the count, but it always performs a full shuffle. cache() keeps data in memory and leaves the partition count unchanged. Raising spark.sql.shuffle.partitions affects future shuffles and would increase, not reduce, the number of output tasks.
Where to go next
Read the Delta Lake explained topic guide for the transaction log, maintenance commands and ingestion patterns. Related practice sets are CI/CD and Git practice questions and Azure storage practice questions. For timed mixed practice, take the free 20-question Data Engineer Associate practice test or the Spark Developer Associate practice test, and read the multiple-choice exam tips.