Home / Data Engineer Associate / Objective 3.4

Domain 3: Data Transformation and Modeling · 22% of the exam · Full exam outline

3.4 Perform data deduplication operations and aggregate operations on DataFrames, such as count, approximate count distinct, and mean, summary.

Practice questions

Question 1easy

While profiling a new silver table, an engineer wants a single DataFrame showing count, mean, standard deviation, min, max and the 25th and 75th percentiles of its numeric columns. Which method provides this?

Included in Master: master every domain.See plans
Question 2medium

A Structured Streaming job deduplicates events with events.dropDuplicates(["event_id"]) before writing to a silver table. Memory use climbs steadily over days. Duplicates only ever arrive within two hours of the original event. Which change addresses the growth?

Included in Master: master every domain.See plans