Associate-Developer-Apache-Spark-3.5 試験問題を無料オンラインアクセス

7 of 55.
A developer has been asked to debug an issue with a Spark application. The developer identified that the data being loaded from a CSV file is being read incorrectly into a DataFrame.
The CSV file has been read using the following Spark SQL statement:
CREATE TABLE locations
USING csv
OPTIONS (path '/data/locations.csv')
The first lines of the command SELECT * FROM locations look like this:
| city | lat | long |
| ALTI Sydney | -33... | ... |
Which parameter can the developer add to the OPTIONS clause in the CREATE TABLE statement to read the CSV data correctly again?

• A.'sep' ','
• B.'sep' '|'
• C.'header' 'false'
• D.'header' 'true'

49 of 55.
In the code block below, aggDF contains aggregations on a streaming DataFrame:
aggDF.writeStream \
.format("console") \
.outputMode("???") \
.start()
Which output mode at line 3 ensures that the entire result table is written to the console during each trigger execution?

• A.AGGREGATE
• B.COMPLETE
• C.REPLACE
• D.APPEND

A developer is trying to join two tables, sales.purchases_fct and sales.customer_dim, using the following code:

fact_df = purch_df.join(cust_df, F.col('customer_id') == F.col('custid')) The developer has discovered that customers in the purchases_fct table that do not exist in the customer_dim table are being dropped from the joined table.
Which change should be made to the code to stop these customer records from being dropped?

• A.fact_df = purch_df.join(cust_df, F.col('cust_id') == F.col('customer_id'))
• B.fact_df = purch_df.join(cust_df, F.col('customer_id') == F.col('custid'), 'right_outer')
• C.fact_df = purch_df.join(cust_df, F.col('customer_id') == F.col('custid'), 'left')
• D.fact_df = cust_df.join(purch_df, F.col('customer_id') == F.col('custid'))

39 of 55.
A Spark developer is developing a Spark application to monitor task performance across a cluster.
One requirement is to track the maximum processing time for tasks on each worker node and consolidate this information on the driver for further analysis.
Which technique should the developer use?

• A.Broadcast a variable to share the maximum time among workers.
• B.Use an accumulator to record the maximum time on the driver.
• C.Configure the Spark UI to automatically collect maximum times.
• D.Use an RDD action like reduce() to compute the maximum time.

23 of 55.
A data scientist is working with a massive dataset that exceeds the memory capacity of a single machine. The data scientist is considering using Apache Spark™ instead of traditional single-machine languages like standard Python scripts.
Which two advantages does Apache Spark™ offer over a normal single-machine language in this scenario? (Choose 2 answers)

• A.It processes data solely on disk storage, reducing the need for memory resources.
• B.It requires specialized hardware to run, making it unsuitable for commodity hardware clusters.
• C.It has built-in fault tolerance, allowing it to recover seamlessly from node failures during computation.
• D.It can distribute data processing tasks across a cluster of machines, enabling horizontal scalability.
• E.It eliminates the need to write any code, automatically handling all data processing.