Associate-Developer-Apache-Spark 試験問題を無料オンラインアクセス

Which of the following code blocks reads in the parquet file stored at location filePath, given that all columns in the parquet file contain only whole numbers and are stored in the most appropriate format for this kind of data?

• A.1.spark.read.schema(
  2. StructType([
  3. StructField("transactionId", IntegerType(), True),
  4. StructField("predError", IntegerType(), True)]
  5. )).format("parquet").load(filePath)
• B.1.spark.read.schema(
  2. StructType(
  3. StructField("transactionId", IntegerType(), True),
  4. StructField("predError", IntegerType(), True)
  5. )).load(filePath)
• C.1.spark.read.schema(
  2. StructType([
  3. StructField("transactionId", StringType(), True),
  4. StructField("predError", IntegerType(), True)]
  5. )).parquet(filePath)
• D.1.spark.read.schema([
  2. StructField("transactionId", NumberType(), True),
  3. StructField("predError", IntegerType(), True)
  4. ]).load(filePath)
• E.1.spark.read.schema([
  2. StructField("transactionId", IntegerType(), True),
  3. StructField("predError", IntegerType(), True)
  4. ]).load(filePath, format="parquet")

Which of the following describes a valid concern about partitioning?

• A.Short partition processing times are indicative of low skew.
• B.A shuffle operation returns 200 partitions if not explicitly set.
• C.No data is exchanged between executors when coalesce() is run.
• D.The coalesce() method should be used to increase the number of partitions.
• E.Decreasing the number of partitions reduces the overall runtime of narrow transformations if there are more executors available than partitions.

Which of the following code blocks performs a join in which the small DataFrame transactionsDf is sent to all executors where it is joined with DataFrame itemsDf on columns storeId and itemId, respectively?

• A.itemsDf.join(transactionsDf, broadcast(itemsDf.itemId == transactionsDf.storeId))
• B.itemsDf.join(broadcast(transactionsDf), itemsDf.itemId == transactionsDf.storeId)
• C.itemsDf.join(transactionsDf, itemsDf.itemId == transactionsDf.storeId, "right_outer")
• D.itemsDf.merge(transactionsDf, "itemsDf.itemId == transactionsDf.storeId", "broadcast")
• E.itemsDf.join(transactionsDf, itemsDf.itemId == transactionsDf.storeId, "broadcast")

Which of the following describes a difference between Spark's cluster and client execution modes?

• A.In cluster mode, executor processes run on worker nodes, while they run on gateway nodes in client mode.
• B.In cluster mode, a gateway machine hosts the driver, while it is co-located with the executor in client mode.
• C.In cluster mode, the Spark driver is not co-located with the cluster manager, while it is co-located in client mode.
• D.In cluster mode, the driver resides on a worker node, while it resides on an edge node in client mode.
• E.In cluster mode, the cluster manager resides on a worker node, while it resides on an edge node in client mode.

Which of the following code blocks generally causes a great amount of network traffic?

• A.DataFrame.select()
• B.DataFrame.count()
• C.DataFrame.coalesce()
• D.DataFrame.rdd.map()
• E.DataFrame.collect()