spark_applications

Creates, updates, deletes, gets or lists a spark_applications resource.

Overview

Name	spark_applications
Type	Resource
Id	google.dataproc.spark_applications

Fields

The following fields are returned by SELECT queries:

SELECT not supported for this resource, use SHOW METHODS to view available operations for the resource.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
projects_locations_sessions_spark_applications_search_stage_attempt_tasks	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	parent, pageToken, pageSize, taskStatus, sortRuntime, stageId, stageAttemptId	Obtain data corresponding to tasks for a spark stage attempt for a Spark Application.
projects_locations_sessions_spark_applications_access_sql_query	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	parent, planDescription, details, executionId	Obtain data corresponding to a particular SQL Query for a Spark Application.
projects_locations_sessions_spark_applications_access_environment_info	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	parent	Obtain environment details for a Spark Application
projects_locations_sessions_spark_applications_search_executors	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	parent, pageToken, pageSize, executorStatus	Obtain data corresponding to executors for a Spark Application.
projects_locations_sessions_spark_applications_search_jobs	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	jobIds, pageToken, parent, jobStatus, pageSize	Obtain list of spark jobs corresponding to a Spark Application.
projects_locations_sessions_spark_applications_access_sql_plan	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	executionId, parent	Obtain Spark Plan Graph for a Spark Application SQL execution. Limits the number of clusters returned as part of the graph to 10000.
projects_locations_sessions_spark_applications_access_job	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	jobId, parent	Obtain data corresponding to a spark job for a Spark Application.
projects_locations_sessions_spark_applications_access	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	parent	Obtain high level information corresponding to a single Spark Application.
projects_locations_sessions_spark_applications_search	exec	projectsId, locationsId, sessionsId	maxTime, maxEndTime, minTime, pageSize, pageToken, applicationStatus, minEndTime	Obtain high level information and list of Spark Applications corresponding to a batch
projects_locations_sessions_spark_applications_access_stage_rdd_graph	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	stageId, parent	Obtain RDD operation graph for a Spark Application Stage. Limits the number of clusters returned as part of the graph to 10000.
projects_locations_sessions_spark_applications_write	exec	projectsId, locationsId, sessionsId, sparkApplicationsId		Write wrapper objects from dataplane to spanner
projects_locations_sessions_spark_applications_search_executor_stage_summary	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	pageToken, parent, stageId, stageAttemptId, pageSize	Obtain executor summary with respect to a spark stage attempt.
projects_locations_sessions_spark_applications_summarize_stage_attempt_tasks	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	parent, stageId, stageAttemptId	Obtain summary of Tasks for a Spark Application Stage Attempt
projects_locations_sessions_spark_applications_summarize_executors	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	parent	Obtain summary of Executor Summary for a Spark Application
projects_locations_sessions_spark_applications_summarize_stages	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	stageIds, parent	Obtain summary of Stages for a Spark Application
projects_locations_sessions_spark_applications_search_sql_queries	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	details, pageSize, parent, planDescription, pageToken, operationIds	Obtain data corresponding to SQL Queries for a Spark Application.
projects_locations_sessions_spark_applications_summarize_jobs	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	parent, jobIds	Obtain summary of Jobs for a Spark Application
projects_locations_sessions_spark_applications_search_stages	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	pageSize, summaryMetricsMask, stageIds, stageStatus, pageToken, parent	Obtain data corresponding to stages for a Spark Application.
projects_locations_sessions_spark_applications_access_stage_attempt	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	parent, stageAttemptId, stageId, summaryMetricsMask	Obtain data corresponding to a spark stage attempt for a Spark Application.
projects_locations_sessions_spark_applications_search_stage_attempts	exec	projectsId, locationsId, sessionsId, sparkApplicationsId	pageToken, pageSize, stageId, parent, summaryMetricsMask	Obtain data corresponding to a spark stage attempts for a Spark Application.
projects_locations_batches_spark_applications_search_executor_stage_summary	exec	projectsId, locationsId, batchesId, sparkApplicationsId	stageId, pageSize, parent, stageAttemptId, pageToken	Obtain executor summary with respect to a spark stage attempt.
projects_locations_batches_spark_applications_search_stage_attempts	exec	projectsId, locationsId, batchesId, sparkApplicationsId	parent, pageSize, pageToken, stageId, summaryMetricsMask	Obtain data corresponding to a spark stage attempts for a Spark Application.
projects_locations_batches_spark_applications_summarize_stages	exec	projectsId, locationsId, batchesId, sparkApplicationsId	parent	Obtain summary of Stages for a Spark Application
projects_locations_batches_spark_applications_search_sql_queries	exec	projectsId, locationsId, batchesId, sparkApplicationsId	details, parent, pageToken, planDescription, pageSize	Obtain data corresponding to SQL Queries for a Spark Application.
projects_locations_batches_spark_applications_access_stage_attempt	exec	projectsId, locationsId, batchesId, sparkApplicationsId	summaryMetricsMask, stageId, stageAttemptId, parent	Obtain data corresponding to a spark stage attempt for a Spark Application.
projects_locations_batches_spark_applications_search_stage_attempt_tasks	exec	projectsId, locationsId, batchesId, sparkApplicationsId	pageSize, sortRuntime, stageId, pageToken, parent, stageAttemptId, taskStatus	Obtain data corresponding to tasks for a spark stage attempt for a Spark Application.
projects_locations_batches_spark_applications_access_stage_rdd_graph	exec	projectsId, locationsId, batchesId, sparkApplicationsId	parent, stageId	Obtain RDD operation graph for a Spark Application Stage. Limits the number of clusters returned as part of the graph to 10000.
projects_locations_batches_spark_applications_access	exec	projectsId, locationsId, batchesId, sparkApplicationsId	parent	Obtain high level information corresponding to a single Spark Application.
projects_locations_batches_spark_applications_write	exec	projectsId, locationsId, batchesId, sparkApplicationsId		Write wrapper objects from dataplane to spanner
projects_locations_batches_spark_applications_search_jobs	exec	projectsId, locationsId, batchesId, sparkApplicationsId	jobStatus, pageToken, pageSize, parent	Obtain list of spark jobs corresponding to a Spark Application.
projects_locations_batches_spark_applications_search	exec	projectsId, locationsId, batchesId	minTime, applicationStatus, maxEndTime, minEndTime, pageToken, pageSize, maxTime	Obtain high level information and list of Spark Applications corresponding to a batch
projects_locations_batches_spark_applications_access_environment_info	exec	projectsId, locationsId, batchesId, sparkApplicationsId	parent	Obtain environment details for a Spark Application
projects_locations_batches_spark_applications_search_executors	exec	projectsId, locationsId, batchesId, sparkApplicationsId	executorStatus, parent, pageToken, pageSize	Obtain data corresponding to executors for a Spark Application.
projects_locations_batches_spark_applications_summarize_executors	exec	projectsId, locationsId, batchesId, sparkApplicationsId	parent	Obtain summary of Executor Summary for a Spark Application
projects_locations_batches_spark_applications_access_sql_query	exec	projectsId, locationsId, batchesId, sparkApplicationsId	parent, planDescription, details, executionId	Obtain data corresponding to a particular SQL Query for a Spark Application.
projects_locations_batches_spark_applications_access_job	exec	projectsId, locationsId, batchesId, sparkApplicationsId	jobId, parent	Obtain data corresponding to a spark job for a Spark Application.
projects_locations_batches_spark_applications_access_sql_plan	exec	projectsId, locationsId, batchesId, sparkApplicationsId	executionId, parent	Obtain Spark Plan Graph for a Spark Application SQL execution. Limits the number of clusters returned as part of the graph to 10000.
projects_locations_batches_spark_applications_summarize_stage_attempt_tasks	exec	projectsId, locationsId, batchesId, sparkApplicationsId	parent, stageAttemptId, stageId	Obtain summary of Tasks for a Spark Application Stage Attempt
projects_locations_batches_spark_applications_summarize_jobs	exec	projectsId, locationsId, batchesId, sparkApplicationsId	parent	Obtain summary of Jobs for a Spark Application
projects_locations_batches_spark_applications_search_stages	exec	projectsId, locationsId, batchesId, sparkApplicationsId	summaryMetricsMask, pageToken, parent, pageSize, stageStatus	Obtain data corresponding to stages for a Spark Application.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
batchesId	string
locationsId	string
projectsId	string
sessionsId	string
sparkApplicationsId	string
applicationStatus	string
details	boolean
executionId	string (int64)
executorStatus	string
jobId	string (int64)
jobIds	string (int64)
jobStatus	string
maxEndTime	string (google-datetime)
maxTime	string (google-datetime)
minEndTime	string (google-datetime)
minTime	string (google-datetime)
operationIds	string
pageSize	integer (int32)
pageToken	string
parent	string
planDescription	boolean
sortRuntime	boolean
stageAttemptId	integer (int32)
stageId	string (int64)
stageIds	string (int64)
stageStatus	string
summaryMetricsMask	string (google-fieldmask)
taskStatus	string

Lifecycle Methods

projects_locations_sessions_spark_applications_search_stage_attempt_tasks

Obtain data corresponding to tasks for a spark stage attempt for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_search_stage_attempt_tasks 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}', 
@pageToken='{{ pageToken }}', 
@pageSize='{{ pageSize }}', 
@taskStatus='{{ taskStatus }}', 
@sortRuntime={{ sortRuntime }}, 
@stageId='{{ stageId }}', 
@stageAttemptId='{{ stageAttemptId }}'
;

projects_locations_sessions_spark_applications_access_sql_query

Obtain data corresponding to a particular SQL Query for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_access_sql_query 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}', 
@planDescription={{ planDescription }}, 
@details={{ details }}, 
@executionId='{{ executionId }}'
;

projects_locations_sessions_spark_applications_access_environment_info

Obtain environment details for a Spark Application

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_access_environment_info 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}'
;

projects_locations_sessions_spark_applications_search_executors

Obtain data corresponding to executors for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_search_executors 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}', 
@pageToken='{{ pageToken }}', 
@pageSize='{{ pageSize }}', 
@executorStatus='{{ executorStatus }}'
;

projects_locations_sessions_spark_applications_search_jobs

Obtain list of spark jobs corresponding to a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_search_jobs 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@jobIds='{{ jobIds }}', 
@pageToken='{{ pageToken }}', 
@parent='{{ parent }}', 
@jobStatus='{{ jobStatus }}', 
@pageSize='{{ pageSize }}'
;

projects_locations_sessions_spark_applications_access_sql_plan

Obtain Spark Plan Graph for a Spark Application SQL execution. Limits the number of clusters returned as part of the graph to 10000.

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_access_sql_plan 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@executionId='{{ executionId }}', 
@parent='{{ parent }}'
;

projects_locations_sessions_spark_applications_access_job

Obtain data corresponding to a spark job for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_access_job 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@jobId='{{ jobId }}', 
@parent='{{ parent }}'
;

projects_locations_sessions_spark_applications_access

Obtain high level information corresponding to a single Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_access 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}'
;

projects_locations_sessions_spark_applications_search

Obtain high level information and list of Spark Applications corresponding to a batch

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_search 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@maxTime='{{ maxTime }}', 
@maxEndTime='{{ maxEndTime }}', 
@minTime='{{ minTime }}', 
@pageSize='{{ pageSize }}', 
@pageToken='{{ pageToken }}', 
@applicationStatus='{{ applicationStatus }}', 
@minEndTime='{{ minEndTime }}'
;

projects_locations_sessions_spark_applications_access_stage_rdd_graph

Obtain RDD operation graph for a Spark Application Stage. Limits the number of clusters returned as part of the graph to 10000.

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_access_stage_rdd_graph 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@stageId='{{ stageId }}', 
@parent='{{ parent }}'
;

projects_locations_sessions_spark_applications_write

Write wrapper objects from dataplane to spanner

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_write 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required 
@@json=
'{
"sparkWrapperObjects": "{{ sparkWrapperObjects }}", 
"parent": "{{ parent }}"
}'
;

projects_locations_sessions_spark_applications_search_executor_stage_summary

Obtain executor summary with respect to a spark stage attempt.

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_search_executor_stage_summary 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@pageToken='{{ pageToken }}', 
@parent='{{ parent }}', 
@stageId='{{ stageId }}', 
@stageAttemptId='{{ stageAttemptId }}', 
@pageSize='{{ pageSize }}'
;

projects_locations_sessions_spark_applications_summarize_stage_attempt_tasks

Obtain summary of Tasks for a Spark Application Stage Attempt

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_summarize_stage_attempt_tasks 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}', 
@stageId='{{ stageId }}', 
@stageAttemptId='{{ stageAttemptId }}'
;

projects_locations_sessions_spark_applications_summarize_executors

Obtain summary of Executor Summary for a Spark Application

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_summarize_executors 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}'
;

projects_locations_sessions_spark_applications_summarize_stages

Obtain summary of Stages for a Spark Application

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_summarize_stages 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@stageIds='{{ stageIds }}', 
@parent='{{ parent }}'
;

projects_locations_sessions_spark_applications_search_sql_queries

Obtain data corresponding to SQL Queries for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_search_sql_queries 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@details={{ details }}, 
@pageSize='{{ pageSize }}', 
@parent='{{ parent }}', 
@planDescription={{ planDescription }}, 
@pageToken='{{ pageToken }}', 
@operationIds='{{ operationIds }}'
;

projects_locations_sessions_spark_applications_summarize_jobs

Obtain summary of Jobs for a Spark Application

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_summarize_jobs 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}', 
@jobIds='{{ jobIds }}'
;

projects_locations_sessions_spark_applications_search_stages

Obtain data corresponding to stages for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_search_stages 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@pageSize='{{ pageSize }}', 
@summaryMetricsMask='{{ summaryMetricsMask }}', 
@stageIds='{{ stageIds }}', 
@stageStatus='{{ stageStatus }}', 
@pageToken='{{ pageToken }}', 
@parent='{{ parent }}'
;

projects_locations_sessions_spark_applications_access_stage_attempt

Obtain data corresponding to a spark stage attempt for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_access_stage_attempt 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}', 
@stageAttemptId='{{ stageAttemptId }}', 
@stageId='{{ stageId }}', 
@summaryMetricsMask='{{ summaryMetricsMask }}'
;

projects_locations_sessions_spark_applications_search_stage_attempts

Obtain data corresponding to a spark stage attempts for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_sessions_spark_applications_search_stage_attempts 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@sessionsId='{{ sessionsId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@pageToken='{{ pageToken }}', 
@pageSize='{{ pageSize }}', 
@stageId='{{ stageId }}', 
@parent='{{ parent }}', 
@summaryMetricsMask='{{ summaryMetricsMask }}'
;

projects_locations_batches_spark_applications_search_executor_stage_summary

Obtain executor summary with respect to a spark stage attempt.

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_search_executor_stage_summary 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@stageId='{{ stageId }}', 
@pageSize='{{ pageSize }}', 
@parent='{{ parent }}', 
@stageAttemptId='{{ stageAttemptId }}', 
@pageToken='{{ pageToken }}'
;

projects_locations_batches_spark_applications_search_stage_attempts

Obtain data corresponding to a spark stage attempts for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_search_stage_attempts 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}', 
@pageSize='{{ pageSize }}', 
@pageToken='{{ pageToken }}', 
@stageId='{{ stageId }}', 
@summaryMetricsMask='{{ summaryMetricsMask }}'
;

projects_locations_batches_spark_applications_summarize_stages

Obtain summary of Stages for a Spark Application

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_summarize_stages 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}'
;

projects_locations_batches_spark_applications_search_sql_queries

Obtain data corresponding to SQL Queries for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_search_sql_queries 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@details={{ details }}, 
@parent='{{ parent }}', 
@pageToken='{{ pageToken }}', 
@planDescription={{ planDescription }}, 
@pageSize='{{ pageSize }}'
;

projects_locations_batches_spark_applications_access_stage_attempt

Obtain data corresponding to a spark stage attempt for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_access_stage_attempt 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@summaryMetricsMask='{{ summaryMetricsMask }}', 
@stageId='{{ stageId }}', 
@stageAttemptId='{{ stageAttemptId }}', 
@parent='{{ parent }}'
;

projects_locations_batches_spark_applications_search_stage_attempt_tasks

Obtain data corresponding to tasks for a spark stage attempt for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_search_stage_attempt_tasks 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@pageSize='{{ pageSize }}', 
@sortRuntime={{ sortRuntime }}, 
@stageId='{{ stageId }}', 
@pageToken='{{ pageToken }}', 
@parent='{{ parent }}', 
@stageAttemptId='{{ stageAttemptId }}', 
@taskStatus='{{ taskStatus }}'
;

projects_locations_batches_spark_applications_access_stage_rdd_graph

Obtain RDD operation graph for a Spark Application Stage. Limits the number of clusters returned as part of the graph to 10000.

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_access_stage_rdd_graph 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}', 
@stageId='{{ stageId }}'
;

projects_locations_batches_spark_applications_access

Obtain high level information corresponding to a single Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_access 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}'
;

projects_locations_batches_spark_applications_write

Write wrapper objects from dataplane to spanner

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_write 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required 
@@json=
'{
"parent": "{{ parent }}", 
"sparkWrapperObjects": "{{ sparkWrapperObjects }}"
}'
;

projects_locations_batches_spark_applications_search_jobs

Obtain list of spark jobs corresponding to a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_search_jobs 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@jobStatus='{{ jobStatus }}', 
@pageToken='{{ pageToken }}', 
@pageSize='{{ pageSize }}', 
@parent='{{ parent }}'
;

projects_locations_batches_spark_applications_search

Obtain high level information and list of Spark Applications corresponding to a batch

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_search 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@minTime='{{ minTime }}', 
@applicationStatus='{{ applicationStatus }}', 
@maxEndTime='{{ maxEndTime }}', 
@minEndTime='{{ minEndTime }}', 
@pageToken='{{ pageToken }}', 
@pageSize='{{ pageSize }}', 
@maxTime='{{ maxTime }}'
;

projects_locations_batches_spark_applications_access_environment_info

Obtain environment details for a Spark Application

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_access_environment_info 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}'
;

projects_locations_batches_spark_applications_search_executors

Obtain data corresponding to executors for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_search_executors 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@executorStatus='{{ executorStatus }}', 
@parent='{{ parent }}', 
@pageToken='{{ pageToken }}', 
@pageSize='{{ pageSize }}'
;

projects_locations_batches_spark_applications_summarize_executors

Obtain summary of Executor Summary for a Spark Application

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_summarize_executors 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}'
;

projects_locations_batches_spark_applications_access_sql_query

Obtain data corresponding to a particular SQL Query for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_access_sql_query 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}', 
@planDescription={{ planDescription }}, 
@details={{ details }}, 
@executionId='{{ executionId }}'
;

projects_locations_batches_spark_applications_access_job

Obtain data corresponding to a spark job for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_access_job 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@jobId='{{ jobId }}', 
@parent='{{ parent }}'
;

projects_locations_batches_spark_applications_access_sql_plan

Obtain Spark Plan Graph for a Spark Application SQL execution. Limits the number of clusters returned as part of the graph to 10000.

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_access_sql_plan 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@executionId='{{ executionId }}', 
@parent='{{ parent }}'
;

projects_locations_batches_spark_applications_summarize_stage_attempt_tasks

Obtain summary of Tasks for a Spark Application Stage Attempt

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_summarize_stage_attempt_tasks 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}', 
@stageAttemptId='{{ stageAttemptId }}', 
@stageId='{{ stageId }}'
;

projects_locations_batches_spark_applications_summarize_jobs

Obtain summary of Jobs for a Spark Application

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_summarize_jobs 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@parent='{{ parent }}'
;

projects_locations_batches_spark_applications_search_stages

Obtain data corresponding to stages for a Spark Application.

EXEC google.dataproc.spark_applications.projects_locations_batches_spark_applications_search_stages 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchesId='{{ batchesId }}' --required, 
@sparkApplicationsId='{{ sparkApplicationsId }}' --required, 
@summaryMetricsMask='{{ summaryMetricsMask }}', 
@pageToken='{{ pageToken }}', 
@parent='{{ parent }}', 
@pageSize='{{ pageSize }}', 
@stageStatus='{{ stageStatus }}'
;