batch_prediction_jobs

Creates, updates, deletes, gets or lists a batch_prediction_jobs resource.

Overview

Name	batch_prediction_jobs
Type	Resource
Id	google.aiplatform.batch_prediction_jobs

Fields

The following fields are returned by SELECT queries:

get

Successful response

Name	Datatype	Description
name	string	Output only. Resource name of the BatchPredictionJob.
completionStats	object	Output only. Statistics on completed and failed prediction instances. (id: GoogleCloudAiplatformV1CompletionStats)
createTime	string (google-datetime)	Output only. Time when the BatchPredictionJob was created.
dedicatedResources	object	The config of resources used by the Model during the batch prediction. If the Model supports DEDICATED_RESOURCES this config may be provided (and the job will use these resources), if the Model doesn't support AUTOMATIC_RESOURCES, this config must be provided. (id: GoogleCloudAiplatformV1BatchDedicatedResources)
disableContainerLogging	boolean	For custom-trained Models and AutoML Tabular Models, the container of the DeployedModel instances will send stderr and stdout streams to Cloud Logging by default. Please note that the logs incur cost, which are subject to Cloud Logging pricing. User can disable container logging by setting this flag to true.
displayName	string	Required. The user-defined name of this BatchPredictionJob.
encryptionSpec	object	Customer-managed encryption key options for a BatchPredictionJob. If this is set, then all resources created by the BatchPredictionJob will be encrypted with the provided encryption key. (id: GoogleCloudAiplatformV1EncryptionSpec)
endTime	string (google-datetime)	Output only. Time when the BatchPredictionJob entered any of the following states: JOB_STATE_SUCCEEDED, JOB_STATE_FAILED, JOB_STATE_CANCELLED.
error	object	Output only. Only populated when the job's state is JOB_STATE_FAILED or JOB_STATE_CANCELLED. (id: GoogleRpcStatus)
explanationSpec	object	Explanation configuration for this BatchPredictionJob. Can be specified only if generate_explanation is set to true. This value overrides the value of Model.explanation_spec. All fields of explanation_spec are optional in the request. If a field of the explanation_spec object is not populated, the corresponding field of the Model.explanation_spec object is inherited. (id: GoogleCloudAiplatformV1ExplanationSpec)
generateExplanation	boolean	Generate explanation with the batch prediction results. When set to true, the batch prediction output changes based on the predictions_format field of the BatchPredictionJob.output_config object: * bigquery: output includes a column named explanation. The value is a struct that conforms to the Explanation object. * jsonl: The JSON objects on each line include an additional entry keyed explanation. The value of the entry is a JSON object that conforms to the Explanation object. * csv: Generating explanations for CSV format is not supported. If this field is set to true, either the Model.explanation_spec or explanation_spec must be populated.
inputConfig	object	Required. Input configuration of the instances on which predictions are performed. The schema of any single instance may be specified via the Model's PredictSchemata's instance_schema_uri. (id: GoogleCloudAiplatformV1BatchPredictionJobInputConfig)
instanceConfig	object	Configuration for how to convert batch prediction input instances to the prediction instances that are sent to the Model. (id: GoogleCloudAiplatformV1BatchPredictionJobInstanceConfig)
labels	object	The labels with user-defined metadata to organize BatchPredictionJobs. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.
manualBatchTuningParameters	object	Immutable. Parameters configuring the batch behavior. Currently only applicable when dedicated_resources are used (in other cases Vertex AI does the tuning itself). (id: GoogleCloudAiplatformV1ManualBatchTuningParameters)
model	string	The name of the Model resource that produces the predictions via this job, must share the same ancestor Location. Starting this job has no impact on any existing deployments of the Model and their resources. Exactly one of model and unmanaged_container_model must be set. The model resource name may contain version id or version alias to specify the version. Example: projects/{project}/locations/{location}/models/{model}@2 or projects/{project}/locations/{location}/models/{model}@golden if no version is specified, the default version will be deployed. The model resource could also be a publisher model. Example: publishers/{publisher}/models/{model} or projects/{project}/locations/{location}/publishers/{publisher}/models/{model}
modelParameters	any	The parameters that govern the predictions. The schema of the parameters may be specified via the Model's PredictSchemata's parameters_schema_uri.
modelVersionId	string	Output only. The version ID of the Model that produces the predictions via this job.
outputConfig	object	Required. The Configuration specifying where output predictions should be written. The schema of any single prediction may be specified as a concatenation of Model's PredictSchemata's instance_schema_uri and prediction_schema_uri. (id: GoogleCloudAiplatformV1BatchPredictionJobOutputConfig)
outputInfo	object	Output only. Information further describing the output of this job. (id: GoogleCloudAiplatformV1BatchPredictionJobOutputInfo)
partialFailures	array	Output only. Partial failures encountered. For example, single files that can't be read. This field never exceeds 20 entries. Status details fields contain standard Google Cloud error details.
resourcesConsumed	object	Output only. Information about resources that had been consumed by this job. Provided in real time at best effort basis, as well as a final value once the job completes. Note: This field currently may be not populated for batch predictions that use AutoML Models. (id: GoogleCloudAiplatformV1ResourcesConsumed)
satisfiesPzi	boolean	Output only. Reserved for future use.
satisfiesPzs	boolean	Output only. Reserved for future use.
serviceAccount	string	The service account that the DeployedModel's container runs as. If not specified, a system generated one will be used, which has minimal permissions and the custom container, if used, may not have enough permission to access other Google Cloud resources. Users deploying the Model must have the iam.serviceAccounts.actAs permission on this service account.
startTime	string (google-datetime)	Output only. Time when the BatchPredictionJob for the first time entered the JOB_STATE_RUNNING state.
state	string	Output only. The detailed state of the job.
unmanagedContainerModel	object	Contains model information necessary to perform batch prediction without requiring uploading to model registry. Exactly one of model and unmanaged_container_model must be set. (id: GoogleCloudAiplatformV1UnmanagedContainerModel)
updateTime	string (google-datetime)	Output only. Time when the BatchPredictionJob was most recently updated.

list

Successful response

Name	Datatype	Description
name	string	Output only. Resource name of the BatchPredictionJob.
completionStats	object	Output only. Statistics on completed and failed prediction instances. (id: GoogleCloudAiplatformV1CompletionStats)
createTime	string (google-datetime)	Output only. Time when the BatchPredictionJob was created.
dedicatedResources	object	The config of resources used by the Model during the batch prediction. If the Model supports DEDICATED_RESOURCES this config may be provided (and the job will use these resources), if the Model doesn't support AUTOMATIC_RESOURCES, this config must be provided. (id: GoogleCloudAiplatformV1BatchDedicatedResources)
disableContainerLogging	boolean	For custom-trained Models and AutoML Tabular Models, the container of the DeployedModel instances will send stderr and stdout streams to Cloud Logging by default. Please note that the logs incur cost, which are subject to Cloud Logging pricing. User can disable container logging by setting this flag to true.
displayName	string	Required. The user-defined name of this BatchPredictionJob.
encryptionSpec	object	Customer-managed encryption key options for a BatchPredictionJob. If this is set, then all resources created by the BatchPredictionJob will be encrypted with the provided encryption key. (id: GoogleCloudAiplatformV1EncryptionSpec)
endTime	string (google-datetime)	Output only. Time when the BatchPredictionJob entered any of the following states: JOB_STATE_SUCCEEDED, JOB_STATE_FAILED, JOB_STATE_CANCELLED.
error	object	Output only. Only populated when the job's state is JOB_STATE_FAILED or JOB_STATE_CANCELLED. (id: GoogleRpcStatus)
explanationSpec	object	Explanation configuration for this BatchPredictionJob. Can be specified only if generate_explanation is set to true. This value overrides the value of Model.explanation_spec. All fields of explanation_spec are optional in the request. If a field of the explanation_spec object is not populated, the corresponding field of the Model.explanation_spec object is inherited. (id: GoogleCloudAiplatformV1ExplanationSpec)
generateExplanation	boolean	Generate explanation with the batch prediction results. When set to true, the batch prediction output changes based on the predictions_format field of the BatchPredictionJob.output_config object: * bigquery: output includes a column named explanation. The value is a struct that conforms to the Explanation object. * jsonl: The JSON objects on each line include an additional entry keyed explanation. The value of the entry is a JSON object that conforms to the Explanation object. * csv: Generating explanations for CSV format is not supported. If this field is set to true, either the Model.explanation_spec or explanation_spec must be populated.
inputConfig	object	Required. Input configuration of the instances on which predictions are performed. The schema of any single instance may be specified via the Model's PredictSchemata's instance_schema_uri. (id: GoogleCloudAiplatformV1BatchPredictionJobInputConfig)
instanceConfig	object	Configuration for how to convert batch prediction input instances to the prediction instances that are sent to the Model. (id: GoogleCloudAiplatformV1BatchPredictionJobInstanceConfig)
labels	object	The labels with user-defined metadata to organize BatchPredictionJobs. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.
manualBatchTuningParameters	object	Immutable. Parameters configuring the batch behavior. Currently only applicable when dedicated_resources are used (in other cases Vertex AI does the tuning itself). (id: GoogleCloudAiplatformV1ManualBatchTuningParameters)
model	string	The name of the Model resource that produces the predictions via this job, must share the same ancestor Location. Starting this job has no impact on any existing deployments of the Model and their resources. Exactly one of model and unmanaged_container_model must be set. The model resource name may contain version id or version alias to specify the version. Example: projects/{project}/locations/{location}/models/{model}@2 or projects/{project}/locations/{location}/models/{model}@golden if no version is specified, the default version will be deployed. The model resource could also be a publisher model. Example: publishers/{publisher}/models/{model} or projects/{project}/locations/{location}/publishers/{publisher}/models/{model}
modelParameters	any	The parameters that govern the predictions. The schema of the parameters may be specified via the Model's PredictSchemata's parameters_schema_uri.
modelVersionId	string	Output only. The version ID of the Model that produces the predictions via this job.
outputConfig	object	Required. The Configuration specifying where output predictions should be written. The schema of any single prediction may be specified as a concatenation of Model's PredictSchemata's instance_schema_uri and prediction_schema_uri. (id: GoogleCloudAiplatformV1BatchPredictionJobOutputConfig)
outputInfo	object	Output only. Information further describing the output of this job. (id: GoogleCloudAiplatformV1BatchPredictionJobOutputInfo)
partialFailures	array	Output only. Partial failures encountered. For example, single files that can't be read. This field never exceeds 20 entries. Status details fields contain standard Google Cloud error details.
resourcesConsumed	object	Output only. Information about resources that had been consumed by this job. Provided in real time at best effort basis, as well as a final value once the job completes. Note: This field currently may be not populated for batch predictions that use AutoML Models. (id: GoogleCloudAiplatformV1ResourcesConsumed)
satisfiesPzi	boolean	Output only. Reserved for future use.
satisfiesPzs	boolean	Output only. Reserved for future use.
serviceAccount	string	The service account that the DeployedModel's container runs as. If not specified, a system generated one will be used, which has minimal permissions and the custom container, if used, may not have enough permission to access other Google Cloud resources. Users deploying the Model must have the iam.serviceAccounts.actAs permission on this service account.
startTime	string (google-datetime)	Output only. Time when the BatchPredictionJob for the first time entered the JOB_STATE_RUNNING state.
state	string	Output only. The detailed state of the job.
unmanagedContainerModel	object	Contains model information necessary to perform batch prediction without requiring uploading to model registry. Exactly one of model and unmanaged_container_model must be set. (id: GoogleCloudAiplatformV1UnmanagedContainerModel)
updateTime	string (google-datetime)	Output only. Time when the BatchPredictionJob was most recently updated.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
get	select	batchPredictionJobsId		Gets a BatchPredictionJob
list	select		parent, filter, pageSize, pageToken, readMask	Lists BatchPredictionJobs in a Location.
create	insert		parent	Creates a BatchPredictionJob. A BatchPredictionJob once created will right away be attempted to start.
delete	delete	projectsId, locationsId, batchPredictionJobsId		Deletes a BatchPredictionJob. Can only be called on jobs that already finished.
cancel	exec	projectsId, locationsId, batchPredictionJobsId		Cancels a BatchPredictionJob. Starts asynchronous cancellation on the BatchPredictionJob. The server makes the best effort to cancel the job, but success is not guaranteed. Clients can use JobService.GetBatchPredictionJob or other methods to check whether the cancellation succeeded or whether the job completed despite cancellation. On a successful cancellation, the BatchPredictionJob is not deleted;instead its BatchPredictionJob.state is set to CANCELLED. Any files already outputted by the job are not deleted.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
batchPredictionJobsId	string
locationsId	string
projectsId	string
filter	string
pageSize	integer (int32)
pageToken	string
parent	string
readMask	string (google-fieldmask)

SELECT examples

get

Gets a BatchPredictionJob

SELECT
name,
completionStats,
createTime,
dedicatedResources,
disableContainerLogging,
displayName,
encryptionSpec,
endTime,
error,
explanationSpec,
generateExplanation,
inputConfig,
instanceConfig,
labels,
manualBatchTuningParameters,
model,
modelParameters,
modelVersionId,
outputConfig,
outputInfo,
partialFailures,
resourcesConsumed,
satisfiesPzi,
satisfiesPzs,
serviceAccount,
startTime,
state,
unmanagedContainerModel,
updateTime
FROM google.aiplatform.batch_prediction_jobs
WHERE batchPredictionJobsId = '{{ batchPredictionJobsId }}' -- required;

list

Lists BatchPredictionJobs in a Location.

SELECT
name,
completionStats,
createTime,
dedicatedResources,
disableContainerLogging,
displayName,
encryptionSpec,
endTime,
error,
explanationSpec,
generateExplanation,
inputConfig,
instanceConfig,
labels,
manualBatchTuningParameters,
model,
modelParameters,
modelVersionId,
outputConfig,
outputInfo,
partialFailures,
resourcesConsumed,
satisfiesPzi,
satisfiesPzs,
serviceAccount,
startTime,
state,
unmanagedContainerModel,
updateTime
FROM google.aiplatform.batch_prediction_jobs
WHERE parent = '{{ parent }}'
AND filter = '{{ filter }}'
AND pageSize = '{{ pageSize }}'
AND pageToken = '{{ pageToken }}'
AND readMask = '{{ readMask }}';

INSERT examples

create

Creates a BatchPredictionJob. A BatchPredictionJob once created will right away be attempted to start.

INSERT INTO google.aiplatform.batch_prediction_jobs (
data__displayName,
data__model,
data__unmanagedContainerModel,
data__inputConfig,
data__instanceConfig,
data__modelParameters,
data__outputConfig,
data__dedicatedResources,
data__serviceAccount,
data__manualBatchTuningParameters,
data__generateExplanation,
data__explanationSpec,
data__labels,
data__encryptionSpec,
data__disableContainerLogging,
parent
)
SELECT 
'{{ displayName }}',
'{{ model }}',
'{{ unmanagedContainerModel }}',
'{{ inputConfig }}',
'{{ instanceConfig }}',
'{{ modelParameters }}',
'{{ outputConfig }}',
'{{ dedicatedResources }}',
'{{ serviceAccount }}',
'{{ manualBatchTuningParameters }}',
{{ generateExplanation }},
'{{ explanationSpec }}',
'{{ labels }}',
'{{ encryptionSpec }}',
{{ disableContainerLogging }},
'{{ parent }}'
RETURNING
name,
completionStats,
createTime,
dedicatedResources,
disableContainerLogging,
displayName,
encryptionSpec,
endTime,
error,
explanationSpec,
generateExplanation,
inputConfig,
instanceConfig,
labels,
manualBatchTuningParameters,
model,
modelParameters,
modelVersionId,
outputConfig,
outputInfo,
partialFailures,
resourcesConsumed,
satisfiesPzi,
satisfiesPzs,
serviceAccount,
startTime,
state,
unmanagedContainerModel,
updateTime
;

Manifest

# Description fields are for documentation purposes
- name: batch_prediction_jobs
  props:
    - name: displayName
      value: string
      description: >
        Required. The user-defined name of this BatchPredictionJob.
        
    - name: model
      value: string
      description: >
        The name of the Model resource that produces the predictions via this job, must share the same ancestor Location. Starting this job has no impact on any existing deployments of the Model and their resources. Exactly one of model and unmanaged_container_model must be set. The model resource name may contain version id or version alias to specify the version. Example: `projects/{project}/locations/{location}/models/{model}@2` or `projects/{project}/locations/{location}/models/{model}@golden` if no version is specified, the default version will be deployed. The model resource could also be a publisher model. Example: `publishers/{publisher}/models/{model}` or `projects/{project}/locations/{location}/publishers/{publisher}/models/{model}`
        
    - name: unmanagedContainerModel
      value: object
      description: >
        Contains model information necessary to perform batch prediction without requiring uploading to model registry. Exactly one of model and unmanaged_container_model must be set.
        
    - name: inputConfig
      value: object
      description: >
        Required. Input configuration of the instances on which predictions are performed. The schema of any single instance may be specified via the Model's PredictSchemata's instance_schema_uri.
        
    - name: instanceConfig
      value: object
      description: >
        Configuration for how to convert batch prediction input instances to the prediction instances that are sent to the Model.
        
    - name: modelParameters
      value: any
      description: >
        The parameters that govern the predictions. The schema of the parameters may be specified via the Model's PredictSchemata's parameters_schema_uri.
        
    - name: outputConfig
      value: object
      description: >
        Required. The Configuration specifying where output predictions should be written. The schema of any single prediction may be specified as a concatenation of Model's PredictSchemata's instance_schema_uri and prediction_schema_uri.
        
    - name: dedicatedResources
      value: object
      description: >
        The config of resources used by the Model during the batch prediction. If the Model supports DEDICATED_RESOURCES this config may be provided (and the job will use these resources), if the Model doesn't support AUTOMATIC_RESOURCES, this config must be provided.
        
    - name: serviceAccount
      value: string
      description: >
        The service account that the DeployedModel's container runs as. If not specified, a system generated one will be used, which has minimal permissions and the custom container, if used, may not have enough permission to access other Google Cloud resources. Users deploying the Model must have the `iam.serviceAccounts.actAs` permission on this service account.
        
    - name: manualBatchTuningParameters
      value: object
      description: >
        Immutable. Parameters configuring the batch behavior. Currently only applicable when dedicated_resources are used (in other cases Vertex AI does the tuning itself).
        
    - name: generateExplanation
      value: boolean
      description: >
        Generate explanation with the batch prediction results. When set to `true`, the batch prediction output changes based on the `predictions_format` field of the BatchPredictionJob.output_config object: * `bigquery`: output includes a column named `explanation`. The value is a struct that conforms to the Explanation object. * `jsonl`: The JSON objects on each line include an additional entry keyed `explanation`. The value of the entry is a JSON object that conforms to the Explanation object. * `csv`: Generating explanations for CSV format is not supported. If this field is set to true, either the Model.explanation_spec or explanation_spec must be populated.
        
    - name: explanationSpec
      value: object
      description: >
        Explanation configuration for this BatchPredictionJob. Can be specified only if generate_explanation is set to `true`. This value overrides the value of Model.explanation_spec. All fields of explanation_spec are optional in the request. If a field of the explanation_spec object is not populated, the corresponding field of the Model.explanation_spec object is inherited.
        
    - name: labels
      value: object
      description: >
        The labels with user-defined metadata to organize BatchPredictionJobs. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.
        
    - name: encryptionSpec
      value: object
      description: >
        Customer-managed encryption key options for a BatchPredictionJob. If this is set, then all resources created by the BatchPredictionJob will be encrypted with the provided encryption key.
        
    - name: disableContainerLogging
      value: boolean
      description: >
        For custom-trained Models and AutoML Tabular Models, the container of the DeployedModel instances will send `stderr` and `stdout` streams to Cloud Logging by default. Please note that the logs incur cost, which are subject to [Cloud Logging pricing](https://cloud.google.com/logging/pricing). User can disable container logging by setting this flag to true.
        
    - name: parent
      value: string

DELETE examples

delete

Deletes a BatchPredictionJob. Can only be called on jobs that already finished.

DELETE FROM google.aiplatform.batch_prediction_jobs
WHERE projectsId = '{{ projectsId }}' --required
AND locationsId = '{{ locationsId }}' --required
AND batchPredictionJobsId = '{{ batchPredictionJobsId }}' --required;

Lifecycle Methods

cancel

Cancels a BatchPredictionJob. Starts asynchronous cancellation on the BatchPredictionJob. The server makes the best effort to cancel the job, but success is not guaranteed. Clients can use JobService.GetBatchPredictionJob or other methods to check whether the cancellation succeeded or whether the job completed despite cancellation. On a successful cancellation, the BatchPredictionJob is not deleted;instead its BatchPredictionJob.state is set to CANCELLED. Any files already outputted by the job are not deleted.

EXEC google.aiplatform.batch_prediction_jobs.cancel 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@batchPredictionJobsId='{{ batchPredictionJobsId }}' --required;