jobs

Creates, updates, deletes, gets or lists a jobs resource.

Overview

Name	jobs
Type	Resource
Id	google.bigquery.jobs

Fields

The following fields are returned by SELECT queries:

get

Name	Datatype	Description
id	string	Output only. Opaque ID field of the job.
configuration	object	Required. Describes the job configuration. (id: JobConfiguration)
etag	string	Output only. A hash of this resource.
jobCreationReason	object	Output only. The reason why a Job was created. (id: JobCreationReason)
jobReference	object	Optional. Reference describing the unique-per-user name of the job. (id: JobReference)
kind	string	Output only. The type of the resource. (default: bigquery#job)
principal_subject	string	Output only. [Full-projection-only] String representation of identity of requesting party. Populated for both first- and third-party identities. Only present for APIs that support third-party identities.
selfLink	string	Output only. A URL that can be used to access the resource again.
statistics	object	Output only. Information about the job, including starting time and ending time of the job. (id: JobStatistics)
status	object	Output only. The status of this job. Examine this value when polling an asynchronous job to see if the job is complete. (id: JobStatus)
user_email	string	Output only. Email address of the user who ran the job.

list

Name	Datatype	Description
id	string	Unique opaque ID of the job.
configuration	object	Required. Describes the job configuration. (id: JobConfiguration)
errorResult	object	Error details. (id: ErrorProto)
jobReference	object	Unique opaque ID of the job. (id: JobReference)
kind	string	The resource type.
principal_subject	string	[Full-projection-only] String representation of identity of requesting party. Populated for both first- and third-party identities. Only present for APIs that support third-party identities.
state	string	Running state of the job. When the state is DONE, errorResult can be checked to determine whether the job succeeded or failed.
statistics	object	Output only. Information about the job, including starting time and ending time of the job. (id: JobStatistics)
status	object	[Full-projection-only] Describes the status of this job. (id: JobStatus)
user_email	string	[Full-projection-only] Email address of the user who ran the job.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
get	select	projectId, +jobId	location	Returns information about a specific job. Job information is available for a six month period after creation. Requires that you're the person who ran the job, or have the Is Owner project role. # IAM Permissions Requires the bigquery.jobs.get permission on the job resource. If the user matches the creator of the job, the bigquery.jobs.create permission on the project is required instead.
list	select	projectId	pageToken, maxCreationTime, allUsers, parentJobId, maxResults, minCreationTime, projection, stateFilter	Lists all jobs that you started in the specified project. Job information is available for a six month period after creation. The job list is sorted in reverse chronological order, by job creation time. Requires the Can View project role, or the Is Owner project role if you set the allUsers property. # IAM Permissions Requires no specific IAM permission(s) to use this method. Users are able to list the jobs they created. Additional access is granted based on the following permissions: - Users with the bigquery.jobs.listAll permission can list all jobs with all metadata. - Users with the bigquery.jobs.list permission can list all jobs, but with redacted information for jobs they did not create.
insert	insert	projectId		Starts a new asynchronous job. This API has two different kinds of endpoint URIs, as this method supports a variety of use cases. * The Metadata URI is used for most interactions, as it accepts the job configuration directly. * The Upload URI is ONLY for the case when you're sending both a load job configuration and a data stream together. In this case, the Upload URI accepts the job configuration and the data as two distinct multipart MIME parts. # IAM Permissions Requires the bigquery.jobs.create permission on the project resource. Additional permissions are required depending on the job type: - Load, Export, and Copy jobs: Generally require data-level permissions such as bigquery.tables.export or access to external storage. - Query jobs: Permissions are dependent on the SQL statement. Complex queries (DDL, DCL) may require additional permissions to create reservations, modify IAM policies, or update project settings.
delete	delete	projectId, +jobId	location	Requests the deletion of the metadata of a job. This call returns when the job's metadata is deleted. # IAM Permissions Requires the bigquery.jobs.delete permission on the job resource.
query	exec	projectId		Runs a BigQuery SQL query synchronously and returns query results if the query completes within a specified timeout. # IAM Permissions Requires the bigquery.jobs.create permission on the project resource. Data-level permissions are highly dependent on the SQL statement being executed. While standard queries require data access (such as bigquery.tables.getData), complex operations like DDL or DCL may require permissions to manage reservations, IAM policies, or project settings.
cancel	exec	projectId, +jobId	location	Requests that a job be cancelled. This call will return immediately, and the client will need to poll for the job status to see if the cancel completed successfully. Cancelled jobs may still incur costs. # IAM Permissions Requires the bigquery.jobs.update permission on the job resource. If the user matches the creator of the job, the bigquery.jobs.create permission on the project is required instead.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
+jobId	string
projectId	string
allUsers	boolean
location	string
maxCreationTime	string (uint64)
maxResults	integer (uint32)
minCreationTime	string (uint64)
pageToken	string
parentJobId	string
projection	string
stateFilter	string

SELECT examples

get

Returns information about a specific job. Job information is available for a six month period after creation. Requires that you're the person who ran the job, or have the Is Owner project role. # IAM Permissions Requires the bigquery.jobs.get permission on the job resource. If the user matches the creator of the job, the bigquery.jobs.create permission on the project is required instead.

SELECT
id,
configuration,
etag,
jobCreationReason,
jobReference,
kind,
principal_subject,
selfLink,
statistics,
status,
user_email
FROM google.bigquery.jobs
WHERE projectId = '{{ projectId }}' -- required
AND +jobId = '{{ +jobId }}' -- required
AND location = '{{ location }}'
;

list

Lists all jobs that you started in the specified project. Job information is available for a six month period after creation. The job list is sorted in reverse chronological order, by job creation time. Requires the Can View project role, or the Is Owner project role if you set the allUsers property. # IAM Permissions Requires no specific IAM permission(s) to use this method. Users are able to list the jobs they created. Additional access is granted based on the following permissions: - Users with the bigquery.jobs.listAll permission can list all jobs with all metadata. - Users with the bigquery.jobs.list permission can list all jobs, but with redacted information for jobs they did not create.

SELECT
id,
configuration,
errorResult,
jobReference,
kind,
principal_subject,
state,
statistics,
status,
user_email
FROM google.bigquery.jobs
WHERE projectId = '{{ projectId }}' -- required
AND pageToken = '{{ pageToken }}'
AND maxCreationTime = '{{ maxCreationTime }}'
AND allUsers = '{{ allUsers }}'
AND parentJobId = '{{ parentJobId }}'
AND maxResults = '{{ maxResults }}'
AND minCreationTime = '{{ minCreationTime }}'
AND projection = '{{ projection }}'
AND stateFilter = '{{ stateFilter }}'
;

INSERT examples

insert

Starts a new asynchronous job. This API has two different kinds of endpoint URIs, as this method supports a variety of use cases. * The Metadata URI is used for most interactions, as it accepts the job configuration directly. * The Upload URI is ONLY for the case when you're sending both a load job configuration and a data stream together. In this case, the Upload URI accepts the job configuration and the data as two distinct multipart MIME parts. # IAM Permissions Requires the bigquery.jobs.create permission on the project resource. Additional permissions are required depending on the job type: - Load, Export, and Copy jobs: Generally require data-level permissions such as bigquery.tables.export or access to external storage. - Query jobs: Permissions are dependent on the SQL statement. Complex queries (DDL, DCL) may require additional permissions to create reservations, modify IAM policies, or update project settings.

INSERT INTO google.bigquery.jobs (
data__configuration,
data__jobReference,
projectId
)
SELECT 
'{{ configuration }}',
'{{ jobReference }}',
'{{ projectId }}'
RETURNING
id,
configuration,
etag,
jobCreationReason,
jobReference,
kind,
principal_subject,
selfLink,
statistics,
status,
user_email
;

Manifest

# Description fields are for documentation purposes
- name: jobs
props:
  - name: projectId
    value: "{{ projectId }}"
    description: Required parameter for the jobs resource.
  - name: configuration
    description: |
      Required. Describes the job configuration.
    value:
      copy:
        destinationExpirationTime: "{{ destinationExpirationTime }}"
        operationType: "{{ operationType }}"
        createDisposition: "{{ createDisposition }}"
        writeDisposition: "{{ writeDisposition }}"
        sourceTables:
          - tableId: "{{ tableId }}"
            datasetId: "{{ datasetId }}"
            projectId: "{{ projectId }}"
        destinationTable:
          tableId: "{{ tableId }}"
          datasetId: "{{ datasetId }}"
          projectId: "{{ projectId }}"
        destinationEncryptionConfiguration:
          kmsKeyName: "{{ kmsKeyName }}"
        sourceTable:
          tableId: "{{ tableId }}"
          datasetId: "{{ datasetId }}"
          projectId: "{{ projectId }}"
      extract:
        sourceModel:
          datasetId: "{{ datasetId }}"
          modelId: "{{ modelId }}"
          projectId: "{{ projectId }}"
        destinationFormat: "{{ destinationFormat }}"
        sourceTable:
          tableId: "{{ tableId }}"
          datasetId: "{{ datasetId }}"
          projectId: "{{ projectId }}"
        compression: "{{ compression }}"
        printHeader: {{ printHeader }}
        destinationUri: "{{ destinationUri }}"
        fieldDelimiter: "{{ fieldDelimiter }}"
        destinationUris:
          - "{{ destinationUris }}"
        modelExtractOptions:
          trialId: "{{ trialId }}"
        useAvroLogicalTypes: {{ useAvroLogicalTypes }}
      query:
        preserveNulls: {{ preserveNulls }}
        createDisposition: "{{ createDisposition }}"
        defaultDataset:
          datasetId: "{{ datasetId }}"
          projectId: "{{ projectId }}"
        parameterMode: "{{ parameterMode }}"
        writeDisposition: "{{ writeDisposition }}"
        rangePartitioning:
          field: "{{ field }}"
          range:
            interval: "{{ interval }}"
            end: "{{ end }}"
            start: "{{ start }}"
        tableDefinitions: "{{ tableDefinitions }}"
        queryParameters:
          - parameterValue:
              arrayValues:
                - arrayValues: "{{ arrayValues }}"
                  value: "{{ value }}"
                  rangeValue:
                    end: "{{ end }}"
                    start: "{{ start }}"
                  structValues: "{{ structValues }}"
              value: "{{ value }}"
              rangeValue:
                end: "{{ end }}"
                start: "{{ start }}"
              structValues: "{{ structValues }}"
            parameterType:
              structTypes:
                - description: "{{ description }}"
                  name: "{{ name }}"
                  type:
                    structTypes: "{{ structTypes }}"
                    rangeElementType: "{{ rangeElementType }}"
                    type: "{{ type }}"
                    arrayType: "{{ arrayType }}"
                    timestampPrecision: "{{ timestampPrecision }}"
              rangeElementType:
                structTypes: "{{ structTypes }}"
                rangeElementType: "{{ rangeElementType }}"
                type: "{{ type }}"
                arrayType: "{{ arrayType }}"
                timestampPrecision: "{{ timestampPrecision }}"
              type: "{{ type }}"
              arrayType:
                structTypes: "{{ structTypes }}"
                rangeElementType: "{{ rangeElementType }}"
                type: "{{ type }}"
                arrayType: "{{ arrayType }}"
                timestampPrecision: "{{ timestampPrecision }}"
              timestampPrecision: "{{ timestampPrecision }}"
            name: "{{ name }}"
        schemaUpdateOptions:
          - "{{ schemaUpdateOptions }}"
        userDefinedFunctionResources:
          - resourceUri: "{{ resourceUri }}"
            inlineCode: "{{ inlineCode }}"
        continuous: {{ continuous }}
        scriptOptions:
          statementByteBudget: "{{ statementByteBudget }}"
          statementTimeoutMs: "{{ statementTimeoutMs }}"
          keyResultStatement: "{{ keyResultStatement }}"
        timePartitioning:
          expirationMs: "{{ expirationMs }}"
          requirePartitionFilter: {{ requirePartitionFilter }}
          field: "{{ field }}"
          type: "{{ type }}"
        clustering:
          fields:
            - "{{ fields }}"
        allowLargeResults: {{ allowLargeResults }}
        flattenResults: {{ flattenResults }}
        systemVariables:
          types: "{{ types }}"
          values: "{{ values }}"
        connectionProperties:
          - key: "{{ key }}"
            value: "{{ value }}"
        useLegacySql: {{ useLegacySql }}
        writeIncrementalResults: {{ writeIncrementalResults }}
        useQueryCache: {{ useQueryCache }}
        maximumBytesBilled: "{{ maximumBytesBilled }}"
        query: "{{ query }}"
        createSession: {{ createSession }}
        destinationEncryptionConfiguration:
          kmsKeyName: "{{ kmsKeyName }}"
        maximumBillingTier: {{ maximumBillingTier }}
        destinationTable:
          tableId: "{{ tableId }}"
          datasetId: "{{ datasetId }}"
          projectId: "{{ projectId }}"
        priority: "{{ priority }}"
      jobType: "{{ jobType }}"
      load:
        createDisposition: "{{ createDisposition }}"
        allowJaggedRows: {{ allowJaggedRows }}
        timeZone: "{{ timeZone }}"
        timeFormat: "{{ timeFormat }}"
        timestampTargetPrecision:
          - {{ timestampTargetPrecision }}
        ignoreUnknownValues: {{ ignoreUnknownValues }}
        sourceFormat: "{{ sourceFormat }}"
        sourceColumnMatch: "{{ sourceColumnMatch }}"
        rangePartitioning:
          field: "{{ field }}"
          range:
            interval: "{{ interval }}"
            end: "{{ end }}"
            start: "{{ start }}"
        writeDisposition: "{{ writeDisposition }}"
        maxBadRecords: {{ maxBadRecords }}
        autodetect: {{ autodetect }}
        dateFormat: "{{ dateFormat }}"
        timestampFormat: "{{ timestampFormat }}"
        schemaUpdateOptions:
          - "{{ schemaUpdateOptions }}"
        allowQuotedNewlines: {{ allowQuotedNewlines }}
        projectionFields:
          - "{{ projectionFields }}"
        clustering:
          fields:
            - "{{ fields }}"
        fieldDelimiter: "{{ fieldDelimiter }}"
        useAvroLogicalTypes: {{ useAvroLogicalTypes }}
        referenceFileSchemaUri: "{{ referenceFileSchemaUri }}"
        sourceUris:
          - "{{ sourceUris }}"
        timePartitioning:
          expirationMs: "{{ expirationMs }}"
          requirePartitionFilter: {{ requirePartitionFilter }}
          field: "{{ field }}"
          type: "{{ type }}"
        schemaInline: "{{ schemaInline }}"
        encoding: "{{ encoding }}"
        fileSetSpecType: "{{ fileSetSpecType }}"
        destinationTableProperties:
          friendlyName: "{{ friendlyName }}"
          expirationTime: "{{ expirationTime }}"
          description: "{{ description }}"
          labels: "{{ labels }}"
        preserveAsciiControlCharacters: {{ preserveAsciiControlCharacters }}
        columnNameCharacterMap: "{{ columnNameCharacterMap }}"
        nullMarkers:
          - "{{ nullMarkers }}"
        connectionProperties:
          - key: "{{ key }}"
            value: "{{ value }}"
        copyFilesOnly: {{ copyFilesOnly }}
        hivePartitioningOptions:
          mode: "{{ mode }}"
          requirePartitionFilter: {{ requirePartitionFilter }}
          sourceUriPrefix: "{{ sourceUriPrefix }}"
          fields:
            - "{{ fields }}"
        quote: "{{ quote }}"
        decimalTargetTypes:
          - "{{ decimalTargetTypes }}"
        schema:
          fields:
            - categories:
                names: "{{ names }}"
              scale: "{{ scale }}"
              roundingMode: "{{ roundingMode }}"
              foreignTypeDefinition: "{{ foreignTypeDefinition }}"
              rangeElementType:
                type: "{{ type }}"
              type: "{{ type }}"
              dataGovernanceTagsInfo:
                dataGovernanceTags: "{{ dataGovernanceTags }}"
              dataPolicies: "{{ dataPolicies }}"
              timestampPrecision: "{{ timestampPrecision }}"
              collation: "{{ collation }}"
              dataPolicyList:
                dataPolicies: "{{ dataPolicies }}"
              name: "{{ name }}"
              precision: "{{ precision }}"
              defaultValueExpression: "{{ defaultValueExpression }}"
              generatedColumn:
                generatedExpressionInfo: "{{ generatedExpressionInfo }}"
                generatedMode: "{{ generatedMode }}"
              fields: "{{ fields }}"
              maxLength: "{{ maxLength }}"
              mode: "{{ mode }}"
              policyTags:
                names: "{{ names }}"
              description: "{{ description }}"
          foreignTypeInfo:
            typeSystem: "{{ typeSystem }}"
        schemaInlineFormat: "{{ schemaInlineFormat }}"
        destinationEncryptionConfiguration:
          kmsKeyName: "{{ kmsKeyName }}"
        createSession: {{ createSession }}
        parquetOptions:
          mapTargetType: "{{ mapTargetType }}"
          enableListInference: {{ enableListInference }}
          enumAsString: {{ enumAsString }}
        nullMarker: "{{ nullMarker }}"
        skipLeadingRows: {{ skipLeadingRows }}
        jsonExtension: "{{ jsonExtension }}"
        datetimeFormat: "{{ datetimeFormat }}"
        destinationTable:
          tableId: "{{ tableId }}"
          datasetId: "{{ datasetId }}"
          projectId: "{{ projectId }}"
      jobTimeoutMs: "{{ jobTimeoutMs }}"
      labels: "{{ labels }}"
      reservation: "{{ reservation }}"
      dryRun: {{ dryRun }}
      maxSlots: {{ maxSlots }}
  - name: jobReference
    description: |
      Optional. Reference describing the unique-per-user name of the job.
    value:
      location: "{{ location }}"
      projectId: "{{ projectId }}"
      jobId: "{{ jobId }}"

DELETE examples

delete

Requests the deletion of the metadata of a job. This call returns when the job's metadata is deleted. # IAM Permissions Requires the bigquery.jobs.delete permission on the job resource.

DELETE FROM google.bigquery.jobs
WHERE projectId = '{{ projectId }}' --required
AND +jobId = '{{ +jobId }}' --required
AND location = '{{ location }}'
;

Lifecycle Methods

query

Runs a BigQuery SQL query synchronously and returns query results if the query completes within a specified timeout. # IAM Permissions Requires the bigquery.jobs.create permission on the project resource. Data-level permissions are highly dependent on the SQL statement being executed. While standard queries require data access (such as bigquery.tables.getData), complex operations like DDL or DCL may require permissions to manage reservations, IAM policies, or project settings.

EXEC google.bigquery.jobs.query 
@projectId='{{ projectId }}' --required 
@@json=
'{
"preserveNulls": {{ preserveNulls }}, 
"location": "{{ location }}", 
"labels": "{{ labels }}", 
"defaultDataset": "{{ defaultDataset }}", 
"queryResultsFormat": "{{ queryResultsFormat }}", 
"timeoutMs": {{ timeoutMs }}, 
"parameterMode": "{{ parameterMode }}", 
"requestId": "{{ requestId }}", 
"queryParameters": "{{ queryParameters }}", 
"continuous": {{ continuous }}, 
"maxResults": {{ maxResults }}, 
"connectionProperties": "{{ connectionProperties }}", 
"kind": "{{ kind }}", 
"jobTimeoutMs": "{{ jobTimeoutMs }}", 
"formatOptions": "{{ formatOptions }}", 
"useLegacySql": {{ useLegacySql }}, 
"writeIncrementalResults": {{ writeIncrementalResults }}, 
"arrowSerializationOptions": "{{ arrowSerializationOptions }}", 
"maximumBytesBilled": "{{ maximumBytesBilled }}", 
"query": "{{ query }}", 
"useQueryCache": {{ useQueryCache }}, 
"destinationEncryptionConfiguration": "{{ destinationEncryptionConfiguration }}", 
"reservation": "{{ reservation }}", 
"createSession": {{ createSession }}, 
"dryRun": {{ dryRun }}, 
"maxSlots": {{ maxSlots }}, 
"jobCreationMode": "{{ jobCreationMode }}"
}'
;

cancel

Requests that a job be cancelled. This call will return immediately, and the client will need to poll for the job status to see if the cancel completed successfully. Cancelled jobs may still incur costs. # IAM Permissions Requires the bigquery.jobs.update permission on the job resource. If the user matches the creator of the job, the bigquery.jobs.create permission on the project is required instead.

EXEC google.bigquery.jobs.cancel 
@projectId='{{ projectId }}' --required, 
@+jobId='{{ +jobId }}' --required, 
@location='{{ location }}'
;