datasets

Creates, updates, deletes, gets or lists a datasets resource.

Overview

Name	`datasets`
Type	Resource
Id	`google.bigquery.datasets`

Fields

The following fields are returned by SELECT queries:

get
list

Successful response

Name	Datatype	Description
`id`	`string`	Output only. The fully-qualified unique name of the dataset in the format projectId:datasetId. The dataset name without the project name is given in the datasetId field. When creating a new dataset, leave this field blank, and instead specify the datasetId field.
`access`	`array`	Optional. An array of objects that define dataset access for one or more entities. You can set this property when inserting or updating a dataset in order to control who is allowed to access the data. If unspecified at dataset creation time, BigQuery adds default dataset access for the following entities: access.specialGroup: projectReaders; access.role: READER; access.specialGroup: projectWriters; access.role: WRITER; access.specialGroup: projectOwners; access.role: OWNER; access.userByEmail: [dataset creator email]; access.role: OWNER; If you patch a dataset, then this field is overwritten by the patched dataset's access field. To add entities, you must supply the entire existing access array in addition to any new entities that you want to add.
`creationTime`	`string (int64)`	Output only. The time when this dataset was created, in milliseconds since the epoch.
`datasetReference`	`object`	Required. A reference that identifies the dataset. (id: DatasetReference)
`defaultCollation`	`string`	Optional. Defines the default collation specification of future tables created in the dataset. If a table is created in this dataset without table-level default collation, then the table inherits the dataset default collation, which is applied to the string fields that do not have explicit collation specified. A change to this field affects only tables created afterwards, and does not alter the existing tables. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
`defaultEncryptionConfiguration`	`object`	The default encryption key for all tables in the dataset. After this property is set, the encryption key of all newly-created tables in the dataset is set to this value unless the table creation request or query explicitly overrides the key. (id: EncryptionConfiguration)
`defaultPartitionExpirationMs`	`string (int64)`	This default partition expiration, expressed in milliseconds. When new time-partitioned tables are created in a dataset where this property is set, the table will inherit this value, propagated as the `TimePartitioning.expirationMs` property on the new table. If you set `TimePartitioning.expirationMs` explicitly when creating a table, the `defaultPartitionExpirationMs` of the containing dataset is ignored. When creating a partitioned table, if `defaultPartitionExpirationMs` is set, the `defaultTableExpirationMs` value is ignored and the table will not be inherit a table expiration deadline.
`defaultRoundingMode`	`string`	Optional. Defines the default rounding mode specification of new tables created within this dataset. During table creation, if this field is specified, the table within this dataset will inherit the default rounding mode of the dataset. Setting the default rounding mode on a table overrides this option. Existing tables in the dataset are unaffected. If columns are defined during that table creation, they will immediately inherit the table's default rounding mode, unless otherwise specified.
`defaultTableExpirationMs`	`string (int64)`	Optional. The default lifetime of all tables in the dataset, in milliseconds. The minimum lifetime value is 3600000 milliseconds (one hour). To clear an existing default expiration with a PATCH request, set to 0. Once this property is set, all newly-created tables in the dataset will have an expirationTime property set to the creation time plus the value in this property, and changing the value will only affect new tables, not existing ones. When the expirationTime for a given table is reached, that table will be deleted automatically. If a table's expirationTime is modified or removed before the table expires, or if you provide an explicit expirationTime when creating a table, that value takes precedence over the default expiration time indicated by this property.
`description`	`string`	Optional. A user-friendly description of the dataset.
`etag`	`string`	Output only. A hash of the resource.
`externalCatalogDatasetOptions`	`object`	Optional. Options defining open source compatible datasets living in the BigQuery catalog. Contains metadata of open source database, schema or namespace represented by the current dataset. (id: ExternalCatalogDatasetOptions)
`externalDatasetReference`	`object`	Optional. Reference to a read-only external dataset defined in data catalogs outside of BigQuery. Filled out when the dataset type is EXTERNAL. (id: ExternalDatasetReference)
`friendlyName`	`string`	Optional. A descriptive name for the dataset.
`isCaseInsensitive`	`boolean`	Optional. TRUE if the dataset and its table names are case-insensitive, otherwise FALSE. By default, this is FALSE, which means the dataset and its table names are case-sensitive. This field does not affect routine references.
`kind`	`string`	Output only. The resource type. (default: bigquery#dataset)
`labels`	`object`	The labels associated with this dataset. You can use these to organize and group your datasets. You can set this property when inserting or updating a dataset. See Creating and Updating Dataset Labels for more information.
`lastModifiedTime`	`string (int64)`	Output only. The date when this dataset was last modified, in milliseconds since the epoch.
`linkedDatasetMetadata`	`object`	Output only. Metadata about the LinkedDataset. Filled out when the dataset type is LINKED. (id: LinkedDatasetMetadata)
`linkedDatasetSource`	`object`	Optional. The source dataset reference when the dataset is of type LINKED. For all other dataset types it is not set. This field cannot be updated once it is set. Any attempt to update this field using Update and Patch API Operations will be ignored. (id: LinkedDatasetSource)
`location`	`string`	The geographic location where the dataset should reside. See https://cloud.google.com/bigquery/docs/locations for supported locations.
`maxTimeTravelHours`	`string (int64)`	Optional. Defines the time travel window in hours. The value can be from 48 to 168 hours (2 to 7 days). The default value is 168 hours if this is not set.
`resourceTags`	`object`	Optional. The tags attached to this dataset. Tag keys are globally unique. Tag key is expected to be in the namespaced format, for example "123456789012/environment" where 123456789012 is the ID of the parent organization or project resource for this tag key. Tag value is expected to be the short name, for example "Production". See Tag definitions for more details.
`restrictions`	`object`	Optional. Output only. Restriction config for all tables and dataset. If set, restrict certain accesses on the dataset and all its tables based on the config. See Data egress for more details. (id: RestrictionConfig)
`satisfiesPzi`	`boolean`	Output only. Reserved for future use.
`satisfiesPzs`	`boolean`	Output only. Reserved for future use.
`selfLink`	`string`	Output only. A URL that can be used to access the resource again. You can use this URL in Get or Update requests to the resource.
`storageBillingModel`	`string`	Optional. Updates storage_billing_model for the dataset.
`tags`	`array`	Output only. Tags for the dataset. To provide tags as inputs, use the `resourceTags` field.
`type`	`string`	Output only. Same as `type` in `ListFormatDataset`. The type of the dataset, one of: * DEFAULT - only accessible by owner and authorized accounts, * PUBLIC - accessible by everyone, * LINKED - linked dataset, * EXTERNAL - dataset with definition in external metadata catalog.

Successful response

Name	Datatype	Description
`id`	`string`	The fully-qualified, unique, opaque ID of the dataset.
`datasetReference`	`object`	The dataset reference. Use this property to access specific parts of the dataset's ID, such as project ID or dataset ID. (id: DatasetReference)
`externalDatasetReference`	`object`	Output only. Reference to a read-only external dataset defined in data catalogs outside of BigQuery. Filled out when the dataset type is EXTERNAL. (id: ExternalDatasetReference)
`friendlyName`	`string`	An alternate name for the dataset. The friendly name is purely decorative in nature.
`kind`	`string`	The resource type. This property always returns the value "bigquery#dataset"
`labels`	`object`	The labels associated with this dataset. You can use these to organize and group your datasets.
`location`	`string`	The geographic location where the dataset resides.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
`get`	`select`	`projectId`, `+datasetId`	`accessPolicyVersion`, `datasetView`	Returns the dataset specified by datasetID.
`list`	`select`	`projectId`	`all`, `filter`, `maxResults`, `pageToken`	Lists all datasets in the specified project to which the user has been granted the READER dataset role.
`insert`	`insert`	`projectId`	`accessPolicyVersion`	Creates a new empty dataset.
`patch`	`update`	`projectId`, `+datasetId`	`accessPolicyVersion`, `updateMode`	Updates information in an existing dataset. The update method replaces the entire dataset resource, whereas the patch method only replaces fields that are provided in the submitted dataset resource. This method supports RFC5789 patch semantics.
`update`	`replace`	`projectId`, `+datasetId`	`accessPolicyVersion`, `updateMode`	Updates information in an existing dataset. The update method replaces the entire dataset resource, whereas the patch method only replaces fields that are provided in the submitted dataset resource.
`delete`	`delete`	`projectId`, `+datasetId`	`deleteContents`	Deletes the dataset specified by the datasetId value. Before you can delete a dataset, you must delete all its tables, either manually or by specifying deleteContents. Immediately after deletion, you can create another dataset with the same name.
`undelete`	`exec`	`projectId`, `+datasetId`		Undeletes a dataset which is within time travel window based on datasetId. If a time is specified, the dataset version deleted at that time is undeleted, else the last live version is undeleted.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
`+datasetId`	`string`
`projectId`	`string`
`accessPolicyVersion`	`integer (int32)`
`all`	`boolean`
`datasetView`	`string`
`deleteContents`	`boolean`
`filter`	`string`
`maxResults`	`integer (uint32)`
`pageToken`	`string`
`updateMode`	`string`

`SELECT` examples

get
list

Returns the dataset specified by datasetID.

SELECT
id,
access,
creationTime,
datasetReference,
defaultCollation,
defaultEncryptionConfiguration,
defaultPartitionExpirationMs,
defaultRoundingMode,
defaultTableExpirationMs,
description,
etag,
externalCatalogDatasetOptions,
externalDatasetReference,
friendlyName,
isCaseInsensitive,
kind,
labels,
lastModifiedTime,
linkedDatasetMetadata,
linkedDatasetSource,
location,
maxTimeTravelHours,
resourceTags,
restrictions,
satisfiesPzi,
satisfiesPzs,
selfLink,
storageBillingModel,
tags,
type
FROM google.bigquery.datasets
WHERE projectId = '{{ projectId }}' -- required
AND +datasetId = '{{ +datasetId }}' -- required
AND accessPolicyVersion = '{{ accessPolicyVersion }}'
AND datasetView = '{{ datasetView }}';

Lists all datasets in the specified project to which the user has been granted the READER dataset role.

SELECT
id,
datasetReference,
externalDatasetReference,
friendlyName,
kind,
labels,
location
FROM google.bigquery.datasets
WHERE projectId = '{{ projectId }}' -- required
AND all = '{{ all }}'
AND filter = '{{ filter }}'
AND maxResults = '{{ maxResults }}'
AND pageToken = '{{ pageToken }}';

`INSERT` examples

insert
Manifest

Creates a new empty dataset.

INSERT INTO google.bigquery.datasets (
data__access,
data__datasetReference,
data__defaultCollation,
data__defaultEncryptionConfiguration,
data__defaultPartitionExpirationMs,
data__defaultRoundingMode,
data__defaultTableExpirationMs,
data__description,
data__externalCatalogDatasetOptions,
data__externalDatasetReference,
data__friendlyName,
data__isCaseInsensitive,
data__labels,
data__linkedDatasetSource,
data__location,
data__maxTimeTravelHours,
data__resourceTags,
data__storageBillingModel,
projectId,
accessPolicyVersion
)
SELECT 
'{{ access }}',
'{{ datasetReference }}',
'{{ defaultCollation }}',
'{{ defaultEncryptionConfiguration }}',
'{{ defaultPartitionExpirationMs }}',
'{{ defaultRoundingMode }}',
'{{ defaultTableExpirationMs }}',
'{{ description }}',
'{{ externalCatalogDatasetOptions }}',
'{{ externalDatasetReference }}',
'{{ friendlyName }}',
{{ isCaseInsensitive }},
'{{ labels }}',
'{{ linkedDatasetSource }}',
'{{ location }}',
'{{ maxTimeTravelHours }}',
'{{ resourceTags }}',
'{{ storageBillingModel }}',
'{{ projectId }}',
'{{ accessPolicyVersion }}'
RETURNING
id,
access,
creationTime,
datasetReference,
defaultCollation,
defaultEncryptionConfiguration,
defaultPartitionExpirationMs,
defaultRoundingMode,
defaultTableExpirationMs,
description,
etag,
externalCatalogDatasetOptions,
externalDatasetReference,
friendlyName,
isCaseInsensitive,
kind,
labels,
lastModifiedTime,
linkedDatasetMetadata,
linkedDatasetSource,
location,
maxTimeTravelHours,
resourceTags,
restrictions,
satisfiesPzi,
satisfiesPzs,
selfLink,
storageBillingModel,
tags,
type
;

# Description fields are for documentation purposes
- name: datasets
  props:
    - name: projectId
      value: string
      description: Required parameter for the datasets resource.
    - name: access
      value: array
      description: >
        Optional. An array of objects that define dataset access for one or more entities. You can set this property when inserting or updating a dataset in order to control who is allowed to access the data. If unspecified at dataset creation time, BigQuery adds default dataset access for the following entities: access.specialGroup: projectReaders; access.role: READER; access.specialGroup: projectWriters; access.role: WRITER; access.specialGroup: projectOwners; access.role: OWNER; access.userByEmail: [dataset creator email]; access.role: OWNER; If you patch a dataset, then this field is overwritten by the patched dataset's access field. To add entities, you must supply the entire existing access array in addition to any new entities that you want to add.
        
    - name: datasetReference
      value: object
      description: >
        Required. A reference that identifies the dataset.
        
    - name: defaultCollation
      value: string
      description: >
        Optional. Defines the default collation specification of future tables created in the dataset. If a table is created in this dataset without table-level default collation, then the table inherits the dataset default collation, which is applied to the string fields that do not have explicit collation specified. A change to this field affects only tables created afterwards, and does not alter the existing tables. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
        
    - name: defaultEncryptionConfiguration
      value: object
      description: >
        The default encryption key for all tables in the dataset. After this property is set, the encryption key of all newly-created tables in the dataset is set to this value unless the table creation request or query explicitly overrides the key.
        
    - name: defaultPartitionExpirationMs
      value: string
      description: >
        This default partition expiration, expressed in milliseconds. When new time-partitioned tables are created in a dataset where this property is set, the table will inherit this value, propagated as the `TimePartitioning.expirationMs` property on the new table. If you set `TimePartitioning.expirationMs` explicitly when creating a table, the `defaultPartitionExpirationMs` of the containing dataset is ignored. When creating a partitioned table, if `defaultPartitionExpirationMs` is set, the `defaultTableExpirationMs` value is ignored and the table will not be inherit a table expiration deadline.
        
    - name: defaultRoundingMode
      value: string
      description: >
        Optional. Defines the default rounding mode specification of new tables created within this dataset. During table creation, if this field is specified, the table within this dataset will inherit the default rounding mode of the dataset. Setting the default rounding mode on a table overrides this option. Existing tables in the dataset are unaffected. If columns are defined during that table creation, they will immediately inherit the table's default rounding mode, unless otherwise specified.
        
      valid_values: ['ROUNDING_MODE_UNSPECIFIED', 'ROUND_HALF_AWAY_FROM_ZERO', 'ROUND_HALF_EVEN']
    - name: defaultTableExpirationMs
      value: string
      description: >
        Optional. The default lifetime of all tables in the dataset, in milliseconds. The minimum lifetime value is 3600000 milliseconds (one hour). To clear an existing default expiration with a PATCH request, set to 0. Once this property is set, all newly-created tables in the dataset will have an expirationTime property set to the creation time plus the value in this property, and changing the value will only affect new tables, not existing ones. When the expirationTime for a given table is reached, that table will be deleted automatically. If a table's expirationTime is modified or removed before the table expires, or if you provide an explicit expirationTime when creating a table, that value takes precedence over the default expiration time indicated by this property.
        
    - name: description
      value: string
      description: >
        Optional. A user-friendly description of the dataset.
        
    - name: externalCatalogDatasetOptions
      value: object
      description: >
        Optional. Options defining open source compatible datasets living in the BigQuery catalog. Contains metadata of open source database, schema or namespace represented by the current dataset.
        
    - name: externalDatasetReference
      value: object
      description: >
        Optional. Reference to a read-only external dataset defined in data catalogs outside of BigQuery. Filled out when the dataset type is EXTERNAL.
        
    - name: friendlyName
      value: string
      description: >
        Optional. A descriptive name for the dataset.
        
    - name: isCaseInsensitive
      value: boolean
      description: >
        Optional. TRUE if the dataset and its table names are case-insensitive, otherwise FALSE. By default, this is FALSE, which means the dataset and its table names are case-sensitive. This field does not affect routine references.
        
    - name: labels
      value: object
      description: >
        The labels associated with this dataset. You can use these to organize and group your datasets. You can set this property when inserting or updating a dataset. See [Creating and Updating Dataset Labels](https://cloud.google.com/bigquery/docs/creating-managing-labels#creating_and_updating_dataset_labels) for more information.
        
    - name: linkedDatasetSource
      value: object
      description: >
        Optional. The source dataset reference when the dataset is of type LINKED. For all other dataset types it is not set. This field cannot be updated once it is set. Any attempt to update this field using Update and Patch API Operations will be ignored.
        
    - name: location
      value: string
      description: >
        The geographic location where the dataset should reside. See https://cloud.google.com/bigquery/docs/locations for supported locations.
        
    - name: maxTimeTravelHours
      value: string
      description: >
        Optional. Defines the time travel window in hours. The value can be from 48 to 168 hours (2 to 7 days). The default value is 168 hours if this is not set.
        
    - name: resourceTags
      value: object
      description: >
        Optional. The [tags](https://cloud.google.com/bigquery/docs/tags) attached to this dataset. Tag keys are globally unique. Tag key is expected to be in the namespaced format, for example "123456789012/environment" where 123456789012 is the ID of the parent organization or project resource for this tag key. Tag value is expected to be the short name, for example "Production". See [Tag definitions](https://cloud.google.com/iam/docs/tags-access-control#definitions) for more details.
        
    - name: storageBillingModel
      value: string
      description: >
        Optional. Updates storage_billing_model for the dataset.
        
      valid_values: ['STORAGE_BILLING_MODEL_UNSPECIFIED', 'LOGICAL', 'PHYSICAL']
    - name: accessPolicyVersion
      value: integer (int32)

`UPDATE` examples

patch

Updates information in an existing dataset. The update method replaces the entire dataset resource, whereas the patch method only replaces fields that are provided in the submitted dataset resource. This method supports RFC5789 patch semantics.

UPDATE google.bigquery.datasets
SET 
data__access = '{{ access }}',
data__datasetReference = '{{ datasetReference }}',
data__defaultCollation = '{{ defaultCollation }}',
data__defaultEncryptionConfiguration = '{{ defaultEncryptionConfiguration }}',
data__defaultPartitionExpirationMs = '{{ defaultPartitionExpirationMs }}',
data__defaultRoundingMode = '{{ defaultRoundingMode }}',
data__defaultTableExpirationMs = '{{ defaultTableExpirationMs }}',
data__description = '{{ description }}',
data__externalCatalogDatasetOptions = '{{ externalCatalogDatasetOptions }}',
data__externalDatasetReference = '{{ externalDatasetReference }}',
data__friendlyName = '{{ friendlyName }}',
data__isCaseInsensitive = {{ isCaseInsensitive }},
data__labels = '{{ labels }}',
data__linkedDatasetSource = '{{ linkedDatasetSource }}',
data__location = '{{ location }}',
data__maxTimeTravelHours = '{{ maxTimeTravelHours }}',
data__resourceTags = '{{ resourceTags }}',
data__storageBillingModel = '{{ storageBillingModel }}'
WHERE 
projectId = '{{ projectId }}' --required
AND +datasetId = '{{ +datasetId }}' --required
AND accessPolicyVersion = '{{ accessPolicyVersion}}'
AND updateMode = '{{ updateMode}}'
RETURNING
id,
access,
creationTime,
datasetReference,
defaultCollation,
defaultEncryptionConfiguration,
defaultPartitionExpirationMs,
defaultRoundingMode,
defaultTableExpirationMs,
description,
etag,
externalCatalogDatasetOptions,
externalDatasetReference,
friendlyName,
isCaseInsensitive,
kind,
labels,
lastModifiedTime,
linkedDatasetMetadata,
linkedDatasetSource,
location,
maxTimeTravelHours,
resourceTags,
restrictions,
satisfiesPzi,
satisfiesPzs,
selfLink,
storageBillingModel,
tags,
type;

`REPLACE` examples

update

Updates information in an existing dataset. The update method replaces the entire dataset resource, whereas the patch method only replaces fields that are provided in the submitted dataset resource.

REPLACE google.bigquery.datasets
SET 
data__access = '{{ access }}',
data__datasetReference = '{{ datasetReference }}',
data__defaultCollation = '{{ defaultCollation }}',
data__defaultEncryptionConfiguration = '{{ defaultEncryptionConfiguration }}',
data__defaultPartitionExpirationMs = '{{ defaultPartitionExpirationMs }}',
data__defaultRoundingMode = '{{ defaultRoundingMode }}',
data__defaultTableExpirationMs = '{{ defaultTableExpirationMs }}',
data__description = '{{ description }}',
data__externalCatalogDatasetOptions = '{{ externalCatalogDatasetOptions }}',
data__externalDatasetReference = '{{ externalDatasetReference }}',
data__friendlyName = '{{ friendlyName }}',
data__isCaseInsensitive = {{ isCaseInsensitive }},
data__labels = '{{ labels }}',
data__linkedDatasetSource = '{{ linkedDatasetSource }}',
data__location = '{{ location }}',
data__maxTimeTravelHours = '{{ maxTimeTravelHours }}',
data__resourceTags = '{{ resourceTags }}',
data__storageBillingModel = '{{ storageBillingModel }}'
WHERE 
projectId = '{{ projectId }}' --required
AND +datasetId = '{{ +datasetId }}' --required
AND accessPolicyVersion = '{{ accessPolicyVersion}}'
AND updateMode = '{{ updateMode}}'
RETURNING
id,
access,
creationTime,
datasetReference,
defaultCollation,
defaultEncryptionConfiguration,
defaultPartitionExpirationMs,
defaultRoundingMode,
defaultTableExpirationMs,
description,
etag,
externalCatalogDatasetOptions,
externalDatasetReference,
friendlyName,
isCaseInsensitive,
kind,
labels,
lastModifiedTime,
linkedDatasetMetadata,
linkedDatasetSource,
location,
maxTimeTravelHours,
resourceTags,
restrictions,
satisfiesPzi,
satisfiesPzs,
selfLink,
storageBillingModel,
tags,
type;

`DELETE` examples

delete

Deletes the dataset specified by the datasetId value. Before you can delete a dataset, you must delete all its tables, either manually or by specifying deleteContents. Immediately after deletion, you can create another dataset with the same name.

DELETE FROM google.bigquery.datasets
WHERE projectId = '{{ projectId }}' --required
AND +datasetId = '{{ +datasetId }}' --required
AND deleteContents = '{{ deleteContents }}';

Lifecycle Methods

undelete

Undeletes a dataset which is within time travel window based on datasetId. If a time is specified, the dataset version deleted at that time is undeleted, else the last live version is undeleted.

EXEC google.bigquery.datasets.undelete 
@projectId='{{ projectId }}' --required, 
@+datasetId='{{ +datasetId }}' --required 
@@json=
'{
"deletionTime": "{{ deletionTime }}"
}';

Overview​

Fields​

Methods​

Parameters​

SELECT examples​

INSERT examples​

UPDATE examples​

REPLACE examples​

DELETE examples​

Lifecycle Methods​