Skip to main content

datasets

Creates, updates, deletes, gets or lists a datasets resource.

Overview​

Namedatasets
TypeResource
Idgoogle.aiplatform.datasets

Fields​

The following fields are returned by SELECT queries:

Successful response

NameDatatypeDescription
namestringOutput only. Identifier. The resource name of the Dataset. Format: projects/{project}/locations/{location}/datasets/{dataset}
createTimestring (google-datetime)Output only. Timestamp when this Dataset was created.
dataItemCountstring (int64)Output only. The number of DataItems in this Dataset. Only apply for non-structured Dataset.
descriptionstringThe description of the Dataset.
displayNamestringRequired. The user-defined name of the Dataset. The name can be up to 128 characters long and can consist of any UTF-8 characters.
encryptionSpecobjectCustomer-managed encryption key spec for a Dataset. If set, this Dataset and all sub-resources of this Dataset will be secured by this key. (id: GoogleCloudAiplatformV1EncryptionSpec)
etagstringUsed to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.
labelsobjectThe labels with user-defined metadata to organize your Datasets. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Dataset (System labels are excluded). See https://goo.gl/xmQnxf for more information and examples of labels. System reserved label keys are prefixed with "aiplatform.googleapis.com/" and are immutable. Following system labels exist for each Dataset: * "aiplatform.googleapis.com/dataset_metadata_schema": output only, its value is the metadata_schema's title.
metadataanyRequired. Additional information about the Dataset.
metadataArtifactstringOutput only. The resource name of the Artifact that was created in MetadataStore when creating the Dataset. The Artifact resource name pattern is projects/{project}/locations/{location}/metadataStores/{metadata_store}/artifacts/{artifact}.
metadataSchemaUristringRequired. Points to a YAML file stored on Google Cloud Storage describing additional information about the Dataset. The schema is defined as an OpenAPI 3.0.2 Schema Object. The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/metadata/.
modelReferencestringOptional. Reference to the public base model last used by the dataset. Only set for prompt datasets.
satisfiesPzibooleanOutput only. Reserved for future use.
satisfiesPzsbooleanOutput only. Reserved for future use.
savedQueriesarrayAll SavedQueries belong to the Dataset will be returned in List/Get Dataset response. The annotation_specs field will not be populated except for UI cases which will only use annotation_spec_count. In CreateDataset request, a SavedQuery is created together if this field is set, up to one SavedQuery can be set in CreateDatasetRequest. The SavedQuery should not contain any AnnotationSpec.
updateTimestring (google-datetime)Output only. Timestamp when this Dataset was last updated.

Methods​

The following methods are available for this resource:

NameAccessible byRequired ParamsOptional ParamsDescription
getselectprojectsId, locationsId, datasetsIdreadMaskGets a Dataset.
listselectprojectsId, locationsIdfilter, pageSize, pageToken, readMask, orderByLists Datasets in a Location.
createinsertprojectsId, locationsIdCreates a Dataset.
patchupdateprojectsId, locationsId, datasetsIdupdateMaskUpdates a Dataset.
deletedeleteprojectsId, locationsId, datasetsIdDeletes a Dataset.
importexecprojectsId, locationsId, datasetsIdImports data into a Dataset.
exportexecprojectsId, locationsId, datasetsIdExports data from a Dataset.
search_data_itemsexecprojectsId, locationsId, datasetsIdorderByDataItem, orderByAnnotation.savedQuery, orderByAnnotation.orderBy, savedQuery, dataLabelingJob, dataItemFilter, annotationsFilter, annotationFilters, fieldMask, annotationsLimit, pageSize, orderBy, pageTokenSearches DataItems in a Dataset.

Parameters​

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

NameDatatypeDescription
datasetsIdstring
locationsIdstring
projectsIdstring
annotationFiltersstring
annotationsFilterstring
annotationsLimitinteger (int32)
dataItemFilterstring
dataLabelingJobstring
fieldMaskstring (google-fieldmask)
filterstring
orderBystring
orderByAnnotation.orderBystring
orderByAnnotation.savedQuerystring
orderByDataItemstring
pageSizeinteger (int32)
pageTokenstring
readMaskstring (google-fieldmask)
savedQuerystring
updateMaskstring (google-fieldmask)

SELECT examples​

Gets a Dataset.

SELECT
name,
createTime,
dataItemCount,
description,
displayName,
encryptionSpec,
etag,
labels,
metadata,
metadataArtifact,
metadataSchemaUri,
modelReference,
satisfiesPzi,
satisfiesPzs,
savedQueries,
updateTime
FROM google.aiplatform.datasets
WHERE projectsId = '{{ projectsId }}' -- required
AND locationsId = '{{ locationsId }}' -- required
AND datasetsId = '{{ datasetsId }}' -- required
AND readMask = '{{ readMask }}';

INSERT examples​

Creates a Dataset.

INSERT INTO google.aiplatform.datasets (
data__displayName,
data__description,
data__metadataSchemaUri,
data__metadata,
data__etag,
data__labels,
data__savedQueries,
data__encryptionSpec,
data__modelReference,
projectsId,
locationsId
)
SELECT 
'{{ displayName }}',
'{{ description }}',
'{{ metadataSchemaUri }}',
'{{ metadata }}',
'{{ etag }}',
'{{ labels }}',
'{{ savedQueries }}',
'{{ encryptionSpec }}',
'{{ modelReference }}',
'{{ projectsId }}',
'{{ locationsId }}'
RETURNING
name,
done,
error,
metadata,
response
;

UPDATE examples​

Updates a Dataset.

UPDATE google.aiplatform.datasets
SET 
data__displayName = '{{ displayName }}',
data__description = '{{ description }}',
data__metadataSchemaUri = '{{ metadataSchemaUri }}',
data__metadata = '{{ metadata }}',
data__etag = '{{ etag }}',
data__labels = '{{ labels }}',
data__savedQueries = '{{ savedQueries }}',
data__encryptionSpec = '{{ encryptionSpec }}',
data__modelReference = '{{ modelReference }}'
WHERE 
projectsId = '{{ projectsId }}' --required
AND locationsId = '{{ locationsId }}' --required
AND datasetsId = '{{ datasetsId }}' --required
AND updateMask = '{{ updateMask}}'
RETURNING
name,
createTime,
dataItemCount,
description,
displayName,
encryptionSpec,
etag,
labels,
metadata,
metadataArtifact,
metadataSchemaUri,
modelReference,
satisfiesPzi,
satisfiesPzs,
savedQueries,
updateTime;

DELETE examples​

Deletes a Dataset.

DELETE FROM google.aiplatform.datasets
WHERE projectsId = '{{ projectsId }}' --required
AND locationsId = '{{ locationsId }}' --required
AND datasetsId = '{{ datasetsId }}' --required;

Lifecycle Methods​

Imports data into a Dataset.

EXEC google.aiplatform.datasets.import 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@datasetsId='{{ datasetsId }}' --required 
@@json=
'{
"importConfigs": "{{ importConfigs }}"
}';