datasets

Creates, updates, deletes, gets or lists a datasets resource.

Overview

Name	datasets
Type	Resource
Id	google.datalabeling.datasets

Fields

The following fields are returned by SELECT queries:

projects_datasets_get

Successful response

Name	Datatype	Description
name	string	Output only. Dataset resource name, format is: projects/{project_id}/datasets/{dataset_id}
blockingResources	array	Output only. The names of any related resources that are blocking changes to the dataset.
createTime	string (google-datetime)	Output only. Time the dataset is created.
dataItemCount	string (int64)	Output only. The number of data items in the dataset.
description	string	Optional. User-provided description of the annotation specification set. The description can be up to 10000 characters long.
displayName	string	Required. The display name of the dataset. Maximum of 64 characters.
inputConfigs	array	Output only. This is populated with the original input configs where ImportData is called. It is available only after the clients import data to this dataset.
lastMigrateTime	string (google-datetime)	Last time that the Dataset is migrated to AI Platform V2. If any of the AnnotatedDataset is migrated, the last_migration_time in Dataset is also updated.

projects_datasets_list

Successful response

Name	Datatype	Description
name	string	Output only. Dataset resource name, format is: projects/{project_id}/datasets/{dataset_id}
blockingResources	array	Output only. The names of any related resources that are blocking changes to the dataset.
createTime	string (google-datetime)	Output only. Time the dataset is created.
dataItemCount	string (int64)	Output only. The number of data items in the dataset.
description	string	Optional. User-provided description of the annotation specification set. The description can be up to 10000 characters long.
displayName	string	Required. The display name of the dataset. Maximum of 64 characters.
inputConfigs	array	Output only. This is populated with the original input configs where ImportData is called. It is available only after the clients import data to this dataset.
lastMigrateTime	string (google-datetime)	Last time that the Dataset is migrated to AI Platform V2. If any of the AnnotatedDataset is migrated, the last_migration_time in Dataset is also updated.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
projects_datasets_get	select	projectsId, datasetsId		Gets dataset by resource name.
projects_datasets_list	select	projectsId	filter, pageSize, pageToken	Lists datasets under a project. Pagination is supported.
projects_datasets_create	insert	projectsId		Creates dataset. If success return a Dataset resource.
projects_datasets_delete	delete	projectsId, datasetsId		Deletes a dataset by resource name.
projects_datasets_import_data	exec	projectsId, datasetsId		Imports data into dataset based on source locations defined in request. It can be called multiple times for the same dataset. Each dataset can only have one long running operation running on it. For example, no labeling task (also long running operation) can be started while importing is still ongoing. Vice versa.
projects_datasets_export_data	exec	projectsId, datasetsId		Exports data and annotations from dataset.
projects_datasets_image_label	exec	projectsId, datasetsId		Starts a labeling task for image. The type of image labeling task is configured by feature in the request.
projects_datasets_video_label	exec	projectsId, datasetsId		Starts a labeling task for video. The type of video labeling task is configured by feature in the request.
projects_datasets_text_label	exec	projectsId, datasetsId		Starts a labeling task for text. The type of text labeling task is configured by feature in the request.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
datasetsId	string
projectsId	string
filter	string
pageSize	integer (int32)
pageToken	string

SELECT examples

projects_datasets_get

Gets dataset by resource name.

SELECT
name,
blockingResources,
createTime,
dataItemCount,
description,
displayName,
inputConfigs,
lastMigrateTime
FROM google.datalabeling.datasets
WHERE projectsId = '{{ projectsId }}' -- required
AND datasetsId = '{{ datasetsId }}' -- required;

projects_datasets_list

Lists datasets under a project. Pagination is supported.

SELECT
name,
blockingResources,
createTime,
dataItemCount,
description,
displayName,
inputConfigs,
lastMigrateTime
FROM google.datalabeling.datasets
WHERE projectsId = '{{ projectsId }}' -- required
AND filter = '{{ filter }}'
AND pageSize = '{{ pageSize }}'
AND pageToken = '{{ pageToken }}';

INSERT examples

projects_datasets_create

Creates dataset. If success return a Dataset resource.

INSERT INTO google.datalabeling.datasets (
data__dataset,
projectsId
)
SELECT 
'{{ dataset }}',
'{{ projectsId }}'
RETURNING
name,
blockingResources,
createTime,
dataItemCount,
description,
displayName,
inputConfigs,
lastMigrateTime
;

Manifest

# Description fields are for documentation purposes
- name: datasets
  props:
    - name: projectsId
      value: string
      description: Required parameter for the datasets resource.
    - name: dataset
      value: object
      description: >
        Required. The dataset to be created.
        

DELETE examples

projects_datasets_delete

Deletes a dataset by resource name.

DELETE FROM google.datalabeling.datasets
WHERE projectsId = '{{ projectsId }}' --required
AND datasetsId = '{{ datasetsId }}' --required;

Lifecycle Methods

projects_datasets_import_data

Imports data into dataset based on source locations defined in request. It can be called multiple times for the same dataset. Each dataset can only have one long running operation running on it. For example, no labeling task (also long running operation) can be started while importing is still ongoing. Vice versa.

EXEC google.datalabeling.datasets.projects_datasets_import_data 
@projectsId='{{ projectsId }}' --required, 
@datasetsId='{{ datasetsId }}' --required 
@@json=
'{
"inputConfig": "{{ inputConfig }}", 
"userEmailAddress": "{{ userEmailAddress }}"
}';

projects_datasets_export_data

Exports data and annotations from dataset.

EXEC google.datalabeling.datasets.projects_datasets_export_data 
@projectsId='{{ projectsId }}' --required, 
@datasetsId='{{ datasetsId }}' --required 
@@json=
'{
"annotatedDataset": "{{ annotatedDataset }}", 
"filter": "{{ filter }}", 
"outputConfig": "{{ outputConfig }}", 
"userEmailAddress": "{{ userEmailAddress }}"
}';

projects_datasets_image_label

Starts a labeling task for image. The type of image labeling task is configured by feature in the request.

EXEC google.datalabeling.datasets.projects_datasets_image_label 
@projectsId='{{ projectsId }}' --required, 
@datasetsId='{{ datasetsId }}' --required 
@@json=
'{
"imageClassificationConfig": "{{ imageClassificationConfig }}", 
"boundingPolyConfig": "{{ boundingPolyConfig }}", 
"polylineConfig": "{{ polylineConfig }}", 
"segmentationConfig": "{{ segmentationConfig }}", 
"basicConfig": "{{ basicConfig }}", 
"feature": "{{ feature }}"
}';

projects_datasets_video_label

Starts a labeling task for video. The type of video labeling task is configured by feature in the request.

EXEC google.datalabeling.datasets.projects_datasets_video_label 
@projectsId='{{ projectsId }}' --required, 
@datasetsId='{{ datasetsId }}' --required 
@@json=
'{
"videoClassificationConfig": "{{ videoClassificationConfig }}", 
"objectDetectionConfig": "{{ objectDetectionConfig }}", 
"objectTrackingConfig": "{{ objectTrackingConfig }}", 
"eventConfig": "{{ eventConfig }}", 
"basicConfig": "{{ basicConfig }}", 
"feature": "{{ feature }}"
}';

projects_datasets_text_label

Starts a labeling task for text. The type of text labeling task is configured by feature in the request.

EXEC google.datalabeling.datasets.projects_datasets_text_label 
@projectsId='{{ projectsId }}' --required, 
@datasetsId='{{ datasetsId }}' --required 
@@json=
'{
"textClassificationConfig": "{{ textClassificationConfig }}", 
"textEntityExtractionConfig": "{{ textEntityExtractionConfig }}", 
"basicConfig": "{{ basicConfig }}", 
"feature": "{{ feature }}"
}';