streams

Creates, updates, deletes, gets or lists a streams resource.

Overview

Name	streams
Type	Resource
Id	google.datastream.streams

Fields

The following fields are returned by SELECT queries:

get

Name	Datatype	Description
name	string	Output only. Identifier. The stream's name.
backfillAll	object	Automatically backfill objects included in the stream source configuration. Specific objects can be excluded. (id: BackfillAllStrategy)
backfillNone	object	Do not automatically backfill any objects. (id: BackfillNoneStrategy)
createTime	string (google-datetime)	Output only. The creation time of the stream.
customerManagedEncryptionKey	string	Immutable. A reference to a KMS encryption key. If provided, it will be used to encrypt the data. If left blank, data will be encrypted using an internal Stream-specific encryption key provisioned through KMS.
destinationConfig	object	Required. Destination connection profile configuration. (id: DestinationConfig)
displayName	string	Required. Display name.
errors	array	Output only. Errors on the Stream.
labels	object	Labels.
lastRecoveryTime	string (google-datetime)	Output only. If the stream was recovered, the time of the last recovery. Note: This field is currently experimental.
ruleSets	array	Optional. Rule sets to apply to the stream.
satisfiesPzi	boolean	Output only. Reserved for future use.
satisfiesPzs	boolean	Output only. Reserved for future use.
sourceConfig	object	Required. Source connection profile configuration. (id: SourceConfig)
state	string	The state of the stream.
updateTime	string (google-datetime)	Output only. The last update time of the stream.

list

Name	Datatype	Description
name	string	Output only. Identifier. The stream's name.
backfillAll	object	Automatically backfill objects included in the stream source configuration. Specific objects can be excluded. (id: BackfillAllStrategy)
backfillNone	object	Do not automatically backfill any objects. (id: BackfillNoneStrategy)
createTime	string (google-datetime)	Output only. The creation time of the stream.
customerManagedEncryptionKey	string	Immutable. A reference to a KMS encryption key. If provided, it will be used to encrypt the data. If left blank, data will be encrypted using an internal Stream-specific encryption key provisioned through KMS.
destinationConfig	object	Required. Destination connection profile configuration. (id: DestinationConfig)
displayName	string	Required. Display name.
errors	array	Output only. Errors on the Stream.
labels	object	Labels.
lastRecoveryTime	string (google-datetime)	Output only. If the stream was recovered, the time of the last recovery. Note: This field is currently experimental.
ruleSets	array	Optional. Rule sets to apply to the stream.
satisfiesPzi	boolean	Output only. Reserved for future use.
satisfiesPzs	boolean	Output only. Reserved for future use.
sourceConfig	object	Required. Source connection profile configuration. (id: SourceConfig)
state	string	The state of the stream.
updateTime	string (google-datetime)	Output only. The last update time of the stream.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
get	select	projectsId, locationsId, streamsId		Use this method to get details about a stream.
list	select	projectsId, locationsId	pageSize, pageToken, filter, orderBy	Use this method to list streams in a project and location.
create	insert	projectsId, locationsId	streamId, requestId, validateOnly, force	Use this method to create a stream.
patch	update	projectsId, locationsId, streamsId	updateMask, requestId, validateOnly, force	Use this method to update the configuration of a stream.
delete	delete	projectsId, locationsId, streamsId	requestId	Use this method to delete a stream.
run	exec	projectsId, locationsId, streamsId		Use this method to start, resume or recover a stream with a non default CDC strategy.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
locationsId	string
projectsId	string
streamsId	string
filter	string
force	boolean
orderBy	string
pageSize	integer (int32)
pageToken	string
requestId	string
streamId	string
updateMask	string (google-fieldmask)
validateOnly	boolean

SELECT examples

get

Use this method to get details about a stream.

SELECT
name,
backfillAll,
backfillNone,
createTime,
customerManagedEncryptionKey,
destinationConfig,
displayName,
errors,
labels,
lastRecoveryTime,
ruleSets,
satisfiesPzi,
satisfiesPzs,
sourceConfig,
state,
updateTime
FROM google.datastream.streams
WHERE projectsId = '{{ projectsId }}' -- required
AND locationsId = '{{ locationsId }}' -- required
AND streamsId = '{{ streamsId }}' -- required
;

list

Use this method to list streams in a project and location.

SELECT
name,
backfillAll,
backfillNone,
createTime,
customerManagedEncryptionKey,
destinationConfig,
displayName,
errors,
labels,
lastRecoveryTime,
ruleSets,
satisfiesPzi,
satisfiesPzs,
sourceConfig,
state,
updateTime
FROM google.datastream.streams
WHERE projectsId = '{{ projectsId }}' -- required
AND locationsId = '{{ locationsId }}' -- required
AND pageSize = '{{ pageSize }}'
AND pageToken = '{{ pageToken }}'
AND filter = '{{ filter }}'
AND orderBy = '{{ orderBy }}'
;

INSERT examples

create

Use this method to create a stream.

INSERT INTO google.datastream.streams (
data__labels,
data__displayName,
data__sourceConfig,
data__destinationConfig,
data__state,
data__backfillAll,
data__backfillNone,
data__customerManagedEncryptionKey,
data__ruleSets,
projectsId,
locationsId,
streamId,
requestId,
validateOnly,
force
)
SELECT 
'{{ labels }}',
'{{ displayName }}',
'{{ sourceConfig }}',
'{{ destinationConfig }}',
'{{ state }}',
'{{ backfillAll }}',
'{{ backfillNone }}',
'{{ customerManagedEncryptionKey }}',
'{{ ruleSets }}',
'{{ projectsId }}',
'{{ locationsId }}',
'{{ streamId }}',
'{{ requestId }}',
'{{ validateOnly }}',
'{{ force }}'
RETURNING
name,
done,
error,
metadata,
response
;

Manifest

# Description fields are for documentation purposes
- name: streams
  props:
    - name: projectsId
      value: string
      description: Required parameter for the streams resource.
    - name: locationsId
      value: string
      description: Required parameter for the streams resource.
    - name: labels
      value: object
      description: >
        Labels.
        
    - name: displayName
      value: string
      description: >
        Required. Display name.
        
    - name: sourceConfig
      value: object
      description: >
        Required. Source connection profile configuration.
        
    - name: destinationConfig
      value: object
      description: >
        Required. Destination connection profile configuration.
        
    - name: state
      value: string
      description: >
        The state of the stream.
        
      valid_values: ['STATE_UNSPECIFIED', 'NOT_STARTED', 'RUNNING', 'PAUSED', 'MAINTENANCE', 'FAILED', 'FAILED_PERMANENTLY', 'STARTING', 'DRAINING']
    - name: backfillAll
      value: object
      description: >
        Automatically backfill objects included in the stream source configuration. Specific objects can be excluded.
        
    - name: backfillNone
      value: object
      description: >
        Do not automatically backfill any objects.
        
    - name: customerManagedEncryptionKey
      value: string
      description: >
        Immutable. A reference to a KMS encryption key. If provided, it will be used to encrypt the data. If left blank, data will be encrypted using an internal Stream-specific encryption key provisioned through KMS.
        
    - name: ruleSets
      value: array
      description: >
        Optional. Rule sets to apply to the stream.
        
    - name: streamId
      value: string
    - name: requestId
      value: string
    - name: validateOnly
      value: boolean
    - name: force
      value: boolean

UPDATE examples

patch

Use this method to update the configuration of a stream.

UPDATE google.datastream.streams
SET 
data__labels = '{{ labels }}',
data__displayName = '{{ displayName }}',
data__sourceConfig = '{{ sourceConfig }}',
data__destinationConfig = '{{ destinationConfig }}',
data__state = '{{ state }}',
data__backfillAll = '{{ backfillAll }}',
data__backfillNone = '{{ backfillNone }}',
data__customerManagedEncryptionKey = '{{ customerManagedEncryptionKey }}',
data__ruleSets = '{{ ruleSets }}'
WHERE 
projectsId = '{{ projectsId }}' --required
AND locationsId = '{{ locationsId }}' --required
AND streamsId = '{{ streamsId }}' --required
AND updateMask = '{{ updateMask}}'
AND requestId = '{{ requestId}}'
AND validateOnly = {{ validateOnly}}
AND force = {{ force}}
RETURNING
name,
done,
error,
metadata,
response;

DELETE examples

delete

Use this method to delete a stream.

DELETE FROM google.datastream.streams
WHERE projectsId = '{{ projectsId }}' --required
AND locationsId = '{{ locationsId }}' --required
AND streamsId = '{{ streamsId }}' --required
AND requestId = '{{ requestId }}'
;

Lifecycle Methods

run

Use this method to start, resume or recover a stream with a non default CDC strategy.

EXEC google.datastream.streams.run 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@streamsId='{{ streamsId }}' --required 
@@json=
'{
"cdcStrategy": "{{ cdcStrategy }}", 
"force": {{ force }}
}'
;