lineage_events

Creates, updates, deletes, gets or lists a lineage_events resource.

Overview

Name	lineage_events
Type	Resource
Id	google.datalineage.lineage_events

Fields

The following fields are returned by SELECT queries:

get

Successful response

Name	Datatype	Description
name	string	Immutable. The resource name of the lineage event. Format: projects/{project}/locations/{location}/processes/{process}/runs/{run}/lineageEvents/{lineage_event}. Can be specified or auto-assigned. {lineage_event} must be not longer than 200 characters and only contain characters in a set: a-zA-Z0-9_-:.
endTime	string (google-datetime)	Optional. The end of the transformation which resulted in this lineage event. For streaming scenarios, it should be the end of the period from which the lineage is being reported.
links	array	Optional. List of source-target pairs. Can't contain more than 100 tuples.
startTime	string (google-datetime)	Required. The beginning of the transformation which resulted in this lineage event. For streaming scenarios, it should be the beginning of the period from which the lineage is being reported.

list

Successful response

Name	Datatype	Description
name	string	Immutable. The resource name of the lineage event. Format: projects/{project}/locations/{location}/processes/{process}/runs/{run}/lineageEvents/{lineage_event}. Can be specified or auto-assigned. {lineage_event} must be not longer than 200 characters and only contain characters in a set: a-zA-Z0-9_-:.
endTime	string (google-datetime)	Optional. The end of the transformation which resulted in this lineage event. For streaming scenarios, it should be the end of the period from which the lineage is being reported.
links	array	Optional. List of source-target pairs. Can't contain more than 100 tuples.
startTime	string (google-datetime)	Required. The beginning of the transformation which resulted in this lineage event. For streaming scenarios, it should be the beginning of the period from which the lineage is being reported.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
get	select	projectsId, locationsId, processesId, runsId, lineageEventsId		Gets details of a specified lineage event.
list	select	projectsId, locationsId, processesId, runsId	pageSize, pageToken	Lists lineage events in the given project and location. The list order is not defined.
create	insert	projectsId, locationsId, processesId, runsId	requestId	Creates a new lineage event.
delete	delete	projectsId, locationsId, processesId, runsId, lineageEventsId	allowMissing	Deletes the lineage event with the specified name.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
lineageEventsId	string
locationsId	string
processesId	string
projectsId	string
runsId	string
allowMissing	boolean
pageSize	integer (int32)
pageToken	string
requestId	string

SELECT examples

get

Gets details of a specified lineage event.

SELECT
name,
endTime,
links,
startTime
FROM google.datalineage.lineage_events
WHERE projectsId = '{{ projectsId }}' -- required
AND locationsId = '{{ locationsId }}' -- required
AND processesId = '{{ processesId }}' -- required
AND runsId = '{{ runsId }}' -- required
AND lineageEventsId = '{{ lineageEventsId }}' -- required;

list

Lists lineage events in the given project and location. The list order is not defined.

SELECT
name,
endTime,
links,
startTime
FROM google.datalineage.lineage_events
WHERE projectsId = '{{ projectsId }}' -- required
AND locationsId = '{{ locationsId }}' -- required
AND processesId = '{{ processesId }}' -- required
AND runsId = '{{ runsId }}' -- required
AND pageSize = '{{ pageSize }}'
AND pageToken = '{{ pageToken }}';

INSERT examples

create

Creates a new lineage event.

INSERT INTO google.datalineage.lineage_events (
data__name,
data__links,
data__startTime,
data__endTime,
projectsId,
locationsId,
processesId,
runsId,
requestId
)
SELECT 
'{{ name }}',
'{{ links }}',
'{{ startTime }}',
'{{ endTime }}',
'{{ projectsId }}',
'{{ locationsId }}',
'{{ processesId }}',
'{{ runsId }}',
'{{ requestId }}'
RETURNING
name,
endTime,
links,
startTime
;

Manifest

# Description fields are for documentation purposes
- name: lineage_events
  props:
    - name: projectsId
      value: string
      description: Required parameter for the lineage_events resource.
    - name: locationsId
      value: string
      description: Required parameter for the lineage_events resource.
    - name: processesId
      value: string
      description: Required parameter for the lineage_events resource.
    - name: runsId
      value: string
      description: Required parameter for the lineage_events resource.
    - name: name
      value: string
      description: >
        Immutable. The resource name of the lineage event. Format: `projects/{project}/locations/{location}/processes/{process}/runs/{run}/lineageEvents/{lineage_event}`. Can be specified or auto-assigned. {lineage_event} must be not longer than 200 characters and only contain characters in a set: `a-zA-Z0-9_-:.`
        
    - name: links
      value: array
      description: >
        Optional. List of source-target pairs. Can't contain more than 100 tuples.
        
    - name: startTime
      value: string
      description: >
        Required. The beginning of the transformation which resulted in this lineage event. For streaming scenarios, it should be the beginning of the period from which the lineage is being reported.
        
    - name: endTime
      value: string
      description: >
        Optional. The end of the transformation which resulted in this lineage event. For streaming scenarios, it should be the end of the period from which the lineage is being reported.
        
    - name: requestId
      value: string

DELETE examples

delete

Deletes the lineage event with the specified name.

DELETE FROM google.datalineage.lineage_events
WHERE projectsId = '{{ projectsId }}' --required
AND locationsId = '{{ locationsId }}' --required
AND processesId = '{{ processesId }}' --required
AND runsId = '{{ runsId }}' --required
AND lineageEventsId = '{{ lineageEventsId }}' --required
AND allowMissing = '{{ allowMissing }}';