locations

Creates, updates, deletes, gets or lists a locations resource.

Overview

Name	locations
Type	Resource
Id	google.datalineage.locations

Fields

The following fields are returned by SELECT queries:

SELECT not supported for this resource, use SHOW METHODS to view available operations for the resource.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
batch_search_link_processes	exec	projectsId, locationsId		Retrieve information about LineageProcesses associated with specific links. LineageProcesses are transformation pipelines that result in data flowing from source to target assets. Links between assets represent this operation. If you have specific link names, you can use this method to verify which LineageProcesses contribute to creating those links. See the SearchLinks method for more information on how to retrieve link name. You can retrieve the LineageProcess information in every project where you have the datalineage.events.get permission. The project provided in the URL is used for Billing and Quota.
process_open_lineage_run_event	exec	projectsId, locationsId	requestId	Creates new lineage events together with their parents: process and run. Updates the process and run if they already exist. Mapped from Open Lineage specification: https://github.com/OpenLineage/OpenLineage/blob/main/spec/OpenLineage.json.
search_links	exec	projectsId, locationsId		Retrieve a list of links connected to a specific asset. Links represent the data flow between source (upstream) and target (downstream) assets in transformation pipelines. Links are stored in the same project as the Lineage Events that create them. You can retrieve links in every project where you have the datalineage.events.get permission. The project provided in the URL is used for Billing and Quota.
search_lineage_streaming	exec	projectsId, locationsId		Retrieves a streaming response of lineage links connected to the requested assets by performing a breadth-first search in the given direction. Links represent the data flow between source (upstream) and target (downstream) assets in transformation pipelines. Links are stored in the same project as the Lineage Events that create them. This method retrieves links from all valid locations provided in the request. This method supports Column-Level Lineage (CLL) along with wildcard support to retrieve all CLL for an Entity FQN. Following permissions are required to retrieve links: * datalineage.events.get permission for the project where the link is stored for entity-level lineage. * datalineage.events.getFields permission for the project where the link is stored for column-level lineage. This method also returns processes that created the links if explicitly requested by setting max_process_per_link is non-zero and full process details are requested via links.processes.process in the FieldMask. Permission required to retrieve processes: * datalineage.processes.get permission for the project where the process is stored.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
locationsId	string
projectsId	string
requestId	string

Lifecycle Methods

batch_search_link_processes

Retrieve information about LineageProcesses associated with specific links. LineageProcesses are transformation pipelines that result in data flowing from source to target assets. Links between assets represent this operation. If you have specific link names, you can use this method to verify which LineageProcesses contribute to creating those links. See the SearchLinks method for more information on how to retrieve link name. You can retrieve the LineageProcess information in every project where you have the datalineage.events.get permission. The project provided in the URL is used for Billing and Quota.

EXEC google.datalineage.locations.batch_search_link_processes 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required 
@@json=
'{
"pageSize": {{ pageSize }}, 
"pageToken": "{{ pageToken }}", 
"links": "{{ links }}"
}'
;

process_open_lineage_run_event

Creates new lineage events together with their parents: process and run. Updates the process and run if they already exist. Mapped from Open Lineage specification: https://github.com/OpenLineage/OpenLineage/blob/main/spec/OpenLineage.json.

EXEC google.datalineage.locations.process_open_lineage_run_event 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@requestId='{{ requestId }}'
;

search_links

Retrieve a list of links connected to a specific asset. Links represent the data flow between source (upstream) and target (downstream) assets in transformation pipelines. Links are stored in the same project as the Lineage Events that create them. You can retrieve links in every project where you have the datalineage.events.get permission. The project provided in the URL is used for Billing and Quota.

EXEC google.datalineage.locations.search_links 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required 
@@json=
'{
"targets": "{{ targets }}", 
"source": "{{ source }}", 
"target": "{{ target }}", 
"pageToken": "{{ pageToken }}", 
"sources": "{{ sources }}", 
"pageSize": {{ pageSize }}
}'
;

search_lineage_streaming

Retrieves a streaming response of lineage links connected to the requested assets by performing a breadth-first search in the given direction. Links represent the data flow between source (upstream) and target (downstream) assets in transformation pipelines. Links are stored in the same project as the Lineage Events that create them. This method retrieves links from all valid locations provided in the request. This method supports Column-Level Lineage (CLL) along with wildcard support to retrieve all CLL for an Entity FQN. Following permissions are required to retrieve links: * datalineage.events.get permission for the project where the link is stored for entity-level lineage. * datalineage.events.getFields permission for the project where the link is stored for column-level lineage. This method also returns processes that created the links if explicitly requested by setting max_process_per_link is non-zero and full process details are requested via links.processes.process in the FieldMask. Permission required to retrieve processes: * datalineage.processes.get permission for the project where the process is stored.

EXEC google.datalineage.locations.search_lineage_streaming 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required 
@@json=
'{
"rootCriteria": "{{ rootCriteria }}", 
"locations": "{{ locations }}", 
"direction": "{{ direction }}", 
"filters": "{{ filters }}", 
"limits": "{{ limits }}"
}'
;