environments

Creates, updates, deletes, gets or lists an environments resource.

Overview

Name	environments
Type	Resource
Id	google.composer.environments

Fields

The following fields are returned by SELECT queries:

get

Name	Datatype	Description
name	string	Identifier. The resource name of the environment, in the form: "projects/{projectId}/locations/{locationId}/environments/{environmentId}" EnvironmentId must start with a lowercase letter followed by up to 63 lowercase letters, numbers, or hyphens, and cannot end with a hyphen.
config	object	Optional. Configuration parameters for this environment. (id: EnvironmentConfig)
createTime	string (google-datetime)	Output only. The time at which this environment was created.
labels	object	Optional. User-defined labels for this environment. The labels map can contain no more than 64 entries. Entries of the labels map are UTF8 strings that comply with the following restrictions: * Keys must conform to regexp: \p{Ll}\p{Lo}{0,62} * Values must conform to regexp: [\p{Ll}\p{Lo}\p{N}_-]{0,63} * Both keys and values are additionally constrained to be <= 128 bytes in size.
satisfiesPzi	boolean	Output only. Reserved for future use.
satisfiesPzs	boolean	Output only. Reserved for future use.
state	string	The current state of the environment. (STATE_UNSPECIFIED, CREATING, RUNNING, UPDATING, DELETING, ERROR)
storageConfig	object	Optional. Storage configuration for this environment. (id: StorageConfig)
updateTime	string (google-datetime)	Output only. The time at which this environment was last modified.
uuid	string	Output only. The UUID (Universally Unique IDentifier) associated with this environment. This value is generated when the environment is created.

list

Name	Datatype	Description
name	string	Identifier. The resource name of the environment, in the form: "projects/{projectId}/locations/{locationId}/environments/{environmentId}" EnvironmentId must start with a lowercase letter followed by up to 63 lowercase letters, numbers, or hyphens, and cannot end with a hyphen.
config	object	Optional. Configuration parameters for this environment. (id: EnvironmentConfig)
createTime	string (google-datetime)	Output only. The time at which this environment was created.
labels	object	Optional. User-defined labels for this environment. The labels map can contain no more than 64 entries. Entries of the labels map are UTF8 strings that comply with the following restrictions: * Keys must conform to regexp: \p{Ll}\p{Lo}{0,62} * Values must conform to regexp: [\p{Ll}\p{Lo}\p{N}_-]{0,63} * Both keys and values are additionally constrained to be <= 128 bytes in size.
satisfiesPzi	boolean	Output only. Reserved for future use.
satisfiesPzs	boolean	Output only. Reserved for future use.
state	string	The current state of the environment. (STATE_UNSPECIFIED, CREATING, RUNNING, UPDATING, DELETING, ERROR)
storageConfig	object	Optional. Storage configuration for this environment. (id: StorageConfig)
updateTime	string (google-datetime)	Output only. The time at which this environment was last modified.
uuid	string	Output only. The UUID (Universally Unique IDentifier) associated with this environment. This value is generated when the environment is created.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
get	select	projectsId, locationsId, environmentsId		Get an existing environment.
list	select	projectsId, locationsId	pageSize, pageToken	List environments.
create	insert	projectsId, locationsId		Create a new environment.
patch	update	projectsId, locationsId, environmentsId	updateMask	Update an environment.
delete	delete	projectsId, locationsId, environmentsId		Delete an environment.
save_snapshot	exec	projectsId, locationsId, environmentsId		Creates a snapshots of a Cloud Composer environment. As a result of this operation, snapshot of environment's state is stored in a location specified in the SaveSnapshotRequest.
restart_web_server	exec	projectsId, locationsId, environmentsId		Restart Airflow web server.
execute_airflow_command	exec	projectsId, locationsId, environmentsId		Executes Airflow CLI command.
poll_airflow_command	exec	projectsId, locationsId, environmentsId		Polls Airflow CLI command execution and fetches logs.
check_upgrade	exec	projectsId, locationsId, environmentsId		Check if an upgrade operation on the environment will succeed. In case of problems detailed info can be found in the returned Operation.
database_failover	exec	projectsId, locationsId, environmentsId		Triggers database failover (only for highly resilient environments).
stop_airflow_command	exec	projectsId, locationsId, environmentsId		Stops Airflow CLI command execution.
load_snapshot	exec	projectsId, locationsId, environmentsId		Loads a snapshot of a Cloud Composer environment. As a result of this operation, a snapshot of environment's specified in LoadSnapshotRequest is loaded into the environment.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
environmentsId	string
locationsId	string
projectsId	string
pageSize	integer (int32)
pageToken	string
updateMask	string (google-fieldmask)

SELECT examples

get

Get an existing environment.

SELECT
name,
config,
createTime,
labels,
satisfiesPzi,
satisfiesPzs,
state,
storageConfig,
updateTime,
uuid
FROM google.composer.environments
WHERE projectsId = '{{ projectsId }}' -- required
AND locationsId = '{{ locationsId }}' -- required
AND environmentsId = '{{ environmentsId }}' -- required
;

list

List environments.

SELECT
name,
config,
createTime,
labels,
satisfiesPzi,
satisfiesPzs,
state,
storageConfig,
updateTime,
uuid
FROM google.composer.environments
WHERE projectsId = '{{ projectsId }}' -- required
AND locationsId = '{{ locationsId }}' -- required
AND pageSize = '{{ pageSize }}'
AND pageToken = '{{ pageToken }}'
;

INSERT examples

create

Create a new environment.

INSERT INTO google.composer.environments (
data__config,
data__createTime,
data__labels,
data__state,
data__storageConfig,
data__name,
data__updateTime,
data__uuid,
projectsId,
locationsId
)
SELECT 
'{{ config }}',
'{{ createTime }}',
'{{ labels }}',
'{{ state }}',
'{{ storageConfig }}',
'{{ name }}',
'{{ updateTime }}',
'{{ uuid }}',
'{{ projectsId }}',
'{{ locationsId }}'
RETURNING
name,
done,
error,
metadata,
response
;

Manifest

# Description fields are for documentation purposes
- name: environments
props:
  - name: projectsId
    value: "{{ projectsId }}"
    description: Required parameter for the environments resource.
  - name: locationsId
    value: "{{ locationsId }}"
    description: Required parameter for the environments resource.
  - name: config
    description: |
      Optional. Configuration parameters for this environment.
    value:
      nodeConfig:
        ipAllocationPolicy:
          servicesIpv4CidrBlock: "{{ servicesIpv4CidrBlock }}"
          clusterSecondaryRangeName: "{{ clusterSecondaryRangeName }}"
          useIpAliases: {{ useIpAliases }}
          clusterIpv4CidrBlock: "{{ clusterIpv4CidrBlock }}"
          servicesSecondaryRangeName: "{{ servicesSecondaryRangeName }}"
        diskSizeGb: {{ diskSizeGb }}
        enableIpMasqAgent: {{ enableIpMasqAgent }}
        composerNetworkAttachment: "{{ composerNetworkAttachment }}"
        composerInternalIpv4CidrBlock: "{{ composerInternalIpv4CidrBlock }}"
        subnetwork: "{{ subnetwork }}"
        serviceAccount: "{{ serviceAccount }}"
        tags:
          - "{{ tags }}"
        network: "{{ network }}"
        oauthScopes:
          - "{{ oauthScopes }}"
        location: "{{ location }}"
        machineType: "{{ machineType }}"
      masterAuthorizedNetworksConfig:
        enabled: {{ enabled }}
        cidrBlocks:
          - displayName: "{{ displayName }}"
            cidrBlock: "{{ cidrBlock }}"
      airflowUri: "{{ airflowUri }}"
      gkeCluster: "{{ gkeCluster }}"
      nodeCount: {{ nodeCount }}
      encryptionConfig:
        kmsKeyName: "{{ kmsKeyName }}"
      dataRetentionConfig:
        taskLogsRetentionConfig:
          storageMode: "{{ storageMode }}"
        airflowMetadataRetentionConfig:
          retentionDays: {{ retentionDays }}
          retentionMode: "{{ retentionMode }}"
      resilienceMode: "{{ resilienceMode }}"
      privateEnvironmentConfig:
        webServerIpv4CidrBlock: "{{ webServerIpv4CidrBlock }}"
        cloudSqlIpv4CidrBlock: "{{ cloudSqlIpv4CidrBlock }}"
        enablePrivatelyUsedPublicIps: {{ enablePrivatelyUsedPublicIps }}
        enablePrivateBuildsOnly: {{ enablePrivateBuildsOnly }}
        cloudComposerNetworkIpv4CidrBlock: "{{ cloudComposerNetworkIpv4CidrBlock }}"
        privateClusterConfig:
          enablePrivateEndpoint: {{ enablePrivateEndpoint }}
          masterIpv4CidrBlock: "{{ masterIpv4CidrBlock }}"
          masterIpv4ReservedRange: "{{ masterIpv4ReservedRange }}"
        cloudComposerNetworkIpv4ReservedRange: "{{ cloudComposerNetworkIpv4ReservedRange }}"
        networkingConfig:
          connectionType: "{{ connectionType }}"
        networkingType: "{{ networkingType }}"
        webServerIpv4ReservedRange: "{{ webServerIpv4ReservedRange }}"
        enablePrivateEnvironment: {{ enablePrivateEnvironment }}
        cloudComposerConnectionSubnetwork: "{{ cloudComposerConnectionSubnetwork }}"
      softwareConfig:
        cloudDataLineageIntegration:
          enabled: {{ enabled }}
        auditLogsReplicationMode: "{{ auditLogsReplicationMode }}"
        pythonVersion: "{{ pythonVersion }}"
        schedulerCount: {{ schedulerCount }}
        imageVersion: "{{ imageVersion }}"
        pypiPackages: "{{ pypiPackages }}"
        webServerPluginsMode: "{{ webServerPluginsMode }}"
        airflowConfigOverrides: "{{ airflowConfigOverrides }}"
        envVariables: "{{ envVariables }}"
      recoveryConfig:
        scheduledSnapshotsConfig:
          snapshotLocation: "{{ snapshotLocation }}"
          enabled: {{ enabled }}
          snapshotCreationSchedule: "{{ snapshotCreationSchedule }}"
          timeZone: "{{ timeZone }}"
      databaseConfig:
        machineType: "{{ machineType }}"
        zone: "{{ zone }}"
      workloadsConfig:
        webServer:
          cpu: {{ cpu }}
          memoryGb: {{ memoryGb }}
          storageGb: {{ storageGb }}
        scheduler:
          memoryGb: {{ memoryGb }}
          cpu: {{ cpu }}
          count: {{ count }}
          storageGb: {{ storageGb }}
        dagProcessor:
          storageGb: {{ storageGb }}
          cpu: {{ cpu }}
          count: {{ count }}
          memoryGb: {{ memoryGb }}
        worker:
          memoryGb: {{ memoryGb }}
          maxCount: {{ maxCount }}
          storageGb: {{ storageGb }}
          minCount: {{ minCount }}
          cpu: {{ cpu }}
        triggerer:
          memoryGb: {{ memoryGb }}
          count: {{ count }}
          cpu: {{ cpu }}
      environmentSize: "{{ environmentSize }}"
      webServerNetworkAccessControl:
        allowedIpRanges:
          - value: "{{ value }}"
            description: "{{ description }}"
      maintenanceWindow:
        endTime: "{{ endTime }}"
        startTime: "{{ startTime }}"
        recurrence: "{{ recurrence }}"
      webServerConfig:
        machineType: "{{ machineType }}"
      airflowByoidUri: "{{ airflowByoidUri }}"
      dagGcsPrefix: "{{ dagGcsPrefix }}"
  - name: createTime
    value: "{{ createTime }}"
    description: |
      Output only. The time at which this environment was created.
  - name: labels
    value: "{{ labels }}"
    description: |
      Optional. User-defined labels for this environment. The labels map can contain no more than 64 entries. Entries of the labels map are UTF8 strings that comply with the following restrictions: * Keys must conform to regexp: p{Ll}p{Lo}{0,62} * Values must conform to regexp: [p{Ll}p{Lo}p{N}_-]{0,63} * Both keys and values are additionally constrained to be <= 128 bytes in size.
  - name: state
    value: "{{ state }}"
    description: |
      The current state of the environment.
    valid_values: ['STATE_UNSPECIFIED', 'CREATING', 'RUNNING', 'UPDATING', 'DELETING', 'ERROR']
  - name: storageConfig
    description: |
      Optional. Storage configuration for this environment.
    value:
      bucket: "{{ bucket }}"
  - name: name
    value: "{{ name }}"
    description: |
      Identifier. The resource name of the environment, in the form: "projects/{projectId}/locations/{locationId}/environments/{environmentId}" EnvironmentId must start with a lowercase letter followed by up to 63 lowercase letters, numbers, or hyphens, and cannot end with a hyphen.
  - name: updateTime
    value: "{{ updateTime }}"
    description: |
      Output only. The time at which this environment was last modified.
  - name: uuid
    value: "{{ uuid }}"
    description: |
      Output only. The UUID (Universally Unique IDentifier) associated with this environment. This value is generated when the environment is created.

UPDATE examples

patch

Update an environment.

UPDATE google.composer.environments
SET 
data__config = '{{ config }}',
data__createTime = '{{ createTime }}',
data__labels = '{{ labels }}',
data__state = '{{ state }}',
data__storageConfig = '{{ storageConfig }}',
data__name = '{{ name }}',
data__updateTime = '{{ updateTime }}',
data__uuid = '{{ uuid }}'
WHERE 
projectsId = '{{ projectsId }}' --required
AND locationsId = '{{ locationsId }}' --required
AND environmentsId = '{{ environmentsId }}' --required
AND updateMask = '{{ updateMask}}'
RETURNING
name,
done,
error,
metadata,
response;

DELETE examples

delete

Delete an environment.

DELETE FROM google.composer.environments
WHERE projectsId = '{{ projectsId }}' --required
AND locationsId = '{{ locationsId }}' --required
AND environmentsId = '{{ environmentsId }}' --required
;

Lifecycle Methods

save_snapshot

Creates a snapshots of a Cloud Composer environment. As a result of this operation, snapshot of environment's state is stored in a location specified in the SaveSnapshotRequest.

EXEC google.composer.environments.save_snapshot 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@environmentsId='{{ environmentsId }}' --required 
@@json=
'{
"snapshotLocation": "{{ snapshotLocation }}"
}'
;

restart_web_server

Restart Airflow web server.

EXEC google.composer.environments.restart_web_server 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@environmentsId='{{ environmentsId }}' --required
;

execute_airflow_command

Executes Airflow CLI command.

EXEC google.composer.environments.execute_airflow_command 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@environmentsId='{{ environmentsId }}' --required 
@@json=
'{
"parameters": "{{ parameters }}", 
"subcommand": "{{ subcommand }}", 
"command": "{{ command }}"
}'
;

poll_airflow_command

Polls Airflow CLI command execution and fetches logs.

EXEC google.composer.environments.poll_airflow_command 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@environmentsId='{{ environmentsId }}' --required 
@@json=
'{
"nextLineNumber": {{ nextLineNumber }}, 
"podNamespace": "{{ podNamespace }}", 
"executionId": "{{ executionId }}", 
"pod": "{{ pod }}"
}'
;

check_upgrade

Check if an upgrade operation on the environment will succeed. In case of problems detailed info can be found in the returned Operation.

EXEC google.composer.environments.check_upgrade 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@environmentsId='{{ environmentsId }}' --required 
@@json=
'{
"imageVersion": "{{ imageVersion }}"
}'
;

database_failover

Triggers database failover (only for highly resilient environments).

EXEC google.composer.environments.database_failover 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@environmentsId='{{ environmentsId }}' --required
;

stop_airflow_command

Stops Airflow CLI command execution.

EXEC google.composer.environments.stop_airflow_command 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@environmentsId='{{ environmentsId }}' --required 
@@json=
'{
"executionId": "{{ executionId }}", 
"pod": "{{ pod }}", 
"force": {{ force }}, 
"podNamespace": "{{ podNamespace }}"
}'
;

load_snapshot

Loads a snapshot of a Cloud Composer environment. As a result of this operation, a snapshot of environment's specified in LoadSnapshotRequest is loaded into the environment.

EXEC google.composer.environments.load_snapshot 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@environmentsId='{{ environmentsId }}' --required 
@@json=
'{
"snapshotPath": "{{ snapshotPath }}", 
"skipPypiPackagesInstallation": {{ skipPypiPackagesInstallation }}, 
"skipGcsDataCopying": {{ skipGcsDataCopying }}, 
"skipAirflowOverridesSetting": {{ skipAirflowOverridesSetting }}, 
"skipEnvironmentVariablesSetting": {{ skipEnvironmentVariablesSetting }}
}'
;