endpoints

Creates, updates, deletes, gets or lists an endpoints resource.

Overview

Name	endpoints
Type	Resource
Id	google.aiplatform.endpoints

Fields

The following fields are returned by SELECT queries:

get

Name	Datatype	Description
name	string	Identifier. The resource name of the Endpoint.
clientConnectionConfig	object	Configurations that are applied to the endpoint for online prediction. (id: GoogleCloudAiplatformV1ClientConnectionConfig)
createTime	string (google-datetime)	Output only. Timestamp when this Endpoint was created.
dedicatedEndpointDns	string	Output only. DNS of the dedicated endpoint. Will only be populated if dedicated_endpoint_enabled is true. Depending on the features enabled, uid might be a random number or a string. For example, if fast_tryout is enabled, uid will be fasttryout. Format: https://{endpoint_id}.{region}-{uid}.prediction.vertexai.goog.
dedicatedEndpointEnabled	boolean	If true, the endpoint will be exposed through a dedicated DNS [Endpoint.dedicated_endpoint_dns]. Your request to the dedicated DNS will be isolated from other users' traffic and will have better performance and reliability. Note: Once you enabled dedicated endpoint, you won't be able to send request to the shared DNS {region}-aiplatform.googleapis.com. The limitation will be removed soon.
deployedModels	array	Output only. The models deployed in this Endpoint. To add or remove DeployedModels use EndpointService.DeployModel and EndpointService.UndeployModel respectively.
description	string	The description of the Endpoint.
displayName	string	Required. The display name of the Endpoint. The name can be up to 128 characters long and can consist of any UTF-8 characters.
enablePrivateServiceConnect	boolean	Deprecated: If true, expose the Endpoint via private service connect. Only one of the fields, network or enable_private_service_connect, can be set.
encryptionSpec	object	Customer-managed encryption key spec for an Endpoint. If set, this Endpoint and all sub-resources of this Endpoint will be secured by this key. (id: GoogleCloudAiplatformV1EncryptionSpec)
etag	string	Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.
gdcConfig	object	Configures the Google Distributed Cloud (GDC) environment for online prediction. Only set this field when the Endpoint is to be deployed in a GDC environment. (id: GoogleCloudAiplatformV1GdcConfig)
genAiAdvancedFeaturesConfig	object	Optional. Configuration for GenAiAdvancedFeatures. If the endpoint is serving GenAI models, advanced features like native RAG integration can be configured. Currently, only Model Garden models are supported. (id: GoogleCloudAiplatformV1GenAiAdvancedFeaturesConfig)
labels	object	The labels with user-defined metadata to organize your Endpoints. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.
modelDeploymentMonitoringJob	string	Output only. Resource name of the Model Monitoring job associated with this Endpoint if monitoring is enabled by JobService.CreateModelDeploymentMonitoringJob. Format: projects/{project}/locations/{location}/modelDeploymentMonitoringJobs/{model_deployment_monitoring_job}
network	string	Optional. The full name of the Google Compute Engine network to which the Endpoint should be peered. Private services access must already be configured for the network. If left unspecified, the Endpoint is not peered with any network. Only one of the fields, network or enable_private_service_connect, can be set. Format: projects/{project}/global/networks/{network}. Where {project} is a project number, as in 12345, and {network} is network name.
predictRequestResponseLoggingConfig	object	Configures the request-response logging for online prediction. (id: GoogleCloudAiplatformV1PredictRequestResponseLoggingConfig)
privateServiceConnectConfig	object	Optional. Configuration for private service connect. network and private_service_connect_config are mutually exclusive. (id: GoogleCloudAiplatformV1PrivateServiceConnectConfig)
satisfiesPzi	boolean	Output only. Reserved for future use.
satisfiesPzs	boolean	Output only. Reserved for future use.
trafficSplit	object	A map from a DeployedModel's ID to the percentage of this Endpoint's traffic that should be forwarded to that DeployedModel. If a DeployedModel's ID is not listed in this map, then it receives no traffic. The traffic percentage values must add up to 100, or map must be empty if the Endpoint is to not accept any traffic at a moment.
updateTime	string (google-datetime)	Output only. Timestamp when this Endpoint was last updated.

list

Name	Datatype	Description
name	string	Identifier. The resource name of the Endpoint.
clientConnectionConfig	object	Configurations that are applied to the endpoint for online prediction. (id: GoogleCloudAiplatformV1ClientConnectionConfig)
createTime	string (google-datetime)	Output only. Timestamp when this Endpoint was created.
dedicatedEndpointDns	string	Output only. DNS of the dedicated endpoint. Will only be populated if dedicated_endpoint_enabled is true. Depending on the features enabled, uid might be a random number or a string. For example, if fast_tryout is enabled, uid will be fasttryout. Format: https://{endpoint_id}.{region}-{uid}.prediction.vertexai.goog.
dedicatedEndpointEnabled	boolean	If true, the endpoint will be exposed through a dedicated DNS [Endpoint.dedicated_endpoint_dns]. Your request to the dedicated DNS will be isolated from other users' traffic and will have better performance and reliability. Note: Once you enabled dedicated endpoint, you won't be able to send request to the shared DNS {region}-aiplatform.googleapis.com. The limitation will be removed soon.
deployedModels	array	Output only. The models deployed in this Endpoint. To add or remove DeployedModels use EndpointService.DeployModel and EndpointService.UndeployModel respectively.
description	string	The description of the Endpoint.
displayName	string	Required. The display name of the Endpoint. The name can be up to 128 characters long and can consist of any UTF-8 characters.
enablePrivateServiceConnect	boolean	Deprecated: If true, expose the Endpoint via private service connect. Only one of the fields, network or enable_private_service_connect, can be set.
encryptionSpec	object	Customer-managed encryption key spec for an Endpoint. If set, this Endpoint and all sub-resources of this Endpoint will be secured by this key. (id: GoogleCloudAiplatformV1EncryptionSpec)
etag	string	Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.
gdcConfig	object	Configures the Google Distributed Cloud (GDC) environment for online prediction. Only set this field when the Endpoint is to be deployed in a GDC environment. (id: GoogleCloudAiplatformV1GdcConfig)
genAiAdvancedFeaturesConfig	object	Optional. Configuration for GenAiAdvancedFeatures. If the endpoint is serving GenAI models, advanced features like native RAG integration can be configured. Currently, only Model Garden models are supported. (id: GoogleCloudAiplatformV1GenAiAdvancedFeaturesConfig)
labels	object	The labels with user-defined metadata to organize your Endpoints. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.
modelDeploymentMonitoringJob	string	Output only. Resource name of the Model Monitoring job associated with this Endpoint if monitoring is enabled by JobService.CreateModelDeploymentMonitoringJob. Format: projects/{project}/locations/{location}/modelDeploymentMonitoringJobs/{model_deployment_monitoring_job}
network	string	Optional. The full name of the Google Compute Engine network to which the Endpoint should be peered. Private services access must already be configured for the network. If left unspecified, the Endpoint is not peered with any network. Only one of the fields, network or enable_private_service_connect, can be set. Format: projects/{project}/global/networks/{network}. Where {project} is a project number, as in 12345, and {network} is network name.
predictRequestResponseLoggingConfig	object	Configures the request-response logging for online prediction. (id: GoogleCloudAiplatformV1PredictRequestResponseLoggingConfig)
privateServiceConnectConfig	object	Optional. Configuration for private service connect. network and private_service_connect_config are mutually exclusive. (id: GoogleCloudAiplatformV1PrivateServiceConnectConfig)
satisfiesPzi	boolean	Output only. Reserved for future use.
satisfiesPzs	boolean	Output only. Reserved for future use.
trafficSplit	object	A map from a DeployedModel's ID to the percentage of this Endpoint's traffic that should be forwarded to that DeployedModel. If a DeployedModel's ID is not listed in this map, then it receives no traffic. The traffic percentage values must add up to 100, or map must be empty if the Endpoint is to not accept any traffic at a moment.
updateTime	string (google-datetime)	Output only. Timestamp when this Endpoint was last updated.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
get	select	projectsId, locationsId, endpointsId		Gets an Endpoint.
list	select	projectsId, locationsId	readMask, pageSize, gdcZone, filter, pageToken, orderBy	Lists Endpoints in a Location.
create	insert	projectsId, locationsId	endpointId	Creates an Endpoint.
update	update	projectsId, locationsId, endpointsId		Updates an Endpoint with a long running operation.
patch	update	projectsId, locationsId, endpointsId	updateMask	Updates an Endpoint.
delete	delete	projectsId, locationsId, endpointsId		Deletes an Endpoint.
fetch_predict_operation	exec	endpointsId		Fetch an asynchronous online prediction operation.
stream_generate_content	exec	projectsId, locationsId, endpointsId		Generate content with multimodal inputs with streaming support.
predict	exec	projectsId, locationsId, endpointsId
generate_content	exec	projectsId, locationsId, endpointsId		Generate content with multimodal inputs.
compute_tokens	exec	projectsId, locationsId, endpointsId		Return a list of tokens based on the input text.
count_tokens	exec	projectsId, locationsId, endpointsId		Perform a token counting.
predict_long_running	exec	projectsId, locationsId, endpointsId
deploy_model	exec	projectsId, locationsId, endpointsId		Deploys a Model into this Endpoint, creating a DeployedModel within it.
stream_raw_predict	exec	projectsId, locationsId, endpointsId		Perform a streaming online prediction with an arbitrary HTTP payload.
direct_raw_predict	exec	projectsId, locationsId, endpointsId		Perform an unary online prediction request to a gRPC model server for custom containers.
undeploy_model	exec	projectsId, locationsId, endpointsId		Undeploys a Model from an Endpoint, removing a DeployedModel from it, and freeing all resources it's using.
explain	exec	projectsId, locationsId, endpointsId		Perform an online explanation. If deployed_model_id is specified, the corresponding DeployModel must have explanation_spec populated. If deployed_model_id is not specified, all DeployedModels must have explanation_spec populated.
raw_predict	exec	projectsId, locationsId, endpointsId		Perform an online prediction with an arbitrary HTTP payload. The response includes the following HTTP headers: * X-Vertex-AI-Endpoint-Id: ID of the Endpoint that served this prediction. * X-Vertex-AI-Deployed-Model-Id: ID of the Endpoint's DeployedModel that served this prediction.
direct_predict	exec	projectsId, locationsId, endpointsId		Perform an unary online prediction request to a gRPC model server for Vertex first-party products and frameworks.
mutate_deployed_model	exec	projectsId, locationsId, endpointsId		Updates an existing deployed model. Updatable fields include min_replica_count, max_replica_count, required_replica_count, autoscaling_metric_specs, disable_container_logging (v1 only), and enable_container_logging (v1beta1 only).
server_streaming_predict	exec	projectsId, locationsId, endpointsId		Perform a server-side streaming online prediction request for Vertex LLM streaming.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
endpointsId	string
locationsId	string
projectsId	string
endpointId	string
filter	string
gdcZone	string
orderBy	string
pageSize	integer (int32)
pageToken	string
readMask	string (google-fieldmask)
updateMask	string (google-fieldmask)

SELECT examples

get

Gets an Endpoint.

SELECT
name,
clientConnectionConfig,
createTime,
dedicatedEndpointDns,
dedicatedEndpointEnabled,
deployedModels,
description,
displayName,
enablePrivateServiceConnect,
encryptionSpec,
etag,
gdcConfig,
genAiAdvancedFeaturesConfig,
labels,
modelDeploymentMonitoringJob,
network,
predictRequestResponseLoggingConfig,
privateServiceConnectConfig,
satisfiesPzi,
satisfiesPzs,
trafficSplit,
updateTime
FROM google.aiplatform.endpoints
WHERE projectsId = '{{ projectsId }}' -- required
AND locationsId = '{{ locationsId }}' -- required
AND endpointsId = '{{ endpointsId }}' -- required
;

list

Lists Endpoints in a Location.

SELECT
name,
clientConnectionConfig,
createTime,
dedicatedEndpointDns,
dedicatedEndpointEnabled,
deployedModels,
description,
displayName,
enablePrivateServiceConnect,
encryptionSpec,
etag,
gdcConfig,
genAiAdvancedFeaturesConfig,
labels,
modelDeploymentMonitoringJob,
network,
predictRequestResponseLoggingConfig,
privateServiceConnectConfig,
satisfiesPzi,
satisfiesPzs,
trafficSplit,
updateTime
FROM google.aiplatform.endpoints
WHERE projectsId = '{{ projectsId }}' -- required
AND locationsId = '{{ locationsId }}' -- required
AND readMask = '{{ readMask }}'
AND pageSize = '{{ pageSize }}'
AND gdcZone = '{{ gdcZone }}'
AND filter = '{{ filter }}'
AND pageToken = '{{ pageToken }}'
AND orderBy = '{{ orderBy }}'
;

INSERT examples

create

Creates an Endpoint.

INSERT INTO google.aiplatform.endpoints (
data__network,
data__enablePrivateServiceConnect,
data__privateServiceConnectConfig,
data__predictRequestResponseLoggingConfig,
data__labels,
data__encryptionSpec,
data__name,
data__clientConnectionConfig,
data__displayName,
data__etag,
data__dedicatedEndpointEnabled,
data__trafficSplit,
data__description,
data__genAiAdvancedFeaturesConfig,
data__gdcConfig,
projectsId,
locationsId,
endpointId
)
SELECT 
'{{ network }}',
{{ enablePrivateServiceConnect }},
'{{ privateServiceConnectConfig }}',
'{{ predictRequestResponseLoggingConfig }}',
'{{ labels }}',
'{{ encryptionSpec }}',
'{{ name }}',
'{{ clientConnectionConfig }}',
'{{ displayName }}',
'{{ etag }}',
{{ dedicatedEndpointEnabled }},
'{{ trafficSplit }}',
'{{ description }}',
'{{ genAiAdvancedFeaturesConfig }}',
'{{ gdcConfig }}',
'{{ projectsId }}',
'{{ locationsId }}',
'{{ endpointId }}'
RETURNING
name,
done,
error,
metadata,
response
;

Manifest

# Description fields are for documentation purposes
- name: endpoints
props:
  - name: projectsId
    value: "{{ projectsId }}"
    description: Required parameter for the endpoints resource.
  - name: locationsId
    value: "{{ locationsId }}"
    description: Required parameter for the endpoints resource.
  - name: network
    value: "{{ network }}"
    description: |
      Optional. The full name of the Google Compute Engine [network](https://cloud.google.com//compute/docs/networks-and-firewalls#networks) to which the Endpoint should be peered. Private services access must already be configured for the network. If left unspecified, the Endpoint is not peered with any network. Only one of the fields, network or enable_private_service_connect, can be set. [Format](https://cloud.google.com/compute/docs/reference/rest/v1/networks/insert): `projects/{project}/global/networks/{network}`. Where `{project}` is a project number, as in `12345`, and `{network}` is network name.
  - name: enablePrivateServiceConnect
    value: {{ enablePrivateServiceConnect }}
    description: |
      Deprecated: If true, expose the Endpoint via private service connect. Only one of the fields, network or enable_private_service_connect, can be set.
  - name: privateServiceConnectConfig
    description: |
      Optional. Configuration for private service connect. network and private_service_connect_config are mutually exclusive.
    value:
      enablePrivateServiceConnect: {{ enablePrivateServiceConnect }}
      projectAllowlist:
        - "{{ projectAllowlist }}"
      serviceAttachment: "{{ serviceAttachment }}"
      pscAutomationConfigs:
        - network: "{{ network }}"
          forwardingRule: "{{ forwardingRule }}"
          state: "{{ state }}"
          errorMessage: "{{ errorMessage }}"
          ipAddress: "{{ ipAddress }}"
          projectId: "{{ projectId }}"
  - name: predictRequestResponseLoggingConfig
    description: |
      Configures the request-response logging for online prediction.
    value:
      bigqueryDestination:
        outputUri: "{{ outputUri }}"
      samplingRate: {{ samplingRate }}
      enabled: {{ enabled }}
  - name: labels
    value: "{{ labels }}"
    description: |
      The labels with user-defined metadata to organize your Endpoints. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.
  - name: encryptionSpec
    description: |
      Customer-managed encryption key spec for an Endpoint. If set, this Endpoint and all sub-resources of this Endpoint will be secured by this key.
    value:
      kmsKeyName: "{{ kmsKeyName }}"
  - name: name
    value: "{{ name }}"
    description: |
      Identifier. The resource name of the Endpoint.
  - name: clientConnectionConfig
    description: |
      Configurations that are applied to the endpoint for online prediction.
    value:
      inferenceTimeout: "{{ inferenceTimeout }}"
  - name: displayName
    value: "{{ displayName }}"
    description: |
      Required. The display name of the Endpoint. The name can be up to 128 characters long and can consist of any UTF-8 characters.
  - name: etag
    value: "{{ etag }}"
    description: |
      Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.
  - name: dedicatedEndpointEnabled
    value: {{ dedicatedEndpointEnabled }}
    description: |
      If true, the endpoint will be exposed through a dedicated DNS [Endpoint.dedicated_endpoint_dns]. Your request to the dedicated DNS will be isolated from other users' traffic and will have better performance and reliability. Note: Once you enabled dedicated endpoint, you won't be able to send request to the shared DNS {region}-aiplatform.googleapis.com. The limitation will be removed soon.
  - name: trafficSplit
    value: "{{ trafficSplit }}"
    description: |
      A map from a DeployedModel's ID to the percentage of this Endpoint's traffic that should be forwarded to that DeployedModel. If a DeployedModel's ID is not listed in this map, then it receives no traffic. The traffic percentage values must add up to 100, or map must be empty if the Endpoint is to not accept any traffic at a moment.
  - name: description
    value: "{{ description }}"
    description: |
      The description of the Endpoint.
  - name: genAiAdvancedFeaturesConfig
    description: |
      Optional. Configuration for GenAiAdvancedFeatures. If the endpoint is serving GenAI models, advanced features like native RAG integration can be configured. Currently, only Model Garden models are supported.
    value:
      ragConfig:
        enableRag: {{ enableRag }}
  - name: gdcConfig
    description: |
      Configures the Google Distributed Cloud (GDC) environment for online prediction. Only set this field when the Endpoint is to be deployed in a GDC environment.
    value:
      zone: "{{ zone }}"
  - name: endpointId
    value: "{{ endpointId }}"

UPDATE examples

update

Updates an Endpoint with a long running operation.

UPDATE google.aiplatform.endpoints
SET 
data__endpoint = '{{ endpoint }}'
WHERE 
projectsId = '{{ projectsId }}' --required
AND locationsId = '{{ locationsId }}' --required
AND endpointsId = '{{ endpointsId }}' --required
RETURNING
name,
done,
error,
metadata,
response;

patch

Updates an Endpoint.

UPDATE google.aiplatform.endpoints
SET 
data__network = '{{ network }}',
data__enablePrivateServiceConnect = {{ enablePrivateServiceConnect }},
data__privateServiceConnectConfig = '{{ privateServiceConnectConfig }}',
data__predictRequestResponseLoggingConfig = '{{ predictRequestResponseLoggingConfig }}',
data__labels = '{{ labels }}',
data__encryptionSpec = '{{ encryptionSpec }}',
data__name = '{{ name }}',
data__clientConnectionConfig = '{{ clientConnectionConfig }}',
data__displayName = '{{ displayName }}',
data__etag = '{{ etag }}',
data__dedicatedEndpointEnabled = {{ dedicatedEndpointEnabled }},
data__trafficSplit = '{{ trafficSplit }}',
data__description = '{{ description }}',
data__genAiAdvancedFeaturesConfig = '{{ genAiAdvancedFeaturesConfig }}',
data__gdcConfig = '{{ gdcConfig }}'
WHERE 
projectsId = '{{ projectsId }}' --required
AND locationsId = '{{ locationsId }}' --required
AND endpointsId = '{{ endpointsId }}' --required
AND updateMask = '{{ updateMask}}'
RETURNING
name,
clientConnectionConfig,
createTime,
dedicatedEndpointDns,
dedicatedEndpointEnabled,
deployedModels,
description,
displayName,
enablePrivateServiceConnect,
encryptionSpec,
etag,
gdcConfig,
genAiAdvancedFeaturesConfig,
labels,
modelDeploymentMonitoringJob,
network,
predictRequestResponseLoggingConfig,
privateServiceConnectConfig,
satisfiesPzi,
satisfiesPzs,
trafficSplit,
updateTime;

DELETE examples

delete

Deletes an Endpoint.

DELETE FROM google.aiplatform.endpoints
WHERE projectsId = '{{ projectsId }}' --required
AND locationsId = '{{ locationsId }}' --required
AND endpointsId = '{{ endpointsId }}' --required
;

Lifecycle Methods

fetch_predict_operation

Fetch an asynchronous online prediction operation.

EXEC google.aiplatform.endpoints.fetch_predict_operation 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"operationName": "{{ operationName }}"
}'
;

stream_generate_content

Generate content with multimodal inputs with streaming support.

EXEC google.aiplatform.endpoints.stream_generate_content 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"systemInstruction": "{{ systemInstruction }}", 
"tools": "{{ tools }}", 
"generationConfig": "{{ generationConfig }}", 
"safetySettings": "{{ safetySettings }}", 
"contents": "{{ contents }}", 
"cachedContent": "{{ cachedContent }}", 
"labels": "{{ labels }}", 
"toolConfig": "{{ toolConfig }}", 
"modelArmorConfig": "{{ modelArmorConfig }}"
}'
;

predict

Successful response

EXEC google.aiplatform.endpoints.predict 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"parameters": "{{ parameters }}", 
"instances": "{{ instances }}", 
"labels": "{{ labels }}"
}'
;

generate_content

Generate content with multimodal inputs.

EXEC google.aiplatform.endpoints.generate_content 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"systemInstruction": "{{ systemInstruction }}", 
"tools": "{{ tools }}", 
"generationConfig": "{{ generationConfig }}", 
"safetySettings": "{{ safetySettings }}", 
"contents": "{{ contents }}", 
"cachedContent": "{{ cachedContent }}", 
"labels": "{{ labels }}", 
"toolConfig": "{{ toolConfig }}", 
"modelArmorConfig": "{{ modelArmorConfig }}"
}'
;

compute_tokens

Return a list of tokens based on the input text.

EXEC google.aiplatform.endpoints.compute_tokens 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"model": "{{ model }}", 
"contents": "{{ contents }}", 
"instances": "{{ instances }}"
}'
;

count_tokens

Perform a token counting.

EXEC google.aiplatform.endpoints.count_tokens 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"model": "{{ model }}", 
"contents": "{{ contents }}", 
"systemInstruction": "{{ systemInstruction }}", 
"tools": "{{ tools }}", 
"generationConfig": "{{ generationConfig }}", 
"instances": "{{ instances }}"
}'
;

predict_long_running

Successful response

EXEC google.aiplatform.endpoints.predict_long_running 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"labels": "{{ labels }}", 
"instances": "{{ instances }}", 
"parameters": "{{ parameters }}"
}'
;

deploy_model

Deploys a Model into this Endpoint, creating a DeployedModel within it.

EXEC google.aiplatform.endpoints.deploy_model 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"deployedModel": "{{ deployedModel }}", 
"trafficSplit": "{{ trafficSplit }}"
}'
;

stream_raw_predict

Perform a streaming online prediction with an arbitrary HTTP payload.

EXEC google.aiplatform.endpoints.stream_raw_predict 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"httpBody": "{{ httpBody }}"
}'
;

direct_raw_predict

Perform an unary online prediction request to a gRPC model server for custom containers.

EXEC google.aiplatform.endpoints.direct_raw_predict 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"methodName": "{{ methodName }}", 
"input": "{{ input }}"
}'
;

undeploy_model

Undeploys a Model from an Endpoint, removing a DeployedModel from it, and freeing all resources it's using.

EXEC google.aiplatform.endpoints.undeploy_model 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"trafficSplit": "{{ trafficSplit }}", 
"deployedModelId": "{{ deployedModelId }}"
}'
;

explain

Perform an online explanation. If deployed_model_id is specified, the corresponding DeployModel must have explanation_spec populated. If deployed_model_id is not specified, all DeployedModels must have explanation_spec populated.

EXEC google.aiplatform.endpoints.explain 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"instances": "{{ instances }}", 
"explanationSpecOverride": "{{ explanationSpecOverride }}", 
"deployedModelId": "{{ deployedModelId }}", 
"parameters": "{{ parameters }}"
}'
;

raw_predict

Perform an online prediction with an arbitrary HTTP payload. The response includes the following HTTP headers: * X-Vertex-AI-Endpoint-Id: ID of the Endpoint that served this prediction. * X-Vertex-AI-Deployed-Model-Id: ID of the Endpoint's DeployedModel that served this prediction.

EXEC google.aiplatform.endpoints.raw_predict 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"httpBody": "{{ httpBody }}"
}'
;

direct_predict

Perform an unary online prediction request to a gRPC model server for Vertex first-party products and frameworks.

EXEC google.aiplatform.endpoints.direct_predict 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"inputs": "{{ inputs }}", 
"parameters": "{{ parameters }}"
}'
;

mutate_deployed_model

Updates an existing deployed model. Updatable fields include min_replica_count, max_replica_count, required_replica_count, autoscaling_metric_specs, disable_container_logging (v1 only), and enable_container_logging (v1beta1 only).

EXEC google.aiplatform.endpoints.mutate_deployed_model 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"deployedModel": "{{ deployedModel }}", 
"updateMask": "{{ updateMask }}"
}'
;

server_streaming_predict

Perform a server-side streaming online prediction request for Vertex LLM streaming.

EXEC google.aiplatform.endpoints.server_streaming_predict 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@endpointsId='{{ endpointsId }}' --required 
@@json=
'{
"inputs": "{{ inputs }}", 
"parameters": "{{ parameters }}"
}'
;