evaluations

Creates, updates, deletes, gets or lists an evaluations resource.

Overview

Name	evaluations
Type	Resource
Id	google.aiplatform.evaluations

Fields

The following fields are returned by SELECT queries:

get

Successful response

Name	Datatype	Description
name	string	Output only. The resource name of the ModelEvaluation.
annotationSchemaUri	string	Points to a YAML file stored on Google Cloud Storage describing EvaluatedDataItemView.predictions, EvaluatedDataItemView.ground_truths, EvaluatedAnnotation.predictions, and EvaluatedAnnotation.ground_truths. The schema is defined as an OpenAPI 3.0.2 Schema Object. This field is not populated if there are neither EvaluatedDataItemViews nor EvaluatedAnnotations under this ModelEvaluation.
createTime	string (google-datetime)	Output only. Timestamp when this ModelEvaluation was created.
dataItemSchemaUri	string	Points to a YAML file stored on Google Cloud Storage describing EvaluatedDataItemView.data_item_payload and EvaluatedAnnotation.data_item_payload. The schema is defined as an OpenAPI 3.0.2 Schema Object. This field is not populated if there are neither EvaluatedDataItemViews nor EvaluatedAnnotations under this ModelEvaluation.
displayName	string	The display name of the ModelEvaluation.
explanationSpecs	array	Describes the values of ExplanationSpec that are used for explaining the predicted values on the evaluated data.
metadata	any	The metadata of the ModelEvaluation. For the ModelEvaluation uploaded from Managed Pipeline, metadata contains a structured value with keys of "pipeline_job_id", "evaluation_dataset_type", "evaluation_dataset_path", "row_based_metrics_path".
metrics	any	Evaluation metrics of the Model. The schema of the metrics is stored in metrics_schema_uri
metricsSchemaUri	string	Points to a YAML file stored on Google Cloud Storage describing the metrics of this ModelEvaluation. The schema is defined as an OpenAPI 3.0.2 Schema Object.
modelExplanation	object	Aggregated explanation metrics for the Model's prediction output over the data this ModelEvaluation uses. This field is populated only if the Model is evaluated with explanations, and only for AutoML tabular Models. (id: GoogleCloudAiplatformV1ModelExplanation)
sliceDimensions	array	All possible dimensions of ModelEvaluationSlices. The dimensions can be used as the filter of the ModelService.ListModelEvaluationSlices request, in the form of slice.dimension = .

list

Successful response

Name	Datatype	Description
name	string	Output only. The resource name of the ModelEvaluation.
annotationSchemaUri	string	Points to a YAML file stored on Google Cloud Storage describing EvaluatedDataItemView.predictions, EvaluatedDataItemView.ground_truths, EvaluatedAnnotation.predictions, and EvaluatedAnnotation.ground_truths. The schema is defined as an OpenAPI 3.0.2 Schema Object. This field is not populated if there are neither EvaluatedDataItemViews nor EvaluatedAnnotations under this ModelEvaluation.
createTime	string (google-datetime)	Output only. Timestamp when this ModelEvaluation was created.
dataItemSchemaUri	string	Points to a YAML file stored on Google Cloud Storage describing EvaluatedDataItemView.data_item_payload and EvaluatedAnnotation.data_item_payload. The schema is defined as an OpenAPI 3.0.2 Schema Object. This field is not populated if there are neither EvaluatedDataItemViews nor EvaluatedAnnotations under this ModelEvaluation.
displayName	string	The display name of the ModelEvaluation.
explanationSpecs	array	Describes the values of ExplanationSpec that are used for explaining the predicted values on the evaluated data.
metadata	any	The metadata of the ModelEvaluation. For the ModelEvaluation uploaded from Managed Pipeline, metadata contains a structured value with keys of "pipeline_job_id", "evaluation_dataset_type", "evaluation_dataset_path", "row_based_metrics_path".
metrics	any	Evaluation metrics of the Model. The schema of the metrics is stored in metrics_schema_uri
metricsSchemaUri	string	Points to a YAML file stored on Google Cloud Storage describing the metrics of this ModelEvaluation. The schema is defined as an OpenAPI 3.0.2 Schema Object.
modelExplanation	object	Aggregated explanation metrics for the Model's prediction output over the data this ModelEvaluation uses. This field is populated only if the Model is evaluated with explanations, and only for AutoML tabular Models. (id: GoogleCloudAiplatformV1ModelExplanation)
sliceDimensions	array	All possible dimensions of ModelEvaluationSlices. The dimensions can be used as the filter of the ModelService.ListModelEvaluationSlices request, in the form of slice.dimension = .

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
get	select	projectsId, locationsId, modelsId, evaluationsId		Gets a ModelEvaluation.
list	select	projectsId, locationsId, modelsId	filter, pageSize, pageToken, readMask	Lists ModelEvaluations in a Model.
import	exec	projectsId, locationsId, modelsId		Imports an externally generated ModelEvaluation.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
evaluationsId	string
locationsId	string
modelsId	string
projectsId	string
filter	string
pageSize	integer (int32)
pageToken	string
readMask	string (google-fieldmask)

SELECT examples

get

Gets a ModelEvaluation.

SELECT
name,
annotationSchemaUri,
createTime,
dataItemSchemaUri,
displayName,
explanationSpecs,
metadata,
metrics,
metricsSchemaUri,
modelExplanation,
sliceDimensions
FROM google.aiplatform.evaluations
WHERE projectsId = '{{ projectsId }}' -- required
AND locationsId = '{{ locationsId }}' -- required
AND modelsId = '{{ modelsId }}' -- required
AND evaluationsId = '{{ evaluationsId }}' -- required;

list

Lists ModelEvaluations in a Model.

SELECT
name,
annotationSchemaUri,
createTime,
dataItemSchemaUri,
displayName,
explanationSpecs,
metadata,
metrics,
metricsSchemaUri,
modelExplanation,
sliceDimensions
FROM google.aiplatform.evaluations
WHERE projectsId = '{{ projectsId }}' -- required
AND locationsId = '{{ locationsId }}' -- required
AND modelsId = '{{ modelsId }}' -- required
AND filter = '{{ filter }}'
AND pageSize = '{{ pageSize }}'
AND pageToken = '{{ pageToken }}'
AND readMask = '{{ readMask }}';

Lifecycle Methods

import

Imports an externally generated ModelEvaluation.

EXEC google.aiplatform.evaluations.import 
@projectsId='{{ projectsId }}' --required, 
@locationsId='{{ locationsId }}' --required, 
@modelsId='{{ modelsId }}' --required 
@@json=
'{
"modelEvaluation": "{{ modelEvaluation }}"
}';