[ML] adds new trained model alias API to simplify trained model updates and deployments

docs/reference/ml/df-analytics/apis/index.asciidoc

@@ -2,6 +2,7 @@ include::ml-df-analytics-apis.asciidoc[leveloffset=+1]
 //CREATE
 include::put-dfanalytics.asciidoc[leveloffset=+2]
 include::put-trained-models.asciidoc[leveloffset=+2]
+include::put-trained-models-aliases.asciidoc[leveloffset=+2]
 //UPDATE
 include::update-dfanalytics.asciidoc[leveloffset=+2]
 //DELETE

docs/reference/ml/df-analytics/apis/ml-df-analytics-apis.asciidoc

@@ -22,8 +22,9 @@ You can use the following APIs to perform {infer} operations.
 * <<get-trained-models>>
 * <<get-trained-models-stats>>
 * <<delete-trained-models>>
+* <<put-trained-models-aliases>>
 
-You can deploy a trained model to make predictions in an ingest pipeline or in 
+You can deploy a trained model to make predictions in an ingest pipeline or in
 an aggregation. Refer to the following documentation to learn more.
 
 * <<inference-processor,{infer-cap} processor>>

docs/reference/ml/df-analytics/apis/put-trained-models-aliases.asciidoc

@@ -0,0 +1,89 @@
+[role="xpack"]
+[testenv="platinum"]
+[[put-trained-models-aliases]]
+= Put Trained Models Aliases API
+[subs="attributes"]
+++++
+<titleabbrev>Put Trained Models Aliases</titleabbrev>
+++++
+
+Creates a trained models alias. These model aliases can be used instead of the trained model ID
+when referencing the model in the stack. Model aliases must be unique, and a trained model can have
+more than one model alias referring to it. But a model alias can only refer to a single trained model.
+
+beta::[]
+
+[[ml-put-trained-models-aliases-request]]
+== {api-request-title}
+
+`PUT _ml/trained_models/<model_id>/model_aliases/<model_alias>`
+
+
+[[ml-put-trained-models-aliases-prereq]]
+== {api-prereq-title}
+
+If the {es} {security-features} are enabled, you must have the following
+built-in roles and privileges:
+
+* `machine_learning_admin`
+
+For more information, see <<built-in-roles>>, <<security-privileges>>, and
+{ml-docs-setup-privileges}.
+
+[[ml-put-trained-models-aliases-desc]]
+== {api-description-title}
+
+This API creates a new model alias to refer to trained models, or updates an existing
+trained model's alias.
+
+When updating an existing model alias to a new model ID, this API will return a warning if the models
+are of different inference types. Example, if attempting to put the model alias
+`flights-delay-prediction` from a regression model to a classification model, the API will error.
+
+The API will return a warning if there are very few input fields in common between the old
+and new models for the model alias.
+
+[[ml-put-trained-models-aliases-path-params]]
+== {api-path-parms-title}
+
+`model_id`::
+(Required, string)
+The trained model ID to which the model alias should refer.
+
+`model_alias`::
+(Required, string)
+The model alias to create or update. The model_alias cannot end in numbers.
+
+[[ml-put-trained-models-aliases-query-params]]
+== {api-query-parms-title}
+
+`reassign`::
+(Optional, boolean)
+Should the `model_alias` get reassigned to the provided `model_id` if it is already
+assigned to a model. Defaults to false. The API will return an error if the `model_alias`
+is already assigned to a model but this parameter is `false`.
+
+[[ml-put-trained-models-aliases-example]]
+== {api-examples-title}
+
+[[ml-put-trained-models-aliases-example-new-alias]]
+=== Creating a new model alias
+
+The following example shows how to create a new model alias for a trained model ID.
+
+[source,console]
+--------------------------------------------------
+PUT _ml/trained_models/flight-delay-prediction-1574775339910/model_aliases/flight_delay_model
+--------------------------------------------------
+// TEST[skip:setup kibana sample data]
+
+[[ml-put-trained-models-aliases-example-put-alias]]
+=== Updating an existing model alias
+
+The following example shows how to reassign an existing model alias for a trained model ID.
+
+[source,console]
+--------------------------------------------------
+PUT _ml/trained_models/flight-delay-prediction-1580004349800/model_aliases/flight_delay_model?reassign=true
+--------------------------------------------------
+// TEST[skip:setup kibana sample data]

server/src/main/java/org/elasticsearch/common/util/set/Sets.java

@@ -62,6 +62,12 @@ public static <T> boolean haveEmptyIntersection(Set<T> left, Set<T> right) {
         return left.stream().noneMatch(right::contains);
     }
 
+    public static <T> boolean haveNonEmptyIntersection(Set<T> left, Set<T> right) {
+        Objects.requireNonNull(left);
+        Objects.requireNonNull(right);
+        return left.stream().anyMatch(right::contains);
+    }
+
     /**
      * The relative complement, or difference, of the specified left and right set. Namely, the resulting set contains all the elements that
      * are in the left set but not in the right set. Neither input is mutated by this operation, an entirely new set is returned.

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ml/action/GetTrainedModelsStatsAction.java

@@ -171,7 +171,7 @@ protected Reader<Response.TrainedModelStats> getReader() {
         public static class Builder {
 
             private long totalModelCount;
-            private Set<String> expandedIds;
+            private Map<String, Set<String>> expandedIdsWithAliases;
             private Map<String, IngestStats> ingestStatsMap;
             private Map<String, InferenceStats> inferenceStatsMap;
 
@@ -180,13 +180,13 @@ public Builder setTotalModelCount(long totalModelCount) {
                 return this;
             }
 
-            public Builder setExpandedIds(Set<String> expandedIds) {
-                this.expandedIds = expandedIds;
+            public Builder setExpandedIdsWithAliases(Map<String, Set<String>> expandedIdsWithAliases) {
+                this.expandedIdsWithAliases = expandedIdsWithAliases;
                 return this;
             }
 
-            public Set<String> getExpandedIds() {
-                return this.expandedIds;
+            public Map<String, Set<String>> getExpandedIdsWithAliases() {
+                return this.expandedIdsWithAliases;
             }
 
             public Builder setIngestStatsByModelId(Map<String, IngestStats> ingestStatsByModelId) {
@@ -200,8 +200,8 @@ public Builder setInferenceStatsByModelId(Map<String, InferenceStats> infereceSt
             }
 
             public Response build() {
-                List<TrainedModelStats> trainedModelStats = new ArrayList<>(expandedIds.size());
-                expandedIds.forEach(id -> {
+                List<TrainedModelStats> trainedModelStats = new ArrayList<>(expandedIdsWithAliases.size());
+                expandedIdsWithAliases.keySet().forEach(id -> {
                     IngestStats ingestStats = ingestStatsMap.get(id);
                     InferenceStats inferenceStats = inferenceStatsMap.get(id);
                     trainedModelStats.add(new TrainedModelStats(

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ml/action/InternalInferModelAction.java

@@ -143,18 +143,25 @@ public int hashCode() {
     public static class Response extends ActionResponse {
 
         private final List<InferenceResults> inferenceResults;
+        private final String modelId;
         private final boolean isLicensed;
 
-        public Response(List<InferenceResults> inferenceResults, boolean isLicensed) {
+        public Response(List<InferenceResults> inferenceResults, String modelId, boolean isLicensed) {
             super();
             this.inferenceResults = Collections.unmodifiableList(ExceptionsHelper.requireNonNull(inferenceResults, "inferenceResults"));
             this.isLicensed = isLicensed;
+            this.modelId = modelId;
         }
 
         public Response(StreamInput in) throws IOException {
             super(in);
             this.inferenceResults = Collections.unmodifiableList(in.readNamedWriteableList(InferenceResults.class));
             this.isLicensed = in.readBoolean();
+            if (in.getVersion().onOrAfter(Version.V_8_0_0)) {
+                this.modelId = in.readOptionalString();
+            } else {
+                this.modelId = null;
+            }
         }
 
         public List<InferenceResults> getInferenceResults() {
@@ -165,23 +172,32 @@ public boolean isLicensed() {
             return isLicensed;
         }
 
+        public String getModelId() {
+            return modelId;
+        }
+
         @Override
         public void writeTo(StreamOutput out) throws IOException {
             out.writeNamedWriteableList(inferenceResults);
             out.writeBoolean(isLicensed);
+            if (out.getVersion().onOrAfter(Version.V_8_0_0)) {
+                out.writeOptionalString(modelId);
+            }
         }
 
         @Override
         public boolean equals(Object o) {
             if (this == o) return true;
             if (o == null || getClass() != o.getClass()) return false;
             InternalInferModelAction.Response that = (InternalInferModelAction.Response) o;
-            return isLicensed == that.isLicensed && Objects.equals(inferenceResults, that.inferenceResults);
+            return isLicensed == that.isLicensed
+                && Objects.equals(inferenceResults, that.inferenceResults)
+                && Objects.equals(modelId, that.modelId);
         }
 
         @Override
         public int hashCode() {
-            return Objects.hash(inferenceResults, isLicensed);
+            return Objects.hash(inferenceResults, isLicensed, modelId);
         }
 
         public static Builder builder() {
@@ -190,6 +206,7 @@ public static Builder builder() {
 
         public static class Builder {
             private List<InferenceResults> inferenceResults;
+            private String modelId;
             private boolean isLicensed;
 
             public Builder setInferenceResults(List<InferenceResults> inferenceResults) {
@@ -202,8 +219,13 @@ public Builder setLicensed(boolean licensed) {
                 return this;
             }
 
+            public Builder setModelId(String modelId) {
+                this.modelId = modelId;
+                return this;
+            }
+
             public Response build() {
-                return new Response(inferenceResults, isLicensed);
+                return new Response(inferenceResults, modelId, isLicensed);
             }
         }
 

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ml/action/PutTrainedModelAliasAction.java

@@ -0,0 +1,119 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+package org.elasticsearch.xpack.core.ml.action;
+
+import org.elasticsearch.action.ActionRequestValidationException;
+import org.elasticsearch.action.ActionType;
+import org.elasticsearch.action.support.master.AcknowledgedRequest;
+import org.elasticsearch.action.support.master.AcknowledgedResponse;
+import org.elasticsearch.common.io.stream.StreamInput;
+import org.elasticsearch.common.io.stream.StreamOutput;
+import org.elasticsearch.xpack.core.ml.inference.TrainedModelConfig;
+import org.elasticsearch.xpack.core.ml.job.messages.Messages;
+import org.elasticsearch.xpack.core.ml.utils.ExceptionsHelper;
+
+import java.io.IOException;
+import java.util.Locale;
+import java.util.Objects;
+import java.util.regex.Pattern;
+
+import static org.elasticsearch.action.ValidateActions.addValidationError;
+import static org.elasticsearch.xpack.core.ml.job.messages.Messages.INVALID_MODEL_ALIAS;
+
+public class PutTrainedModelAliasAction extends ActionType<AcknowledgedResponse> {
+
+    // NOTE this is similar to our valid ID check. The difference here is that model_aliases cannot end in numbers
+    // This is to protect our automatic model naming conventions from hitting weird model_alias conflicts
+    private static final Pattern VALID_MODEL_ALIAS_CHAR_PATTERN = Pattern.compile("[a-z0-9](?:[a-z0-9_\\-\\.]*[a-z])?");
+
+    public static final PutTrainedModelAliasAction INSTANCE = new PutTrainedModelAliasAction();
+    public static final String NAME = "cluster:admin/xpack/ml/inference/model_aliases/put";
+
+    private PutTrainedModelAliasAction() {
+        super(NAME, AcknowledgedResponse::readFrom);
+    }
+
+    public static class Request extends AcknowledgedRequest<Request> {
+
+        public static final String MODEL_ALIAS = "model_alias";
+        public static final String REASSIGN = "reassign";
+
+        private final String modelAlias;
+        private final String modelId;
+        private final boolean reassign;
+
+        public Request(String modelAlias, String modelId, boolean reassign) {
+            this.modelAlias = ExceptionsHelper.requireNonNull(modelAlias, MODEL_ALIAS);
+            this.modelId = ExceptionsHelper.requireNonNull(modelId, TrainedModelConfig.MODEL_ID);
+            this.reassign = reassign;
+        }
+
+        public Request(StreamInput in) throws IOException {
+            super(in);
+            this.modelAlias = in.readString();
+            this.modelId = in.readString();
+            this.reassign = in.readBoolean();
+        }
+
+        public String getModelAlias() {
+            return modelAlias;
+        }
+
+        public String getModelId() {
+            return modelId;
+        }
+
+        public boolean isReassign() {
+            return reassign;
+        }
+
+        @Override
+        public void writeTo(StreamOutput out) throws  IOException {
+            super.writeTo(out);
+            out.writeString(modelAlias);
+            out.writeString(modelId);
+            out.writeBoolean(reassign);
+        }
+
+        @Override
+        public ActionRequestValidationException validate() {
+            ActionRequestValidationException validationException = null;
+            if (modelAlias.equals(modelId)) {
+                validationException = addValidationError(
+                    String.format(
+                        Locale.ROOT,
+                        "model_alias [%s] cannot equal model_id [%s]",
+                        modelAlias,
+                        modelId
+                    ),
+                    validationException
+                );
+            }
+            if (VALID_MODEL_ALIAS_CHAR_PATTERN.matcher(modelAlias).matches() == false) {
+                validationException = addValidationError(Messages.getMessage(INVALID_MODEL_ALIAS, modelAlias), validationException);
+            }
+            return validationException;
+        }
+
+        @Override
+        public boolean equals(Object o) {
+            if (this == o) return true;
+            if (o == null || getClass() != o.getClass()) return false;
+            Request request = (Request) o;
+            return Objects.equals(modelAlias, request.modelAlias)
+                && Objects.equals(modelId, request.modelId)
+                && Objects.equals(reassign, request.reassign);
+        }
+
+        @Override
+        public int hashCode() {
+            return Objects.hash(modelAlias, modelId, reassign);
+        }
+
+    }
+}