-
Notifications
You must be signed in to change notification settings - Fork 1.7k
[AMBARI-22878] Update Service Group API to take list of mpack name associated with the service group #234
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[AMBARI-22878] Update Service Group API to take list of mpack name associated with the service group #234
Changes from 1 commit
dcb871b
e97eb29
da41ca0
158cd85
decbf54
68c1f04
a2340a6
04345f8
90f354f
a331735
3de0e93
f74af11
afc6a3b
fb3e005
7bcd6fe
cfe5295
d370602
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -17,16 +17,20 @@ | |
| */ | ||
| package org.apache.ambari.server.controller; | ||
|
|
||
| import java.util.HashSet; | ||
| import java.util.Objects; | ||
| import java.util.Set; | ||
|
|
||
| public class ServiceGroupRequest { | ||
|
|
||
| private String clusterName; // REF | ||
| private String serviceGroupName; // GET/CREATE/UPDATE/DELETE | ||
| private Set<String> mpackNames; // Associated mpack names | ||
|
|
||
| public ServiceGroupRequest(String clusterName, String serviceGroupName) { | ||
| this.clusterName = clusterName; | ||
| this.serviceGroupName = serviceGroupName; | ||
| mpackNames = new HashSet<>(); | ||
| } | ||
|
|
||
| /** | ||
|
|
@@ -57,9 +61,30 @@ public void setServiceGroupName(String serviceGroupName) { | |
| this.serviceGroupName = serviceGroupName; | ||
| } | ||
|
|
||
| /** | ||
| * @return a list of associated mpack names | ||
| */ | ||
| public Set<String> getMpackNames() { | ||
| return mpackNames; | ||
| } | ||
|
|
||
| /** | ||
| * @param mpackNames a list of associated mpack names | ||
| */ | ||
| public void setMpackNames(Set<String> mpackNames) { | ||
| this.mpackNames.addAll(mpackNames); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The method name says
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You are right. I copied and wrote,but forgot to update the name. |
||
| } | ||
|
|
||
| @Override | ||
| public String toString() { | ||
| return String.format("clusterName=%s, serviceGroupName=%s", clusterName, serviceGroupName); | ||
| StringBuilder sb = new StringBuilder(); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We've been gradually moving to ToStringBuilder here instead
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, we should switch this to ToStringBuilder. |
||
| sb.append("clusterName="+clusterName+",serviceGroupName="+serviceGroupName); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please avoid |
||
| if (!mpackNames.isEmpty()) { | ||
| sb.append(",mpackNames="); | ||
| mpackNames.stream().map(mpackName -> sb.append(mpackName + ",")); | ||
| sb.deleteCharAt(sb.length()-1); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think simply using built-in
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated |
||
| } | ||
| return sb.toString(); | ||
| } | ||
|
|
||
| @Override | ||
|
|
@@ -73,6 +98,8 @@ public boolean equals(Object obj) { | |
|
|
||
| ServiceGroupRequest other = (ServiceGroupRequest) obj; | ||
|
|
||
| // ignore mpackNames, even if they are different, we still consider sgrequests are the same | ||
|
|
||
| return Objects.equals(clusterName, other.clusterName) && | ||
| Objects.equals(serviceGroupName, other.serviceGroupName); | ||
| } | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -24,6 +24,7 @@ | |||||||||||
| import java.util.Iterator; | ||||||||||||
| import java.util.Map; | ||||||||||||
| import java.util.Set; | ||||||||||||
| import java.util.stream.Collectors; | ||||||||||||
|
|
||||||||||||
| import org.apache.ambari.server.AmbariException; | ||||||||||||
| import org.apache.ambari.server.ClusterNotFoundException; | ||||||||||||
|
|
@@ -78,6 +79,7 @@ public class ServiceGroupResourceProvider extends AbstractControllerResourceProv | |||||||||||
| public static final String SERVICE_GROUP_CLUSTER_NAME_PROPERTY_ID = RESPONSE_KEY + PropertyHelper.EXTERNAL_PATH_SEP + "cluster_name"; | ||||||||||||
| public static final String SERVICE_GROUP_SERVICE_GROUP_ID_PROPERTY_ID = RESPONSE_KEY + PropertyHelper.EXTERNAL_PATH_SEP + "service_group_id"; | ||||||||||||
| public static final String SERVICE_GROUP_SERVICE_GROUP_NAME_PROPERTY_ID = RESPONSE_KEY + PropertyHelper.EXTERNAL_PATH_SEP + "service_group_name"; | ||||||||||||
| public static final String SERVICE_GROUP_SERVICE_GROUP_MPACKNAME_PROPERTY_ID = RESPONSE_KEY + PropertyHelper.EXTERNAL_PATH_SEP + "mpacks"; | ||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we simply call it SERVICE_GROUP_MPACKNAME_PROPERTY_ID?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We need to add this property to -
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK, I will change it to SERVICE_GROUP_MPACKNAME_PROPERTY_ID |
||||||||||||
|
|
||||||||||||
|
|
||||||||||||
| private static Set<String> pkPropertyIds = | ||||||||||||
|
|
@@ -269,6 +271,11 @@ private ServiceGroupRequest getRequest(Map<String, Object> properties) { | |||||||||||
| String clusterName = (String) properties.get(SERVICE_GROUP_CLUSTER_NAME_PROPERTY_ID); | ||||||||||||
| String serviceGroupName = (String) properties.get(SERVICE_GROUP_SERVICE_GROUP_NAME_PROPERTY_ID); | ||||||||||||
| ServiceGroupRequest svcRequest = new ServiceGroupRequest(clusterName, serviceGroupName); | ||||||||||||
| Set<Object> mpackNames = (Set<Object>) properties.get(SERVICE_GROUP_SERVICE_GROUP_MPACKNAME_PROPERTY_ID); | ||||||||||||
| if (mpackNames != null) { | ||||||||||||
| Set<String> mpackNamesSet = mpackNames.stream().flatMap(mpack -> ((Map<String, String>)mpack).values().stream()).collect(Collectors.toSet()); | ||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you minimize the number of unchecked casts by avoiding the intermediate |
||||||||||||
| svcRequest.setMpackNames(mpackNamesSet); | ||||||||||||
| } | ||||||||||||
| return svcRequest; | ||||||||||||
| } | ||||||||||||
|
|
||||||||||||
|
|
@@ -292,6 +299,7 @@ public synchronized Set<ServiceGroupResponse> createServiceGroups(Set<ServiceGro | |||||||||||
|
|
||||||||||||
| // Already checked that service group does not exist | ||||||||||||
| ServiceGroup sg = cluster.addServiceGroup(request.getServiceGroupName()); | ||||||||||||
| sg.setServiceGroupMpackNames(request.getMpackNames()); | ||||||||||||
| createdSvcGrps.add(sg.convertToResponse()); | ||||||||||||
| } | ||||||||||||
| return createdSvcGrps; | ||||||||||||
|
|
@@ -422,6 +430,11 @@ private void validateCreateRequests(Set<ServiceGroupRequest> requests, Clusters | |||||||||||
| } | ||||||||||||
| serviceGroupNames.get(clusterName).add(serviceGroupName); | ||||||||||||
|
|
||||||||||||
| if (request.getMpackNames().size() != 1) { | ||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since this change introduces mpacks for service groups, and it also makes exactly one mpack a requirement, it breaks existing code that uses Can you please address those? I think most could be fixed by adding some dummy mpack name in Lines 39 to 43 in 00f29b9
|
||||||||||||
| String errmsg = "Invalid arguments, only one mpack is allowed in the service group " + serviceGroupName; | ||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please distinguish between 0 mpacks and 2+ mpacks. The error message "only one mpack is allowed" is a bit confusing when the request doesn't specify any mpacks for the service group.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I did not write the description of this bug clearly. At this time, temporarily we only support one mpack per servicegroup, but finally each servicegroup must have one or more mpacks when multiple mpacks support code is implemented. Again, I will add more clearer errmsg here.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. if (request.getMpackNames().size() < 1) |
||||||||||||
| throw new IllegalArgumentException(errmsg); | ||||||||||||
| } | ||||||||||||
|
|
||||||||||||
| Cluster cluster; | ||||||||||||
| try { | ||||||||||||
| cluster = clusters.getCluster(clusterName); | ||||||||||||
|
|
||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,9 +18,12 @@ | |
|
|
||
| package org.apache.ambari.server.orm.entities; | ||
|
|
||
| import java.util.HashSet; | ||
| import java.util.List; | ||
| import java.util.Set; | ||
|
|
||
| import javax.persistence.Column; | ||
| import javax.persistence.ElementCollection; | ||
| import javax.persistence.Entity; | ||
| import javax.persistence.GeneratedValue; | ||
| import javax.persistence.GenerationType; | ||
|
|
@@ -80,6 +83,10 @@ public class ServiceGroupEntity { | |
| @OneToMany(mappedBy="serviceGroupDependency") | ||
| private List<ServiceGroupDependencyEntity> dependencies; | ||
|
|
||
| @ElementCollection | ||
| @Column(name = "name") | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Don't we need a corresponding change in the DDL scripts? |
||
| private Set<String> mpackNames = new HashSet<>(); | ||
|
|
||
| public Long getClusterId() { | ||
| return clusterId; | ||
| } | ||
|
|
@@ -121,6 +128,14 @@ public void setServiceGroupDependencies(List<ServiceGroupDependencyEntity> servi | |
| this.serviceGroupDependencies = serviceGroupDependencies; | ||
| } | ||
|
|
||
| public Set<String> getMpackNames() { | ||
| return mpackNames; | ||
| } | ||
|
|
||
| public void setMpackNames(Set<String> mpackNames) { | ||
| this.mpackNames = mpackNames; | ||
| } | ||
|
|
||
| @Override | ||
| public boolean equals(Object o) { | ||
| if (this == o) return true; | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -46,6 +46,14 @@ public interface ServiceGroup { | |
|
|
||
| Set<ServiceGroupDependencyResponse> getServiceGroupDependencyResponses(); | ||
|
|
||
| Set<String> getServiceGroupMpackNames(); | ||
|
|
||
| void addServiceGroupMpackName(String mpackName); | ||
|
|
||
| void addServiceGroupMpackNames(Set<String> mpackNames); | ||
|
|
||
| void setServiceGroupMpackNames(Set<String> mpackNames); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you please omit
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Seems other methods have ServiceGroup, anyway, I will remove it. |
||
|
|
||
| void debugDump(StringBuilder sb); | ||
|
|
||
| void refresh(); | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -135,6 +135,34 @@ public long getClusterId() { | |
| return cluster.getClusterId(); | ||
| } | ||
|
|
||
| @Override | ||
| public Set<String> getServiceGroupMpackNames() { | ||
| ServiceGroupEntity entity = getServiceGroupEntity(); | ||
| return entity.getMpackNames(); | ||
| } | ||
|
|
||
| @Override | ||
| public void addServiceGroupMpackName(String mpackName){ | ||
| ServiceGroupEntity entity = getServiceGroupEntity(); | ||
| if (entity.getMpackNames().add(mpackName)) { | ||
| serviceGroupDAO.merge(entity); | ||
| } | ||
| } | ||
|
|
||
| @Override | ||
| public void addServiceGroupMpackNames(Set<String> mpackNames) { | ||
| ServiceGroupEntity entity = getServiceGroupEntity(); | ||
| entity.getMpackNames().addAll(mpackNames); | ||
| serviceGroupDAO.merge(entity); | ||
| } | ||
|
|
||
| @Override | ||
| public void setServiceGroupMpackNames(Set<String> mpackNames) { | ||
| ServiceGroupEntity entity = getServiceGroupEntity(); | ||
| entity.setMpackNames(mpackNames); | ||
| serviceGroupDAO.merge(entity); | ||
| } | ||
|
|
||
| @Override | ||
| public Set<ServiceGroupKey> getServiceGroupDependencies() { | ||
| return serviceGroupDependencies; | ||
|
|
@@ -149,6 +177,7 @@ public void setServiceGroupDependencies(Set<ServiceGroupKey> serviceGroupDepende | |
| public ServiceGroupResponse convertToResponse() { | ||
| ServiceGroupResponse r = new ServiceGroupResponse(cluster.getClusterId(), | ||
| cluster.getClusterName(), getServiceGroupId(), getServiceGroupName()); | ||
| r.setServiceGroupMpackNames(getServiceGroupMpackNames()); | ||
| return r; | ||
| } | ||
|
|
||
|
|
@@ -217,7 +246,14 @@ public Cluster getCluster() { | |
| @Override | ||
| public void debugDump(StringBuilder sb) { | ||
| sb.append("ServiceGroup={ serviceGroupName=" + getServiceGroupName() + ", clusterName=" | ||
| + cluster.getClusterName() + ", clusterId=" + cluster.getClusterId() + "}"); | ||
| + cluster.getClusterName() + ", clusterId=" + cluster.getClusterId() ); | ||
| Set<String> mpackNames = getServiceGroupMpackNames(); | ||
| if (!mpackNames.isEmpty()) { | ||
| sb.append(", mpackNames="); | ||
| mpackNames.stream().map(mpackName ->sb.append(mpackName + ",")); | ||
| sb.deleteCharAt(sb.length()-1); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same as above for similar logic in
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated |
||
| } | ||
| sb.append("}"); | ||
| } | ||
|
|
||
| /** | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think there's still a misunderstanding here - I thought it would be cleaner to separate this out into 2 fields so that it's clear to the user what is expected: mpack name and then mpack version. You can combine this and call it a stack ID internally, that's fine - but we should take 2 different properties.
Same goes for API requests where we're currently given
name-versionas a string. This should be broken out into 2 fields.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am ok for this change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jayush What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lets keep it as "version" for now for legacy purposes. When we upgrade to ambari 3.0 assuming the existing cluster is on HDP-2.6 stack, calling this as mpack name and mpack version would be misleading.
We should handle this in a separate JIRA if we really want to change this convention for all APIs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But "version" in legacy terms meant only the numeric value. If we don't want to break it out into 2 references, I think we'd rather use stackId, right?