-
Notifications
You must be signed in to change notification settings - Fork 1.7k
[AMBARI-22878] Update Service Group API to take list of mpack name associated with the service group #234
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[AMBARI-22878] Update Service Group API to take list of mpack name associated with the service group #234
Changes from 9 commits
dcb871b
e97eb29
da41ca0
158cd85
decbf54
68c1f04
a2340a6
04345f8
90f354f
a331735
3de0e93
f74af11
afc6a3b
fb3e005
7bcd6fe
cfe5295
d370602
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -17,16 +17,20 @@ | |
| */ | ||
| package org.apache.ambari.server.controller; | ||
|
|
||
| import java.util.HashSet; | ||
| import java.util.Objects; | ||
| import java.util.Set; | ||
|
|
||
| public class ServiceGroupRequest { | ||
|
|
||
| private String clusterName; // REF | ||
| private String serviceGroupName; // GET/CREATE/UPDATE/DELETE | ||
| private String stackId; // Associated stack version info | ||
|
|
||
| public ServiceGroupRequest(String clusterName, String serviceGroupName) { | ||
| public ServiceGroupRequest(String clusterName, String serviceGroupName, String stackId) { | ||
| this.clusterName = clusterName; | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think there's still a misunderstanding here - I thought it would be cleaner to separate this out into 2 fields so that it's clear to the user what is expected: mpack name and then mpack version. You can combine this and call it a stack ID internally, that's fine - but we should take 2 different properties. Same goes for API requests where we're currently given
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am ok for this change.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @jayush What do you think?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Lets keep it as "version" for now for legacy purposes. When we upgrade to ambari 3.0 assuming the existing cluster is on HDP-2.6 stack, calling this as mpack name and mpack version would be misleading. We should handle this in a separate JIRA if we really want to change this convention for all APIs.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. But "version" in legacy terms meant only the numeric value. If we don't want to break it out into 2 references, I think we'd rather use stackId, right? |
||
| this.serviceGroupName = serviceGroupName; | ||
| this.stackId = stackId; | ||
| } | ||
|
|
||
| /** | ||
|
|
@@ -57,9 +61,25 @@ public void setServiceGroupName(String serviceGroupName) { | |
| this.serviceGroupName = serviceGroupName; | ||
| } | ||
|
|
||
| /** | ||
| * @return the stackId | ||
| */ | ||
| public String getStackId() { | ||
| return stackId; | ||
| } | ||
|
|
||
| /** | ||
| * @param stackId the stackId to set | ||
| */ | ||
| public void setStackId(String stackId) { | ||
| this.stackId = stackId; | ||
| } | ||
|
|
||
| @Override | ||
| public String toString() { | ||
| return String.format("clusterName=%s, serviceGroupName=%s", clusterName, serviceGroupName); | ||
| StringBuilder sb = new StringBuilder(); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We've been gradually moving to ToStringBuilder here instead
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, we should switch this to ToStringBuilder. |
||
| sb.append("clusterName=").append(clusterName).append(", serviceGroupName=").append(serviceGroupName).append(", stackId=").append(stackId); | ||
| return sb.toString(); | ||
| } | ||
|
|
||
| @Override | ||
|
|
@@ -73,12 +93,11 @@ public boolean equals(Object obj) { | |
|
|
||
| ServiceGroupRequest other = (ServiceGroupRequest) obj; | ||
|
|
||
| return Objects.equals(clusterName, other.clusterName) && | ||
| Objects.equals(serviceGroupName, other.serviceGroupName); | ||
| return Objects.equals(clusterName, other.clusterName) && Objects.equals(serviceGroupName, other.serviceGroupName) && Objects.equals(stackId, other.stackId); | ||
| } | ||
|
|
||
| @Override | ||
| public int hashCode() { | ||
| return Objects.hash(clusterName, serviceGroupName); | ||
| return Objects.hash(clusterName, serviceGroupName, stackId); | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,11 +19,13 @@ | |
| package org.apache.ambari.server.controller.internal; | ||
|
|
||
| import java.util.Arrays; | ||
| import java.util.Collection; | ||
| import java.util.HashMap; | ||
| import java.util.HashSet; | ||
| import java.util.Iterator; | ||
| import java.util.Map; | ||
| import java.util.Set; | ||
| import java.util.stream.Collectors; | ||
|
|
||
| import org.apache.ambari.server.AmbariException; | ||
| import org.apache.ambari.server.ClusterNotFoundException; | ||
|
|
@@ -78,6 +80,7 @@ public class ServiceGroupResourceProvider extends AbstractControllerResourceProv | |
| public static final String SERVICE_GROUP_CLUSTER_NAME_PROPERTY_ID = RESPONSE_KEY + PropertyHelper.EXTERNAL_PATH_SEP + "cluster_name"; | ||
| public static final String SERVICE_GROUP_SERVICE_GROUP_ID_PROPERTY_ID = RESPONSE_KEY + PropertyHelper.EXTERNAL_PATH_SEP + "service_group_id"; | ||
| public static final String SERVICE_GROUP_SERVICE_GROUP_NAME_PROPERTY_ID = RESPONSE_KEY + PropertyHelper.EXTERNAL_PATH_SEP + "service_group_name"; | ||
| public static final String SERVICE_GROUP_STACK_ID_PROPERTY_ID = RESPONSE_KEY + PropertyHelper.EXTERNAL_PATH_SEP + "stack_id"; | ||
|
|
||
|
|
||
| private static Set<String> pkPropertyIds = | ||
|
|
@@ -103,10 +106,12 @@ public class ServiceGroupResourceProvider extends AbstractControllerResourceProv | |
| PROPERTY_IDS.add(SERVICE_GROUP_CLUSTER_NAME_PROPERTY_ID); | ||
| PROPERTY_IDS.add(SERVICE_GROUP_SERVICE_GROUP_ID_PROPERTY_ID); | ||
| PROPERTY_IDS.add(SERVICE_GROUP_SERVICE_GROUP_NAME_PROPERTY_ID); | ||
| PROPERTY_IDS.add(SERVICE_GROUP_STACK_ID_PROPERTY_ID); | ||
|
|
||
| // keys | ||
| KEY_PROPERTY_IDS.put(Resource.Type.Cluster, SERVICE_GROUP_CLUSTER_NAME_PROPERTY_ID); | ||
| KEY_PROPERTY_IDS.put(Resource.Type.ServiceGroup, SERVICE_GROUP_SERVICE_GROUP_NAME_PROPERTY_ID); | ||
| KEY_PROPERTY_IDS.put(Resource.Type.Stack, SERVICE_GROUP_STACK_ID_PROPERTY_ID); | ||
| } | ||
|
|
||
| private Clusters clusters; | ||
|
|
@@ -136,7 +141,7 @@ protected RequestStatus createResourcesAuthorized(Request request) | |
| throws SystemException, | ||
| UnsupportedPropertyException, | ||
| ResourceAlreadyExistsException, | ||
| NoSuchParentResourceException { | ||
| NoSuchParentResourceException, IllegalArgumentException{ | ||
|
|
||
| final Set<ServiceGroupRequest> requests = new HashSet<>(); | ||
| for (Map<String, Object> propertyMap : request.getProperties()) { | ||
|
|
@@ -145,7 +150,7 @@ protected RequestStatus createResourcesAuthorized(Request request) | |
| Set<ServiceGroupResponse> createServiceGroups = null; | ||
| createServiceGroups = createResources(new Command<Set<ServiceGroupResponse>>() { | ||
| @Override | ||
| public Set<ServiceGroupResponse> invoke() throws AmbariException, AuthorizationException { | ||
| public Set<ServiceGroupResponse> invoke() throws AmbariException, AuthorizationException, IllegalArgumentException { | ||
| return createServiceGroups(requests); | ||
| } | ||
| }); | ||
|
|
@@ -160,6 +165,7 @@ public Set<ServiceGroupResponse> invoke() throws AmbariException, AuthorizationE | |
| resource.setProperty(SERVICE_GROUP_CLUSTER_NAME_PROPERTY_ID, response.getClusterName()); | ||
| resource.setProperty(SERVICE_GROUP_SERVICE_GROUP_ID_PROPERTY_ID, response.getServiceGroupId()); | ||
| resource.setProperty(SERVICE_GROUP_SERVICE_GROUP_NAME_PROPERTY_ID, response.getServiceGroupName()); | ||
| resource.setProperty(SERVICE_GROUP_STACK_ID_PROPERTY_ID, response.getStackId()); | ||
|
|
||
| associatedResources.add(resource); | ||
| } | ||
|
|
@@ -199,6 +205,8 @@ public Set<ServiceGroupResponse> invoke() throws AmbariException { | |
| response.getServiceGroupId(), requestedIds); | ||
| setResourceProperty(resource, SERVICE_GROUP_SERVICE_GROUP_NAME_PROPERTY_ID, | ||
| response.getServiceGroupName(), requestedIds); | ||
| setResourceProperty(resource, SERVICE_GROUP_STACK_ID_PROPERTY_ID, | ||
| response.getStackId(), requestedIds); | ||
| resources.add(resource); | ||
| } | ||
| return resources; | ||
|
|
@@ -268,13 +276,14 @@ protected Set<String> getPKPropertyIds() { | |
| private ServiceGroupRequest getRequest(Map<String, Object> properties) { | ||
| String clusterName = (String) properties.get(SERVICE_GROUP_CLUSTER_NAME_PROPERTY_ID); | ||
| String serviceGroupName = (String) properties.get(SERVICE_GROUP_SERVICE_GROUP_NAME_PROPERTY_ID); | ||
| ServiceGroupRequest svcRequest = new ServiceGroupRequest(clusterName, serviceGroupName); | ||
| String stackId = (String) properties.get(SERVICE_GROUP_STACK_ID_PROPERTY_ID); | ||
| ServiceGroupRequest svcRequest = new ServiceGroupRequest(clusterName, serviceGroupName, stackId); | ||
| return svcRequest; | ||
| } | ||
|
|
||
| // Create services from the given request. | ||
| public synchronized Set<ServiceGroupResponse> createServiceGroups(Set<ServiceGroupRequest> requests) | ||
| throws AmbariException, AuthorizationException { | ||
| throws AmbariException, AuthorizationException, IllegalArgumentException { | ||
|
|
||
| if (requests.isEmpty()) { | ||
| LOG.warn("Received an empty requests set"); | ||
|
|
@@ -291,7 +300,7 @@ public synchronized Set<ServiceGroupResponse> createServiceGroups(Set<ServiceGro | |
| Cluster cluster = clusters.getCluster(request.getClusterName()); | ||
|
|
||
| // Already checked that service group does not exist | ||
| ServiceGroup sg = cluster.addServiceGroup(request.getServiceGroupName()); | ||
| ServiceGroup sg = cluster.addServiceGroup(request.getServiceGroupName(), request.getStackId()); | ||
| createdSvcGrps.add(sg.convertToResponse()); | ||
| } | ||
| return createdSvcGrps; | ||
|
|
@@ -389,21 +398,29 @@ ResourceType.CLUSTER, getClusterResourceId(serviceGroupRequest.getClusterName()) | |
|
|
||
|
|
||
| private void validateCreateRequests(Set<ServiceGroupRequest> requests, Clusters clusters) | ||
| throws AuthorizationException, AmbariException { | ||
| throws AuthorizationException, AmbariException, IllegalArgumentException { | ||
|
|
||
| AmbariMetaInfo ambariMetaInfo = getManagementController().getAmbariMetaInfo(); | ||
| Map<String, Set<String>> serviceGroupNames = new HashMap<>(); | ||
| Set<String> duplicates = new HashSet<>(); | ||
| for (ServiceGroupRequest request : requests) { | ||
| final String clusterName = request.getClusterName(); | ||
| final String serviceGroupName = request.getServiceGroupName(); | ||
| String stackVersion = request.getStackId(); | ||
| //TODO: ******* For test ********* this piece of code should be removed after web UI adds default stack id | ||
| if (StringUtils.isBlank(stackVersion)) { | ||
| stackVersion = "HDP-3.0.0.0"; | ||
| request.setStackId(stackVersion); | ||
| LOG.info("***** Add fake stack id in order to create cluster from web UI, this code will be removed after web UI adds default stack id"); | ||
| } | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is this fake code needed? If the web client is sending down any information about mpacks, then you should be able to use that, right? If no, then just get the first mpack registered for now.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Don't hardcode this. Take the value from cluster/version to make it work for single mpack install for now. |
||
|
|
||
| Validate.notNull(clusterName, "Cluster name should be provided when creating a service group"); | ||
| Validate.notEmpty(serviceGroupName, "Service group name should be provided when creating a service group"); | ||
| Validate.notEmpty(stackVersion, "Stack version should be provided when creating a service group"); | ||
|
|
||
| if (LOG.isDebugEnabled()) { | ||
| LOG.debug("Received a createServiceGroup request" + | ||
| ", clusterName=" + clusterName + ", serviceGroupName=" + serviceGroupName + ", request=" + request); | ||
| ", clusterName=" + clusterName + ", serviceGroupName=" + serviceGroupName + ", stackVersion=" + stackVersion + ", request=" + request); | ||
| } | ||
|
|
||
| if (!AuthorizationHelper.isAuthorized(ResourceType.CLUSTER, | ||
|
|
@@ -422,6 +439,8 @@ private void validateCreateRequests(Set<ServiceGroupRequest> requests, Clusters | |
| } | ||
| serviceGroupNames.get(clusterName).add(serviceGroupName); | ||
|
|
||
| // TODO: Validate stack version | ||
|
|
||
| Cluster cluster; | ||
| try { | ||
| cluster = clusters.getCluster(clusterName); | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,9 +18,13 @@ | |
|
|
||
| package org.apache.ambari.server.orm.entities; | ||
|
|
||
| import java.util.HashSet; | ||
| import java.util.List; | ||
| import java.util.Set; | ||
|
|
||
| import javax.persistence.CollectionTable; | ||
| import javax.persistence.Column; | ||
| import javax.persistence.ElementCollection; | ||
| import javax.persistence.Entity; | ||
| import javax.persistence.GeneratedValue; | ||
| import javax.persistence.GenerationType; | ||
|
|
@@ -70,6 +74,9 @@ public class ServiceGroupEntity { | |
| @Column(name = "service_group_name", nullable = false, insertable = true, updatable = true) | ||
| private String serviceGroupName; | ||
|
|
||
| @Column(name = "stack_id", nullable = false, insertable = true, updatable = true) | ||
| private String stackId; | ||
|
|
||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we need to make this an entity relationship instead since it's most likely that we'll need the entity data when dealing with service groups.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this stackId is the foreign key of stack table. We can get stack entity from this key.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That is true, but this means we'd need an extra step, like this: vs just I guess I'm OK leaving it, but why not change it now before consumers begin using it? |
||
| @ManyToOne | ||
| @JoinColumn(name = "cluster_id", referencedColumnName = "cluster_id", nullable = false) | ||
| private ClusterEntity clusterEntity; | ||
|
|
@@ -96,7 +103,6 @@ public void setServiceGroupId(Long serviceGroupId) { | |
| this.serviceGroupId = serviceGroupId; | ||
| } | ||
|
|
||
|
|
||
| public String getServiceGroupName() { | ||
| return serviceGroupName; | ||
| } | ||
|
|
@@ -105,6 +111,14 @@ public void setServiceGroupName(String serviceGroupName) { | |
| this.serviceGroupName = serviceGroupName; | ||
| } | ||
|
|
||
| public String getStackId() { | ||
| return stackId; | ||
| } | ||
|
|
||
| public void setStackId(String stackId) { | ||
| this.stackId = stackId; | ||
| } | ||
|
|
||
| public List<ServiceGroupDependencyEntity> getDependencies() { | ||
| return dependencies; | ||
| } | ||
|
|
@@ -131,6 +145,8 @@ public boolean equals(Object o) { | |
| if (clusterId != null ? !clusterId.equals(that.clusterId) : that.clusterId != null) return false; | ||
| if (serviceGroupName != null ? !serviceGroupName.equals(that.serviceGroupName) : that.serviceGroupName != null) | ||
| return false; | ||
| if (stackId != null ? !stackId.equals(that.stackId) : that.stackId != null) | ||
| return false; | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Convert to Objects.equals() or EqualsBuilder() |
||
|
|
||
| return true; | ||
| } | ||
|
|
@@ -139,6 +155,7 @@ public boolean equals(Object o) { | |
| public int hashCode() { | ||
| int result = clusterId != null ? clusterId.intValue() : 0; | ||
| result = 31 * result + (serviceGroupName != null ? serviceGroupName.hashCode() : 0); | ||
| result = 31 * result + (stackId != null ? stackId.hashCode() : 0); | ||
| return result; | ||
| } | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -95,7 +95,7 @@ Service addDependencyToService(String serviceGroupName, String serviceName, | |
| * @return | ||
| * @throws AmbariException | ||
| */ | ||
| ServiceGroup addServiceGroup(String serviceGroupName) throws AmbariException; | ||
| ServiceGroup addServiceGroup(String serviceGroupName, String stackId) throws AmbariException; | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Rename "String stackId" to "String version" in the interface.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The change is already there.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ok, changed it. |
||
|
|
||
| /** | ||
| * Add service group dependency to the service group | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -46,6 +46,10 @@ public interface ServiceGroup { | |
|
|
||
| Set<ServiceGroupDependencyResponse> getServiceGroupDependencyResponses(); | ||
|
|
||
| String getStackId(); | ||
|
|
||
| void setStackId(String stackId); | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Add documentation here. "StackID" can mean anything - it should be explicitly stated that a Stack ID represents a specific mpack version: MPackName-version So, for example:
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In ambari code, StackId is widely used as the combination of stackName-stackVersion, there has a class StackId is for this, so I do not think it is caused confusion.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why not use
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In Ambari < 3.0, the Stack ID was typically a 2-digit umbrella, like "HDP 2.6". It did not represent unique versions, like HDP 2.6.0.0-1234. We're changing this now so that Stack ID refers to the unique, fully-qualified version. I guess that's why it's a bit confusing to me. |
||
|
|
||
| void debugDump(StringBuilder sb); | ||
|
|
||
| void refresh(); | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -29,6 +29,7 @@ public interface ServiceGroupFactory { | |
|
|
||
| ServiceGroup createNew(Cluster cluster, | ||
| @Assisted("serviceGroupName") String serviceGroupName, | ||
| @Assisted("stackId") String stackId, | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. stackId should be the object StackId in this method
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. +1 for
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I will change to stackId instance |
||
| @Assisted("serviceGroupDependencies") Set<ServiceGroupKey> serviceGroupDependencies); | ||
|
|
||
| ServiceGroup createExisting(Cluster cluster, ServiceGroupEntity serviceGroupEntity); | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a bit confused by this. Why does a service group have a stack association? An mpack has the knowledge of which stack it belongs to, so the service group should have an mpack instance association.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jonathan-hurley
Our DB schemas are all coupled to stack-id. The only association we have between stack and mpack is in the stacks table. We will keep the association to stack-id to keep things consistent as well as to support creating service groups for old stacks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But multiple mpacks could have the same stack ID, right? How does this allow us to determine which mpack a service group is associated with?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think using the term "stackId" is a bit confusing here - it should be differentiated from the "stackId" foreign key value. In this case, it's actually the Mpack Name and specific MPack version (with build number), right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lets just call the string as "version" like we do current for cluster.
POST /api/v1/clusters
{
"Clusters" : {
"cluster_name": "cl1",
"version" : "HDP-2.5"
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will use version instead of stackId on rest api layer, and internally use stackId instance to represent stackName-stackVersion.
@jonathan-hurley you are right, the only association between stack and mpack is in stack table, so if servicegroup has stackName+stackVersion, it can identify stack entity and then can get Mapck entity.(name, version etc)