Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add: Support Custom Sequence as a Backend of an API #12524

Merged
merged 56 commits into from
Sep 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
5c4b8b9
Init Commit
BLasan Aug 1, 2024
c5c9149
Add: New REST API to upload Custom Sequence
BLasan Aug 12, 2024
b55dabb
Add: Custom Sequence support for the APIS
BLasan Aug 15, 2024
74443fb
Fix: Analytics Type conversion issue
BLasan Aug 16, 2024
65628a4
Revamp: Endpoint Config update for API import via APICTL and REST
BLasan Aug 17, 2024
e7f8f2f
Fix: Sequence generating Issue
BLasan Aug 25, 2024
ab62a27
Changed: REST API of Custom Backend update
BLasan Sep 3, 2024
64fb6e5
Add: Custom Backend to APICTL file created on revision deployment
BLasan Sep 4, 2024
f16e756
Remove: Unnecessary Codes
BLasan Sep 5, 2024
011fdf0
Remove: Unwanted Codes in SynapseArtifactGenerator
BLasan Sep 5, 2024
d3355fb
Remove: APIDTO Sequence update
BLasan Sep 5, 2024
081b0cf
Remove: APIDTO Sequence update unwanted comments
BLasan Sep 5, 2024
9e18a89
Remove: APIDTO unwanted imports and spaces
BLasan Sep 5, 2024
fb45ec7
Remove: Unwnated Imports APIMgtDao
BLasan Sep 5, 2024
8d58008
Fix: Revision create, delete, revision deploy, revision undeploy, rev…
BLasan Sep 6, 2024
6bf0333
Fix: APICTL Import scenario
BLasan Sep 7, 2024
76fb15d
Update: Custom Backend REST APIs
BLasan Sep 9, 2024
e99d50a
Fix: Check Style Issues
BLasan Sep 9, 2024
485ba08
Remove: Unused Imports
BLasan Sep 9, 2024
5f3f42e
Fix: Ununsed imports and checkstyle issues
BLasan Sep 9, 2024
a98b09d
Fix: CheckStyle issue in If clause
BLasan Sep 9, 2024
10cebc6
Fix: APIProduct Issue
BLasan Sep 9, 2024
471a6f4
Fix: APIProduct deployment error
BLasan Sep 9, 2024
769bcab
Fix: Custom Backend Get Issue
BLasan Sep 9, 2024
e4f8d8a
Add: Custom Backend Delete
BLasan Sep 11, 2024
a310af7
Fix: API Product Endpoint adding issue
BLasan Sep 12, 2024
6b8e22d
Update: REST APIs of Custom Backend
BLasan Sep 12, 2024
4af356e
Fix: REST API Issues
BLasan Sep 14, 2024
5b57348
Fix: Checkstyle Issues
BLasan Sep 14, 2024
e1317ec
Fix: Import Order Issue
BLasan Sep 14, 2024
4209a86
Add: Endpoint Configurations data
BLasan Sep 14, 2024
aa58c51
Fix: Bugs in Revision scenarios
BLasan Sep 16, 2024
1798fa5
Add: API Validation for Sequence Backend
BLasan Sep 17, 2024
63f10f6
Add: NPE handling
BLasan Sep 17, 2024
df82c20
Fix: Sequence Backend Download Issue
BLasan Sep 17, 2024
63ae2cf
Fix: Advertised only API Issue
BLasan Sep 17, 2024
2edd31e
Add: Validations for APICTL Export scenario
BLasan Sep 18, 2024
720909f
Rename: Custom Backend dir to Sequence Backend
BLasan Sep 19, 2024
ffac3a3
Rename: Custom Backend dir to Sequence Backend
BLasan Sep 19, 2024
a5c791b
Add: Schema for all database types
BLasan Sep 19, 2024
3784395
Add: APICTL Project export Endpoint Configuration
BLasan Sep 19, 2024
87f2e12
Fix: Sequence Backend storing issue in APICTL project
BLasan Sep 20, 2024
1620f6c
Add: Composite Primary Key
BLasan Sep 20, 2024
b758283
Fix: JSON Parsing error with Number format exception
BLasan Sep 20, 2024
c47b485
Fix: Null Pointer Exception for non endpoint configs
BLasan Sep 20, 2024
052e53a
Hadneld: Number Format Exceptions when parsing
BLasan Sep 21, 2024
8bd45c3
Revert: SESSION TIMEOUT CONFIG in Publisher CommonUtil
BLasan Sep 21, 2024
58e7b20
Remove: Unused Import
BLasan Sep 21, 2024
be1cb81
Fix: Commented Issues
BLasan Sep 23, 2024
d1bb088
Fix: Input Stream Closing Issue
BLasan Sep 23, 2024
15c6b2f
Fix: TemplateUtil Test error
BLasan Sep 23, 2024
d7415a8
Handle: Transaction Rollbacks on failure
BLasan Sep 26, 2024
67d5b98
Fix: Type in debug log
BLasan Sep 26, 2024
54ae42a
Fix: Lint Issues
BLasan Sep 26, 2024
43047d7
Add: Sequence Backend schema for MultiDC
BLasan Sep 26, 2024
d12fca0
Fix: Merge Conflicts
BLasan Sep 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
import org.wso2.carbon.apimgt.api.model.policy.SubscriptionPolicy;

import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.util.List;
import java.util.Map;
import java.util.Set;
Expand Down Expand Up @@ -65,6 +66,31 @@ public interface APIProvider extends APIManager {
Comment getComment(ApiTypeWrapper apiTypeWrapper, String commentId, Integer replyLimit, Integer replyOffset) throws
APIManagementException;

/**
* This method is to delete Sequence Backend by type
*
* @param apiUUID API Id
* @param type Key type
* @throws APIManagementException If failed to delete Sequence Backend
*/
void deleteCustomBackendByID(String apiUUID, String type) throws APIManagementException;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets add Java doc comments. Check other places as well


/**
* This method is to delete all Sequence Backends by APIID
*
* @param apiUUID API ID
* @throws APIManagementException If failed to delete Sequence Backend
*/
void deleteCustomBackendByAPIID(String apiUUID) throws APIManagementException;

/**
* This method is to delete Sequence Backends of a specific revision
* @param apiUUID API Id
* @param revisionId Revision Id
* @throws APIManagementException If failed to delete Sequence Backend
*/
void deleteSequenceBackendByRevision(String apiUUID, String revisionId) throws APIManagementException;

/**
* @param apiTypeWrapper Api type wrapper
* @param parentCommentID
Expand Down Expand Up @@ -315,6 +341,39 @@ List<SubscribedAPI> getSubscriptionsOfAPI(String apiName, String apiVersion, Str
*/
API updateAPI(API api, API existingAPI) throws APIManagementException, FaultGatewaysException;

/**
* This method is to update Sequence Backend
*
* @param api API
* @param type Key Type
* @param sequence Sequence Content
* @param seqName Sequence Name
* @param customBackendUUID Sequence Id
* @throws APIManagementException If not updated
*/
void updateCustomBackend(String api, String type, String sequence, String seqName, String customBackendUUID)
throws APIManagementException;

/**
* THis method is to retrieve Sequence Backend data
*
* @param apiUUID API Id
* @param type Key Type
* @return SequenceBackendData object
* @throws APIManagementException If data is not properly retrieved
*/
SequenceBackendData getCustomBackendByAPIUUID(String apiUUID, String type) throws APIManagementException;

/**
* This method is to retrieve all Sequence Backends of an API
*
* @param apiUUID API Id
* @return List of Sequence Backends
* @throws APIManagementException If not found
*/

List<SequenceBackendData> getAllSequenceBackendsByAPIUUID(String apiUUID) throws APIManagementException;

/**
* Create a new version of the <code>api</code>, with version <code>newVersion</code>
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -482,6 +482,9 @@ public enum ExceptionCodes implements ErrorHandler {
// API import/export related codes
ERROR_READING_META_DATA(900907, "Error while reading meta information from the definition", 400,
"Error while reading meta information from the definition"),

ERROR_READING_CUSTOM_SEQUENCE(900908, "Error while reading Custom Sequence from the API Endpoint Configuration",
400, "Error while reading Custom Sequence from the API Endpoint Configuration"),
ERROR_READING_PARAMS_FILE(900908, "Error while reading meta information from the params file", 400,
"Error while reading meta information from the params file"),
ERROR_FETCHING_DEFINITION_FILE(900909, "Cannot find the definition file of the project", 400,
Expand Down Expand Up @@ -531,6 +534,8 @@ public enum ExceptionCodes implements ErrorHandler {
"Required attributes(s) %s for api policy specification %s are either missing or empty"),
OPERATION_POLICY_NOT_FOUND(902010, "API Policy Not Found", 404,
"Requested api policy with id '%s' not found"),
CUSTOM_BACKEND_NOT_FOUND(903250, "Sequence Backend not found",
404, "Requested Sequence Backend of API '%s' not found"),

OPERATION_POLICY_ALREADY_EXISTS(903001, "The API Policy already exists.", 409, "An Operation Policy with name '%s' and version '%s' already exists"),

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ public class API implements Serializable {
private boolean apiResourcePatternsChanged;

private String status;
private String sequence;

private String technicalOwner;
private String technicalOwnerEmail;
Expand Down Expand Up @@ -786,6 +787,14 @@ public void setRating(float rating) {
this.rating = rating;
}

public void setSequence(String sequence) {
this.sequence = sequence;
}

public String getSequence() {
return sequence;
}

public void setLatest(boolean latest) {
isLatest = latest;
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
/*
* Copyright (c) 2024, WSO2 Inc. (http://www.wso2.org) All Rights Reserved.
*
* WSO2 Inc. licenses this file to you under the Apache License,
* Version 2.0 (the "License"); you may not use this file except
* in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

package org.wso2.carbon.apimgt.api.model;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add license header


public class SequenceBackendData {
private String Id;
private String sequence;
private String type;
private String name;
private String apiUUID;
private String revisionUUID;

public String getSequence() {
return sequence;
}

public void setSequence(String sequence) {
this.sequence = sequence;
}

public String getId() {
return Id;
}

public void setId(String id) {
Id = id;
}

public String getType() {
return type;
}

public void setType(String type) {
this.type = type;
}

public String getName() {
return name;
}

public void setName(String name) {
this.name = name;
}

public String getApiUUID() {
return apiUUID;
}

public void setApiUUID(String apiUUID) {
this.apiUUID = apiUUID;
}

public String getRevisionUUID() {
return revisionUUID;
}

public void setRevisionUUID(String revisionUUID) {
this.revisionUUID = revisionUUID;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@
import org.apache.synapse.transport.dynamicconfigurations.DynamicProfileReloaderHolder;
import org.wso2.carbon.apimgt.api.APIManagementException;
import org.wso2.carbon.apimgt.api.ExceptionCodes;
import org.wso2.carbon.apimgt.api.APIManagementException;
import org.wso2.carbon.apimgt.api.gateway.GatewayAPIDTO;
import org.wso2.carbon.apimgt.api.gateway.GatewayContentDTO;
import org.wso2.carbon.apimgt.api.gateway.GraphQLSchemaDTO;
Expand Down Expand Up @@ -330,6 +329,8 @@ private void unDeployAPI(APIGatewayAdmin apiGatewayAdmin, DeployAPIInGatewayEven
gatewayAPIDTO);
GatewayUtils.setEndpointsToBeRemoved(apiProductIdentifier, associatedApi.getUuid(),
gatewayAPIDTO);
GatewayUtils.setCustomBackendToBeRemoved(apiProductIdentifier, associatedApi.getUuid(),
gatewayAPIDTO);
}
} else {
API api = new API(new APIIdentifier(gatewayEvent.getProvider(), gatewayEvent.getName(),
Expand All @@ -350,6 +351,8 @@ private void unDeployAPI(APIGatewayAdmin apiGatewayAdmin, DeployAPIInGatewayEven
}

GatewayUtils.setCustomSequencesToBeRemoved(api, gatewayAPIDTO);
GatewayUtils.setCustomBackendToBeRemoved(gatewayAPIDTO);

}
gatewayAPIDTO.setLocalEntriesToBeRemove(
GatewayUtils
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -109,11 +109,12 @@ public boolean handleResponse(MessageContext messageContext) {
org.apache.axis2.context.MessageContext.setCurrentMessageContext(axis2MC);
if (messageContext.getProperty(APIMgtGatewayConstants.BACKEND_REQUEST_END_TIME) == null) {
messageContext.setProperty(APIMgtGatewayConstants.BACKEND_REQUEST_END_TIME, System.currentTimeMillis());
if (APIUtil.isAnalyticsEnabled()) {
long executionStartTime = Long.parseLong((String) messageContext.getProperty(APIMgtGatewayConstants
.BACKEND_REQUEST_START_TIME));
messageContext.setProperty(APIMgtGatewayConstants.BACKEND_LATENCY, System.currentTimeMillis() -
executionStartTime);
if (APIUtil.isAnalyticsEnabled()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will this be applied in Sequence as a backend scenario

&& messageContext.getProperty(APIMgtGatewayConstants.BACKEND_REQUEST_START_TIME) != null) {
long executionStartTime = Long.parseLong(
(String) messageContext.getProperty(APIMgtGatewayConstants.BACKEND_REQUEST_START_TIME));
messageContext.setProperty(APIMgtGatewayConstants.BACKEND_LATENCY,
System.currentTimeMillis() - executionStartTime);
}
}
return true;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1783,6 +1783,8 @@ private ConfigParameters() {
public static final String ENDPOINT_TYPE_SERVICE = "service";
public static final String ENDPOINT_TYPE_ADDRESS = "address";
public static final String ENDPOINT_TYPE_AWSLAMBDA = "awslambda";
public static final String ENDPOINT_TYPE_SEQUENCE = "sequence_backend";
public static final String SEQUENCE_DATA = "sequence";
public static final String ENDPOINT_PRODUCTION_FAILOVERS = "production_failovers";
public static final String ENDPOINT_SANDBOX_FAILOVERS = "sandbox_failovers";
public static final String ENDPOINT_PRODUCTION_ENDPOINTS = "production_endpoints";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@
import org.wso2.carbon.apimgt.api.model.BlockConditionsDTO;
import org.wso2.carbon.apimgt.api.model.Comment;
import org.wso2.carbon.apimgt.api.model.CommentList;
import org.wso2.carbon.apimgt.api.model.SequenceBackendData;
import org.wso2.carbon.apimgt.api.model.DeployedAPIRevision;
import org.wso2.carbon.apimgt.api.model.Documentation;
import org.wso2.carbon.apimgt.api.model.Documentation.DocumentSourceType;
Expand Down Expand Up @@ -1167,6 +1168,30 @@ private void updateAPIPolicies(API api, String tenantDomain) throws APIManagemen
apiMgtDAO.updateAPIPoliciesMapping(api.getUuid(), api.getUriTemplates(), api.getApiPolicies(), tenantDomain);
}

@Override
public void updateCustomBackend(String apiUUID, String type, String sequence, String seqName,
String customBackendUUID) throws APIManagementException {
apiMgtDAO.updateCustomBackend(apiUUID, seqName, sequence, type, customBackendUUID);
}

@Override
public SequenceBackendData getCustomBackendByAPIUUID(String apiUUID, String type) throws APIManagementException {
return apiMgtDAO.getCustomBackendByAPIUUID(apiUUID, type);
}

public List<SequenceBackendData> getAllSequenceBackendsByAPIUUID(String apiUUID) throws APIManagementException {
return apiMgtDAO.getSequenceBackendsByAPIUUID(apiUUID);
}

@Override
public void deleteCustomBackendByAPIID(String apiUUID) throws APIManagementException {
apiMgtDAO.deleteCustomBackendByAPIID(apiUUID);
}
@Override
public void deleteCustomBackendByID(String apiUUID, String type) throws APIManagementException {
apiMgtDAO.deleteCustomBackend(apiUUID, type);
}

private void validateKeyManagers(API api) throws APIManagementException {

Map<String, KeyManagerDto> tenantKeyManagers = KeyManagerHolder.getGlobalAndTenantKeyManagers(tenantDomain);
Expand Down Expand Up @@ -2561,6 +2586,8 @@ public void deleteAPI(String apiUuid, String organization) throws APIManagementE
// DB delete operations
if (!isError && api != null) {
try {
// Remove Custom Backend entries of the API
deleteCustomBackendByAPIID(apiUuid);
deleteAPIRevisions(apiUuid, organization);
deleteAPIFromDB(api);
if (log.isDebugEnabled()) {
Expand Down Expand Up @@ -5361,6 +5388,12 @@ public boolean isSubscriptionValidationDisabled(String uuid) throws APIManagemen
return !"ENABLED".equalsIgnoreCase(status);
}

@Override
public void deleteSequenceBackendByRevision(String apiUUID, String revisionId)
throws APIManagementException {
apiMgtDAO.deleteCustomBackendByRevision(apiUUID, "0");
}

@Override
public API getAPIbyUUID(String uuid, String organization) throws APIManagementException {
Organization org = new Organization(organization);
Expand All @@ -5383,6 +5416,7 @@ public API getAPIbyUUID(String uuid, String organization) throws APIManagementEx
if (APIConstants.API_SUBTYPE_AI_API.equals(api.getSubtype())) {
populateAiConfiguration(api);
}

if (APIUtil.isSequenceDefined(api.getInSequence()) || APIUtil.isSequenceDefined(api.getOutSequence())
|| APIUtil.isSequenceDefined(api.getFaultSequence())) {
if (migrationEnabled == null) {
Expand Down Expand Up @@ -6099,6 +6133,7 @@ public String addAPIRevision(APIRevision apiRevision, String organization) throw
ExceptionCodes.from(ExceptionCodes.API_REVISION_UUID_NOT_FOUND));
}
apiRevision.setRevisionUUID(revisionUUID);

try {
apiMgtDAO.addAPIRevision(apiRevision);
AIConfiguration configuration = apiMgtDAO.getAIConfiguration(apiRevision.getApiUUID(), null, organization);
Expand Down Expand Up @@ -6302,6 +6337,7 @@ public void deployAPIRevision(String apiId, String apiRevisionUUID,
}
}
apiMgtDAO.addAPIRevisionDeployment(apiRevisionUUID, apiRevisionDeployments);

WorkflowExecutor revisionDeploymentWFExecutor = getWorkflowExecutor(
WorkflowConstants.WF_TYPE_AM_REVISION_DEPLOYMENT);

Expand Down
Loading
Loading