Add support for managing gateway flow rules and customized API group in Sentinel dashboard #869

cdfive · 2019-06-27T06:01:02Z

Describe what this PR does / why we need it

Support managing gateway flow rules and customized API group definitions via dashboard.

Does this pull request fix one issue?

Fixes #699

Describe how you did it

Use appType in heartbeat message of client to distinguish normal service or gateway service.
Add appType in AppInfo and MachineInfo class, add two menus for gateway type app in
sidebar.html.
Add GatewayApiController, GatewayFlowRuleController and corresponding entity,repository,vo
in gateway package(controller/gateway,datasource/entity/gateway,repository/gateway,vo/gateway).
Add fetch and modify method in SentinelApiClient to interactive with gateway command handler.
Add api.html,flow.html and js files in corresponding gateway folder.

Describe how to verify it

Start dashboard with VM argument:
-Dproject.name=sentinel-dashboard -Dcsp.sentinel.dashboard.server=localhost:8080
Start sentinel-demo-spring-cloud-gateway with VM argument:
-Dproject.name=sc-gateway-demo -Dcsp.sentinel.dashboard.server=localhost:8080 -Dcsp.sentinel.app.type=1
Visit http://localhost:8090/httpbin/json to make sentinel init in
sc-gateway-demo.
View and operate list/add/update/delete API group and gateway flow rules via dashboard pages,
check sc-gateway-demo's api groups or flow rules by visiting http://localhost:8720/gateway/getApiDefinitions,http://localhost:8720/gateway/getRules
Check via UI and view logs to verify.

A simple test class ApiDefinitionControllerTest is added, I'll try to improve the unit test cases later.

Special notes for reviews

To make the gateway flow rules in order, the type of rule map in GatewayRuleManager class is changed from ConcurrentHashMap to Collections.synchronizedMap(new LinkedHashMap()
HashSet to LinkedhashSet.
In this way, when user add or update one rule, then refresh the list page, the order of rules can remain
(in adding order).
To support set interval by different unit of second,minute,hour,day via dashboard, a property named intervalUnit is added in GatewayFlowRule class, and intervalSec change to interval, the calIntervalSec method is added to get the interval second by interval and intervalUnit.
Previously, a variable name in login.js isn't a good naming, though it doesn't affect functionality, I change the LoginService to AuthService, to make the meaning accurate.
Besides, I post a blog about how to build the front-end developing environment of sentinel dashboard, which may help users who are in first participation, and add it in awesome-sentinel.md.

PTAL.

sczyh30 · 2019-06-27T06:03:00Z

👍

codecov-io · 2019-06-27T06:14:48Z

Codecov Report

Merging #869 into master will decrease coverage by 0.09%.
The diff coverage is 0%.

@@             Coverage Diff             @@
##             master     #869     +/-   ##
===========================================
- Coverage     42.57%   42.48%   -0.1%     
+ Complexity     1437     1436      -1     
===========================================
  Files           310      310             
  Lines          8974     8987     +13     
  Branches       1217     1219      +2     
===========================================
- Hits           3821     3818      -3     
- Misses         4685     4699     +14     
- Partials        468      470      +2

Impacted Files	Coverage Δ	Complexity Δ
...UpdateGatewayApiDefinitionGroupCommandHandler.java	`0% <0%> (ø)`	`0 <0> (ø)`	⬇️
.../slots/block/flow/controller/WarmUpController.java	`78.84% <0%> (-3.85%)`	`10% <0%> (-1%)`
...a/csp/sentinel/slots/statistic/base/LeapArray.java	`67.32% <0%> (-2.98%)`	`33% <0%> (-1%)`
...inel/datasource/zookeeper/ZookeeperDataSource.java	`66.33% <0%> (-1%)`	`15% <0%> (-1%)`
...m/alibaba/csp/sentinel/log/DateFileLogHandler.java	`57.57% <0%> (+3.03%)`	`7% <0%> (+1%)`	⬆️
...ava/com/alibaba/csp/sentinel/node/ClusterNode.java	`100% <0%> (+4.76%)`	`8% <0%> (+1%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 90662a4...a2fce6f. Read the comment docs.

...mmon/src/main/java/com/alibaba/csp/sentinel/adapter/gateway/common/rule/GatewayFlowRule.java

...n/src/main/java/com/alibaba/csp/sentinel/adapter/gateway/common/rule/GatewayRuleManager.java

sczyh30 · 2019-07-02T11:00:57Z

sentinel-dashboard/src/main/webapp/resources/app/views/dialog/gateway/flow-rule-dialog.html

+						<div>
+							<label class="col-sm-2 control-label">间隔</label>
+							<div class="col-sm-3">
+								<input type='number' min="1" class="form-control highlight-border" ng-model='currentRule.interval' placeholder="单位(秒)" />


Maybe we could update the placeholder here?

Meticulous! Yes, it should be updated when user change different interval unit.

Oh, as the select after the input displayed the interval unit, how about the placeholder here just "请输入"?

Fixed, with tip text 统计窗口时间长度 in this placeholder.

sczyh30 · 2019-07-02T11:02:26Z

sentinel-dashboard/src/main/webapp/resources/app/views/dialog/gateway/flow-rule-dialog.html

+						<label class="col-sm-5 control-label">
+							<div class="form-control highlight-border" align="center">
+								<input type="radio" value="0" checked ng-model="currentRule.paramItem.matchStrategy" title="精确" />&nbsp;精确&nbsp;&nbsp;
+								<input type="radio" value="1" ng-model="currentRule.paramItem.matchStrategy" title="前缀" />&nbsp;前缀&nbsp&nbsp;


matchStrategy 中前缀模式不需要，我们只支持这几种就可以：

精确：0

子串：3

正则：2

对应的值可参考 SentinelGatewayConstants

matchStrategy中的前缀已去掉，目前3种matchStrategy 精确：0 子串：1 正则：2
对应的界面和SentinelGatewayConstants类都已修改。

sczyh30 · 2019-07-02T12:07:54Z

sentinel-dashboard/src/main/webapp/resources/app/scripts/controllers/gateway/flow.js

+    };
+
+    function saveRule(rule, edit) {
+      GatewayFlowService.saveRule(rule).success(function (data) {


Maybe we need to check whether the rule is valid here?

In 115 line, before calling addNewRule and saveRule , call GatewayFlowService.checkRuleValid($scope.currentRule) to check, and the check method has been improved.

sczyh30 · 2019-07-02T15:14:01Z

...java/com/alibaba/csp/sentinel/dashboard/datasource/entity/gateway/GatewayFlowRuleEntity.java

+
+    private String resource;
+    private Integer resourceMode;
+


Missing grade here?

Yes， grade propertity has been added.

sentinel-dashboard/src/main/webapp/resources/app/scripts/controllers/gateway/flow.js

sczyh30 · 2019-07-05T13:49:43Z

Fabulous! Just one more advice: we need to keep the customized API name unique. We could add validation in the "new API definition" dialog to check whether the current API list contains the API group name.

cdfive · 2019-07-05T14:29:43Z

Fabulous! Just one more advice: we need to keep the customized API name unique. We could add validation in the "new API definition" dialog to check whether the current API list contains the API group name.

Good idea, the customized API name should be unique.

cdfive · 2019-07-05T15:27:48Z

The unique validation for customized API name has been added, both front and back ends. Since the api name can't be modified when editing, only validate when adding new customized API .

.../src/main/java/com/alibaba/csp/sentinel/adapter/gateway/common/SentinelGatewayConstants.java

...control/src/main/java/com/alibaba/csp/sentinel/slots/block/flow/param/ParamFlowRuleUtil.java

sczyh30 · 2019-07-09T01:51:18Z

sentinel-dashboard/src/main/webapp/resources/gulpfile.js

@@ -111,7 +111,8 @@ gulp.task('clean', function () {
 });

 // 总任务
-gulp.task('build', ['clean', 'jshint', 'lib', 'js', 'css']);
+// gulp.task('build', ['clean', 'jshint', 'lib', 'js', 'css']);
+gulp.task('build', ['clean', 'lib', 'js', 'css']);


Why is the jshint task removed?

Oh I'm sorry for commit local modify file, I'll fix it soon.

sczyh30 · 2019-07-09T15:42:31Z

Nice work! Could you please squash and separate your commits according to the corresponding module? For the commit related to the dashboard, it can be prefixed with dashboard: :)

… of dashboard in awesome-sentinel.md

…eGatewayApiDefinitionGroupCommandHandler

sczyh30

LGTM

sczyh30 · 2019-07-10T01:37:49Z

Thanks for your awesome contribution!

Signed-off-by: Eric Zhao <[email protected]>

Signed-off-by: Eric Zhao <[email protected]> (cherry picked from commit e50be35)

[RIP-10]fix-bug: ScheduleMessageServiceTest

sczyh30 added area/dashboard Issues or PRs about Sentinel Dashboard to-review To review labels Jun 27, 2019

sczyh30 requested changes Jun 27, 2019

View reviewed changes

...mmon/src/main/java/com/alibaba/csp/sentinel/adapter/gateway/common/rule/GatewayFlowRule.java Outdated Show resolved Hide resolved

...mmon/src/main/java/com/alibaba/csp/sentinel/adapter/gateway/common/rule/GatewayFlowRule.java Outdated Show resolved Hide resolved

sczyh30 requested changes Jun 27, 2019

View reviewed changes

...n/src/main/java/com/alibaba/csp/sentinel/adapter/gateway/common/rule/GatewayRuleManager.java Outdated Show resolved Hide resolved

sczyh30 requested changes Jul 2, 2019

View reviewed changes

sczyh30 reviewed Jul 8, 2019

View reviewed changes

.../src/main/java/com/alibaba/csp/sentinel/adapter/gateway/common/SentinelGatewayConstants.java Outdated Show resolved Hide resolved

...control/src/main/java/com/alibaba/csp/sentinel/slots/block/flow/param/ParamFlowRuleUtil.java Outdated Show resolved Hide resolved

sczyh30 reviewed Jul 9, 2019

View reviewed changes

cdfive added 3 commits July 10, 2019 09:23

doc: add a blog link about building front-end development environment…

2e1d690

… of dashboard in awesome-sentinel.md

Add parse method for parse ApiPredicateItem from json string in Updat…

d0357b8

…eGatewayApiDefinitionGroupCommandHandler

dashboard: add support for managing gateway flow rules (alibaba#699)

a2fce6f

cdfive force-pushed the dashboard/gateway-rules branch from b80fedb to a2fce6f Compare July 10, 2019 01:30

sczyh30 approved these changes Jul 10, 2019

View reviewed changes

sczyh30 merged commit 856ff31 into alibaba:master Jul 10, 2019

sczyh30 removed the to-review To review label Jul 10, 2019

sczyh30 added this to the 1.7.0 milestone Jul 10, 2019

sentinel-bot mentioned this pull request Jul 12, 2019

[WeeklyReport] Weekly report for Sentinel 2019-7-5 to 2019-7-12 #907

Closed

sczyh30 added a commit that referenced this pull request Jul 23, 2019

dashboard: code and frontend style refinement for #927 and #869

e50be35

Signed-off-by: Eric Zhao <[email protected]>

sczyh30 modified the milestones: 1.7.0, 1.6.3 Jul 25, 2019

sczyh30 added a commit that referenced this pull request Jul 29, 2019

dashboard: code and frontend style refinement for #927 and #869

a802472

Signed-off-by: Eric Zhao <[email protected]> (cherry picked from commit e50be35)

sczyh30 mentioned this pull request Apr 28, 2021

解决Dashboard网关流控规则推送丢失间隔时间的问题 #2171

Closed

CST11021 pushed a commit to CST11021/Sentinel that referenced this pull request Nov 3, 2021

[RIP-10]fix-bug: ScheduleMessageServiceTest (alibaba#869)

2e04ca7

[RIP-10]fix-bug: ScheduleMessageServiceTest

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for managing gateway flow rules and customized API group in Sentinel dashboard #869

Add support for managing gateway flow rules and customized API group in Sentinel dashboard #869

cdfive commented Jun 27, 2019

sczyh30 commented Jun 27, 2019

codecov-io commented Jun 27, 2019 •

edited

Loading

sczyh30 Jul 2, 2019

cdfive Jul 2, 2019

cdfive Jul 2, 2019

cdfive Jul 2, 2019

sczyh30 Jul 2, 2019

cdfive Jul 2, 2019

sczyh30 Jul 2, 2019

cdfive Jul 2, 2019

sczyh30 Jul 2, 2019

cdfive Jul 2, 2019

sczyh30 commented Jul 5, 2019

cdfive commented Jul 5, 2019

cdfive commented Jul 5, 2019 •

edited

Loading

sczyh30 Jul 9, 2019

cdfive Jul 9, 2019

sczyh30 commented Jul 9, 2019

sczyh30 left a comment

sczyh30 commented Jul 10, 2019

Add support for managing gateway flow rules and customized API group in Sentinel dashboard #869

Add support for managing gateway flow rules and customized API group in Sentinel dashboard #869

Conversation

cdfive commented Jun 27, 2019

Describe what this PR does / why we need it

Does this pull request fix one issue?

Describe how you did it

Describe how to verify it

Special notes for reviews

sczyh30 commented Jun 27, 2019

codecov-io commented Jun 27, 2019 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sczyh30 commented Jul 5, 2019

cdfive commented Jul 5, 2019

cdfive commented Jul 5, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sczyh30 commented Jul 9, 2019

sczyh30 left a comment

Choose a reason for hiding this comment

sczyh30 commented Jul 10, 2019

codecov-io commented Jun 27, 2019 •

edited

Loading

cdfive commented Jul 5, 2019 •

edited

Loading