Multilple configs export #3088

Anilople · 2020-05-20T14:01:38Z

What's the purpose of this PR

Enhance the PR #1767

Which issue(s) this PR fixes:

issues

#1760
#1681

Brief changelog

Export multiple configs to a single .zip file.

Follow this checklist to help us incorporate your contribution quickly and easily:

Read the Contributing Guide before making this pull request.
Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
Write necessary unit tests to verify the code.
Run mvn clean test to make sure this pull request doesn't break anything.

Update fork

update forked code

merge across forks

Allow repeatable deletions and enhance thread safety (apolloconfig#3069)

…owner name

codecov-commenter · 2020-05-20T14:08:59Z

Codecov Report

Merging #3088 into master will decrease coverage by 0.26%.
The diff coverage is 25.68%.

@@             Coverage Diff              @@
##             master    #3088      +/-   ##
============================================
- Coverage     51.55%   51.28%   -0.27%     
- Complexity     2255     2275      +20     
============================================
  Files           433      439       +6     
  Lines         13447    13622     +175     
  Branches       1378     1385       +7     
============================================
+ Hits           6932     6986      +54     
- Misses         6036     6150     +114     
- Partials        479      486       +7

Impacted Files	Coverage Δ	Complexity Δ
...ip/framework/apollo/portal/entity/bo/ConfigBO.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
...ramework/apollo/portal/util/ConfigToFileUtils.java	`0.00% <ø> (ø)`	`0.00 <0.00> (ø)`
...framework/apollo/portal/util/NamespaceBOUtils.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
...rk/apollo/portal/service/ConfigsExportService.java	`12.30% <12.30%> (ø)`	`2.00 <2.00> (?)`
...rk/apollo/portal/service/ConfigsImportService.java	`15.62% <15.62%> (ø)`	`1.00 <1.00> (?)`
...llo/portal/controller/ConfigsExportController.java	`21.73% <23.07%> (+13.57%)`	`2.00 <2.00> (+1.00)`
...llo/portal/controller/ConfigsImportController.java	`37.50% <37.50%> (ø)`	`1.00 <1.00> (?)`
.../framework/apollo/portal/util/ConfigFileUtils.java	`64.58% <64.58%> (ø)`	`13.00 <13.00> (?)`
...mework/apollo/portal/service/NamespaceService.java	`67.30% <75.00%> (+0.20%)`	`27.00 <2.00> (ø)`
...ervice/service/ReleaseMessageServiceWithCache.java	`85.88% <0.00%> (-1.18%)`	`24.00% <0.00%> (-1.00%)`
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0341e5c...25fd38d. Read the comment docs.

… exporting

…roblem.

nobodyiam · 2020-07-05T05:00:51Z

@Anilople That's all right, we may just focus on the config export function in this pull request

apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/util/NamespaceBOUtils.java

…d value contain special char like '\n' due wrong format.

nobodyiam · 2020-07-12T02:06:46Z

apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/entity/bo/ConfigBO.java

+    this.format = null;
+  }
+
+  public ConfigBO(Env env, String ownerName, String appId, String clusterName,


It looks like this constructor is not used?

Change the constructor's logic. com.ctrip.framework.apollo.portal.entity.bo.ConfigBO#ConfigBO(com.ctrip.framework.apollo.portal.environment.Env, java.lang.String, java.lang.String, java.lang.String, com.ctrip.framework.apollo.portal.entity.bo.NamespaceBO) use it now.

public ConfigBO(Env env, String ownerName, String appId, String clusterName, NamespaceBO namespaceBO) { this(env, ownerName, appId, clusterName, namespaceBO.getBaseInfo().getNamespaceName(), NamespaceBOUtils.convert2configFileContent(namespaceBO), ConfigFileFormat.fromString(namespaceBO.getFormat()) ); }

apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/entity/bo/ConfigBO.java

apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/service/AppService.java

apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/service/ConfigsExportService.java

apollo-portal/src/main/resources/static/i18n/en.json

nobodyiam · 2020-07-12T02:30:55Z

apollo-portal/src/main/resources/static/views/common/nav.html

@@ -34,18 +34,20 @@
                        <li value="zh-CN"><a href="javascript:void(0)" ng-click="changeLanguage('zh-CN')">简体中文</a></li>
                    </ul>
                </li>
-                <li class="dropdown" ng-if="hasRootPermission">


I would suggest adding another drop down menu e.g. System Tools and leave the Admin Tools only for Admin Users

Add other menu in position 1 ?

How to name new menu better? tool?

Use different menu name when user is admin or non-admin.

For admin user

For non-admin user

How about this implementation?

This looks good

...lo-portal/src/main/java/com/ctrip/framework/apollo/portal/component/PermissionValidator.java

...rtal/src/main/java/com/ctrip/framework/apollo/portal/controller/ConfigsExportController.java

apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/service/ConfigsImportService.java

…ed to private

…changed to private

nobodyiam · 2020-07-19T03:00:12Z

apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/service/ConfigsExportService.java

+   * @param outputStream receive zip file output stream
+   * @throws IOException
+   */
+  private static void writeAsZipOutputStream(Stream<ConfigBO> configBOStream, OutputStream outputStream)


Is it necessarily marked as static?

The method write Stream<ConfigBO>'s content to an OutputStream.

Everytime the method is invoked, those parameters are different with before, and not use field in object,

so I think use static for this method is better.

nobodyiam · 2020-07-19T03:00:18Z

apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/service/ConfigsExportService.java

+   * @param configBO a namespace represent
+   * @return zip file output stream same as parameter zipOutputStream
+   */
+  private static ZipOutputStream write2ZipOutputStream(


Is it necessarily marked as static?

Same reson as above.

apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/service/ConfigsExportService.java

apollo-portal/src/main/resources/static/i18n/zh-CN.json

apollo-portal/src/main/resources/static/i18n/en.json

nobodyiam

OK

Anilople and others added 21 commits March 5, 2020 14:26

Merge pull request #1 from ctripcorp/master

27b3249

Update fork

Merge pull request #2 from ctripcorp/master

34e18a3

update forked code

sql(apollo-portal): apollo.portal.meta.servers

9e11624

fix(apollo-portal): sql lack comma ','

bd3a573

Merge branch 'master' into master

6921f1a

Merge pull request #3 from ctripcorp/master

6e4299c

merge across forks

Merge pull request #4 from ctripcorp/master

dcc6eb6

Allow repeatable deletions and enhance thread safety (apolloconfig#3069)

feat(apollo-portal): separate configs import and export

e660414

refactor(apollo-portal): export config file

7211e89

refactor(apollo-portal): import config file

86b7287

refactor(apollo-portal): uri

8be329b

refactor(apollo-portal): move the check of MultipartFile

06740d8

refactor(apollo-portal): The check of namespace

e045542

refactor(apollo-portal): import one config from file

c0a6d7c

refactor(apollo-portal): import one config from file

414f27a

feat(apollo-portal): import multiple config from files

2e92a68

feat(apollo-portal): export one environment's all configs

18fe3a3

feat(apollo-portal): export one environment's all configs, divide by …

47f0170

…owner name

refactor(apollo-portal): import config from file

7c1c4e2

refactor(apollo-portal): import config from zip file

0561285

feat(apollo-portal): export path in zip

6a1d455

Anilople added 8 commits May 20, 2020 23:08

fix(apollo-portal): auto close zip input stream

0f92de0

feat(apollo-portal): move uri "/apps" to method

c9f8e9b

feat(apollo-portal): use parallel java.util.stream.Stream to speed up…

9c4fa5d

… exporting

feat(apollo-portal): export api. reduce with Stream.

c0664be

feat(apollo-portal): configs file's path in the exported zip

4f9c269

feat(apollo-portal): use synchronized to prevent concurrent writing p…

4cfeceb

…roblem.

feat(apollo-portal): add permission validator

4a55549

feat(apollo-portal): import configs by env and app id

4d71522

feat(apollo-portal): Delete config import.

b6022e0

Anilople changed the title ~~Multilple configs import and export~~ Multilple configs export Jul 5, 2020

Anilople commented Jul 5, 2020

View reviewed changes

apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/util/NamespaceBOUtils.java Outdated Show resolved Hide resolved

Anilople added 3 commits July 11, 2020 12:32

feat(apollo-portal): use PropertiesUtil.toString(properties) to forbi…

c3cac5c

…d value contain special char like '\n' due wrong format.

Merge branch 'master' into multilple-configs-import-and-export

17c61c0

Merge branch 'master' into multilple-configs-import-and-export

2ed2911

nobodyiam requested changes Jul 12, 2020

View reviewed changes

nobodyiam reviewed Jul 12, 2020

View reviewed changes

apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/service/ConfigsImportService.java Outdated Show resolved Hide resolved

Anilople added 10 commits July 12, 2020 14:21

feat(apollo-portal): delete useless content in ConfigBO

fd6d812

feat(apollo-portal): delete findByAppId in AppService

3478580

feat(apollo-portal): public methods in ConfigsExportService are chang…

f36791e

…ed to private

feat(apollo-portal): delete permission check in ConfigsImportService

c995b30

feat(apollo-portal): some public methods in ConfigsImportService are …

14960c2

…changed to private

feat(apollo-portal): delete permission check in ConfigsImportService

e312ecc

feat(apollo-portal): predicate namespace's format

a17ee12

feat(apollo-portal): english message in front end.

9429c20

feat(apollo-portal): delete method shouldShowConfigToCurrentUser.

f26a38f

feat(apollo-portal): show 'Tools' menu when user is not admin

cd8b8be

Anilople requested a review from nobodyiam July 12, 2020 09:11

nobodyiam mentioned this pull request Jul 18, 2020

export datas from Apollo portal #3180

Closed

Merge branch 'master' into multilple-configs-import-and-export

0bf58d7

nobodyiam requested changes Jul 19, 2020

View reviewed changes

Anilople added 3 commits July 24, 2020 23:08

style(apollo-portal): format code

45225b2

doc(apollo-portal): ConfigExport.Download 点击下载 => 下载

b0309b3

doc(apollo-portal): ConfigExport.Download download => Download

25fd38d

Anilople requested a review from nobodyiam July 24, 2020 15:18

nobodyiam approved these changes Jul 25, 2020

View reviewed changes

nobodyiam merged commit f8f3706 into apolloconfig:master Jul 25, 2020

Anilople added the area/portal apollo-portal label Dec 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multilple configs export #3088

Multilple configs export #3088

Anilople commented May 20, 2020 •

edited

Loading

codecov-commenter commented May 20, 2020 •

edited

Loading

nobodyiam commented Jul 5, 2020

nobodyiam Jul 12, 2020

Anilople Jul 12, 2020

nobodyiam Jul 12, 2020

Anilople Jul 12, 2020

Anilople Jul 12, 2020

nobodyiam Jul 12, 2020

nobodyiam Jul 19, 2020

Anilople Jul 24, 2020 •

edited

Loading

nobodyiam Jul 19, 2020

Anilople Jul 24, 2020

nobodyiam left a comment

Multilple configs export #3088

Multilple configs export #3088

Conversation

Anilople commented May 20, 2020 • edited Loading

What's the purpose of this PR

Which issue(s) this PR fixes:

issues

Brief changelog

codecov-commenter commented May 20, 2020 • edited Loading

Codecov Report

nobodyiam commented Jul 5, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

For admin user

For non-admin user

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Anilople Jul 24, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nobodyiam left a comment

Choose a reason for hiding this comment

Anilople commented May 20, 2020 •

edited

Loading

codecov-commenter commented May 20, 2020 •

edited

Loading

Anilople Jul 24, 2020 •

edited

Loading