Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] [Style] Add Spotless plugin to fix formatting and style errors automatically #2641

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 4 additions & 7 deletions .github/workflows/backend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ jobs:
with:
submodules: true
- name: Check code style
run: ./mvnw --batch-mode --quiet --no-snapshot-updates clean checkstyle:check
run: ./mvnw --batch-mode --quiet --no-snapshot-updates clean spotless:check

dead-link:
if: github.repository == 'apache/incubator-seatunnel'
Expand Down Expand Up @@ -100,9 +100,8 @@ jobs:
cache: 'maven'
- name: Build distribution tar
run: >-
./mvnw -B install scalastyle:check
./mvnw -B install
-D"maven.test.skip"=true
-D"checkstyle.skip"=true
-D"license.skipAddThirdParty"=true
-D"http.keepAlive"=false
-D"maven.wagon.http.pool"=false
Expand Down Expand Up @@ -130,8 +129,6 @@ jobs:
./mvnw -B -q install -DskipTests
-D"maven.test.skip"=true
-D"maven.javadoc.skip"=true
-D"scalastyle.skip"=true
-D"checkstyle.skip"=true
-D"license.skipAddThirdParty"
- name: Check Dependencies Licenses
run: tools/dependencies/checkLicense.sh
Expand All @@ -155,7 +152,7 @@ jobs:
cache: 'maven'
- name: Run Unit tests
run: |
./mvnw -B -T 1C clean verify -D"maven.test.skip"=false -D"checkstyle.skip"=true -D"scalastyle.skip"=true -D"license.skipAddThirdParty"=true --no-snapshot-updates
./mvnw -B -T 1C clean verify -D"maven.test.skip"=false -D"license.skipAddThirdParty"=true --no-snapshot-updates
env:
MAVEN_OPTS: -Xmx2048m

Expand All @@ -178,6 +175,6 @@ jobs:
cache: 'maven'
- name: Run Integration tests
run: |
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"checkstyle.skip"=true -D"scalastyle.skip"=true -D"license.skipAddThirdParty"=true --no-snapshot-updates
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"license.skipAddThirdParty"=true --no-snapshot-updates
env:
MAVEN_OPTS: -Xmx2048m
2 changes: 1 addition & 1 deletion .github/workflows/codeql.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 60
env:
JAVA_TOOL_OPTIONS: -Xmx2G -Xms2G -Dhttp.keepAlive=false -Dmaven.test.skip=true -Dcheckstyle.skip=true -Dlicense.skipAddThirdParty=true -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false -Dmaven.wagon.http.retryHandler.count=3 -Dmaven.wagon.httpconnectionManager.ttlSeconds=120
JAVA_TOOL_OPTIONS: -Xmx2G -Xms2G -Dhttp.keepAlive=false -Dmaven.test.skip=true -Dlicense.skipAddThirdParty=true -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false -Dmaven.wagon.http.retryHandler.count=3 -Dmaven.wagon.httpconnectionManager.ttlSeconds=120

strategy:
fail-fast: false
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@ jobs:
run: |
./mvnw -B package \
-Dmaven.test.skip=true \
-Dcheckstyle.skip=true \
-Dlicense.skipAddThirdParty=true \
-Dhttp.keepAlive=false \
-Dmaven.wagon.http.pool=false \
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/license.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ jobs:
${{ runner.os }}-maven-
- name: Generate THIRD-PARTY
run: |
./mvnw clean license:aggregate-add-third-party -DskipTests -Dcheckstyle.skip -U
./mvnw clean license:aggregate-add-third-party -DskipTests -U
- name: Check LICENSE file
run: |
python3 tools/dependencies/license.py seatunnel-dist/target/THIRD-PARTY.txt seatunnel-dist/release-docs/LICENSE true
1 change: 0 additions & 1 deletion .github/workflows/publish-docker.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,6 @@ jobs:
./mvnw -B clean deploy \
-Dmaven.test.skip \
-Dmaven.javadoc.skip \
-Dmaven.checkstyle.skip \
-Dlicense.skipAddThirdParty=true \
-Dmaven.deploy.skip \
-Ddocker.tag=${{ github.sha }} \
Expand Down
24 changes: 16 additions & 8 deletions docs/en/contribution/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,9 @@ Otherwise, your code could not start in JetBrains IntelliJ IDEA correctly.
```

### Building SeaTunnel from source

After you install the maven, you can use the follow command to compile and package.

```
mvn clean package -pl seatunnel-dist -am -Dmaven.test.skip=true
```
Expand All @@ -52,13 +54,19 @@ See [install plugins for IDEA](https://www.jetbrains.com/help/idea/managing-plug
Before running the following example, you should also install JetBrains IntelliJ IDEA's [Lombok plugin](https://plugins.jetbrains.com/plugin/6317-lombok).
See [install plugins for IDEA](https://www.jetbrains.com/help/idea/managing-plugins.html#install-plugins) if you want to.

### Install JetBrains IDEA CheckStyle-IDEA Plugin
### Code Style

Before coding, you should also install JetBrains IntelliJ IDEA's [CheckStyle-IDEA plugin](https://plugins.jetbrains.com/plugin/1065-checkstyle-idea).
See [install plugins for IDEA](https://www.jetbrains.com/help/idea/managing-plugins.html#install-plugins) if you want to.
Next, you should go to `Preferences -> Editor -> Code style -> Scheme -> Import Scheme -> CheckStyle Configration` and import `tools/checkstyle/checkStyle.xml`
![checkstyle.png](../images/checkstyle.png)
If you want to change to automatically formatting, these configurations are also required.
SeaTunnel uses `Spotless` for code style and formatting checks.
You could run the following command and `Spotless` will automatically fix
the code style and formatting errors for you:

```shell
./mvnw spotless:apply
```

You could copy the `pre-commit hook` file `/tools/style/pre-commit` to your `.git/hooks/`
directory so that every time you commit your code with `git commit`, `Spotless` will automatically
fix things for you.

## Run Simple Example

Expand All @@ -69,7 +77,6 @@ it in IDEA](https://www.jetbrains.com/help/idea/run-debug-configuration.html) as
Here we use `seatunnel-examples/seatunnel-flink-examples/src/main/java/org/apache/seatunnel/example/flink/LocalFlinkExample.java`
as an example, when you run it successfully you could see the output as below:


```log
+I[Ricky Huo, 71]
+I[Gary, 12]
Expand All @@ -83,7 +90,7 @@ as an example, when you run it successfully you could see the output as below:

All our examples use simple source and sink to make it less dependent and easy to run. You can change the example configuration
in `resources/examples`. You could change your configuration as below, if you want to use PostgreSQL as the source and
sink to console.
sink to console.

```conf
env {
Expand All @@ -103,3 +110,4 @@ sink {
ConsoleSink {}
}
```

29 changes: 14 additions & 15 deletions docs/en/transform/nullRate.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,12 @@ This transform **ONLY** supported by Spark.

## Options

| name | type | required | default value |
| -------------------------| ------------ | -------- | ------------- |
| fields | string_list | yes | - |
| rates | double_list | yes | - |
| throw_exception_enable | boolean | no | - |
| save_to_table_name | string | no | - |


| name | type | required | default value |
|------------------------|-------------|----------|---------------|
| fields | string_list | yes | - |
| rates | double_list | yes | - |
| throw_exception_enable | boolean | no | - |
| save_to_table_name | string | no | - |

### field [string_list]

Expand Down Expand Up @@ -58,12 +56,13 @@ Transform plugin common parameters, please refer to [Transform Plugin](common-op
Use `NullRate` in transform's Dataset.

```bash
transform {
NullRate {
fields = ["msg", "name"]
rates = [10.0,3.45]
save_to_table_name = "tmp"
throw_exception_enable = true
}
transform {
NullRate {
fields = ["msg", "name"]
rates = [10.0,3.45]
save_to_table_name = "tmp"
throw_exception_enable = true
}
}
```

Loading