Add token string normalization

[brodmo]

This adds the framework for token string normalization. It also generates semantic information and adds it to the tokens in the Java language module. This cannot be disabled. I couldn't measure a difference in runtime, however.

[dfuchss]

Some further comments. Code in general lgtm

[tsaglam]

Looks good! Just one conflict to solve, @mbrdl, then I can merge.

[dfuchss]

Why did you revert the switch? f3f061e

It should enforce that you have to define behavior for all enum values. (If you don't use default)

[brodmo]

Why did you revert the switch? f3f061e

It should enforce that you have to define behavior for all enum values. (If you don't use default)

But the default is doing nothing? I don't understand, maybe you could provide a code snippet to show me what you mean. I reverted the switch because Sonar didn't like it.

[dfuchss]

Why did you revert the switch? f3f061e

It should enforce that you have to define behavior for all enum values. (If you don't use default)

But the default is doing nothing? I don't understand, maybe you could provide a code snippet to show me what you mean. I reverted the switch because Sonar didn't like it.

I can do that later. The idea would be to use the new switch without default . Then you have to define all cases (therefore, it will cause a compile error if new enum values will be added)

[brodmo]

Why did you revert the switch? f3f061e
It should enforce that you have to define behavior for all enum values. (If you don't use default)

But the default is doing nothing? I don't understand, maybe you could provide a code snippet to show me what you mean. I reverted the switch because Sonar didn't like it.

I can do that later. The idea would be to use the new switch without default . Then you have to define all cases (therefore, it will cause a compile error if new enum values will be added)

From my understanding, you only have to define all cases in a switch expression, not in a switch statement.

        switch (type) {
            case VARIABLE_FLOW -> isVariableFlow = true;
            case VARIABLE_REVERSE_FLOW -> isVariableReverseFlow = true;
        }

compiles just fine, even though more types exist.

[dfuchss]

Why did you revert the switch? f3f061e
It should enforce that you have to define behavior for all enum values. (If you don't use default)

But the default is doing nothing? I don't understand, maybe you could provide a code snippet to show me what you mean. I reverted the switch because Sonar didn't like it.

I can do that later. The idea would be to use the new switch without default . Then you have to define all cases (therefore, it will cause a compile error if new enum values will be added)

From my understanding, you only have to define all cases in a switch expression, not in a switch statement.

        switch (type) {
            case VARIABLE_FLOW -> isVariableFlow = true;
            case VARIABLE_REVERSE_FLOW -> isVariableReverseFlow = true;
        }

compiles just fine, even though more types exist.

I have to check that. There was some kind of switch that enforced that all types are mentioned. As far as I remember.

[dfuchss]

E.g., https://www.happycoders.eu/java/switch-expression/

-> Completeness Analysis

[dfuchss]

But I've recognized in the case where I've marked the code, this does not make sense. So just ignore my initial comment there :)

[dfuchss]

My problem was that I did not realized that you set two different variables based on the enum value.

[brodmo]

Now that I've turned in my thesis, I've decided to add some tests before the presentation on 5/5. Could be good to have them in this PR so we can see the coverage, on the other hand it would be nice to have this PR out of the way. What do you think @tsaglam

[tsaglam]

Now that I've turned in my thesis, I've decided to add some tests before the presentation on 5/5. Could be good to have them in this PR so we can see the coverage, on the other hand it would be nice to have this PR out of the way. What do you think

@mbrdl both is fine with me, whatever you prefer goes. Merging this PR now has the benefit of avoiding conflicts when other PRs are merged.

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

89.7% Coverage
0.0% Duplication

[brodmo]

Test is here, ready to merge 👍 Coverage looking a lot better now 🙃 @tsaglam