getTypeReference(TypeReference) returns unexpected result when a type has multiple parameters

[tiwatsuka]

Abstract

getTypeReference(TypeReference) in spoon.support.compiler.jdt.ReferenceBuilder returns unexpected result when a type has multiple parameters. It can cause exception when building a model.

Cause of the problem and mitigation

spoon/src/main/java/spoon/support/compiler/jdt/ReferenceBuilder.java

Line 660 in c3ed4be

if (!res.toStringDebug().replace(", ?", ",?").endsWith(nameParameterized)) {

When a type has multiple parameters, the result of res.toStringDebug() contains a whitespace after commas between the parameters (e.g., Foo<?, ?>). In contrast, nameParameterized does not contain any whitespaces (e.g., Foo<?,?>). To my understanding, replace(", ?", ",?") was inserted to fill the gap. However, it's worked only when the parameters are wildcard. Therefore, it should be replace(", ", ",") rather than replace(", ?", ",?").

How to reproduce

I couldn't create a minimum code to reproduce the problem. I found this problem when I try to build a model from following code with no class path mode.
https://github.com/google/guava/blob/559e1ac1359e8d5dc0941aa72691698185d65407/android/guava/src/com/google/common/collect/Maps.java

[andrewbwogi]

Hi @tiwatsuka, can you give an example of how you are using ReferenceBuilder and what the unexpected result is? Perhaps the code that is causing an exception when you build the model?

[tiwatsuka]

Thank you for your reply. I have just created a minimum repository to reproduce the problem.
https://github.com/tiwatsuka/spoon-3913

When I build the model, following exception is thrown.

Exception in thread "main" spoon.SpoonException: Unexpected QualifiedNameReference tokens Maps.NavigableKeySet<K, V> for typeRef: Maps.NavigableKeySet
at spoon.support.compiler.jdt.JDTTreeBuilderHelper.handleImplicit(JDTTreeBuilderHelper.java:553)
at spoon.support.compiler.jdt.JDTTreeBuilderHelper.handleImplicit(JDTTreeBuilderHelper.java:487)
at spoon.support.compiler.jdt.ReferenceBuilder.getTypeReference(ReferenceBuilder.java:524)
at spoon.support.compiler.jdt.ReferenceBuilder.getExecutableReference(ReferenceBuilder.java:457)
at spoon.support.compiler.jdt.JDTTreeBuilder.visit(JDTTreeBuilder.java:895)
at org.eclipse.jdt.internal.compiler.ast.QualifiedAllocationExpression.traverse(QualifiedAllocationExpression.java:677)
at org.eclipse.jdt.internal.compiler.ast.ReturnStatement.traverse(ReturnStatement.java:387)
at org.eclipse.jdt.internal.compiler.ast.MethodDeclaration.traverse(MethodDeclaration.java:411)
at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.traverse(TypeDeclaration.java:1821)
at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.traverse(TypeDeclaration.java:1673)
at org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.traverse(CompilationUnitDeclaration.java:822)
at org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.traverse(CompilationUnitDeclaration.java:783)
at spoon.support.compiler.jdt.JDTBasedSpoonCompiler.traverseUnitDeclaration(JDTBasedSpoonCompiler.java:480)
at spoon.support.compiler.jdt.JDTBasedSpoonCompiler.lambda$buildModel$0(JDTBasedSpoonCompiler.java:437)
at spoon.support.compiler.jdt.JDTBasedSpoonCompiler.forEachCompilationUnit(JDTBasedSpoonCompiler.java:464)
at spoon.support.compiler.jdt.JDTBasedSpoonCompiler.buildModel(JDTBasedSpoonCompiler.java:435)
at spoon.support.compiler.jdt.JDTBasedSpoonCompiler.buildUnitsAndModel(JDTBasedSpoonCompiler.java:372)
at spoon.support.compiler.jdt.JDTBasedSpoonCompiler.buildSources(JDTBasedSpoonCompiler.java:335)
at spoon.support.compiler.jdt.JDTBasedSpoonCompiler.build(JDTBasedSpoonCompiler.java:116)
at spoon.support.compiler.jdt.JDTBasedSpoonCompiler.build(JDTBasedSpoonCompiler.java:99)
at spoon.Launcher.buildModel(Launcher.java:772)
at example.App.main(App.java:11)

I tried to find the cause of the problem. I finally found that line 660 in ReferenceBuilder does not behave expectedly. The value of res.toStringDebug() is com.google.common.collect.Maps.NavigableKeySet<K, V> and nameParameterized is Maps.NavigableKeySet<K,V>. I think the whitespace after comma should be removed to evaluate the condition correctly.

[slarse]

I'm looking into this.

[slarse]

I tinkered with this a little bit before lunch and found minimal test input to reproduce it:

public final class Test<K,V> {
    public void test() {
        new Test.GenericType<K, V>(this) {
        };
    }

    static class GenericType<K, V> {
    }
}

There are a few important details to trigger the behavior:

1. There has to be some form of error with the constructor invocation, such that the proper binding is not resolved (e.g. constructor does not exist or is not on the classpath). In the above example, there's no constructor that takes parameters, yet it's invoked with this.
   • If there is no problem with the constructor invocation, then the behavior is not triggered as the binding is resolved by JDT, and Spoon does not have to do any heavy lifting.
2. There must be a creation of an anonymous class. Otherwise, for some reason, JDT creates an AllocationExpression instead of a QualifiedAllocationExpression (see point 4), which traverses a different code path.
3. The type that is subtyped with the anonymous class must be referenced with a qualified within the type (here, Test.GenericType).
   • Not to be mixed up with fully qualified name, just qualified within the enclosing type.

This causes a pretty weird state in JDT where we have a QualifiedAllocationExpression for which the binding is unknown, but the type can be without it being a problem reference, probably due to the creation of the anonymous type. To summarize, this is a "noclasspath" problem that only occurs on creation of anonymous types such that the constructor used for some reason cannot be resolved to a binding.

The fix suggested by @tiwatsuka does solve the symptoms, but I'm not yet entirely sure that's where the problem needs to be solved. That thing needs to be fixed regardless, but I need to dig a little more to determine if it solves the problem here, or just hides it.

[slarse]

This is now tfixed, thanks for reporting it @tiwatsuka