Bug?? Not resolving CtExecutableReference type and declared type depending on environment (Docker and local Spring Boot) #3926
Comments
|
I found some clue when I enabled Spoon Launcher logging by setting it to 'ALL' Here is what I got in Docker environment ...
The method findAll() is undefined for the type ValuesRepository at /opt/skillcounters/filestorage/github/vipinjo/assignment-consumer/work-tree/d4e050cf3566981ba386e336c5889f8d107abfbb/src/main/java/com/vipinjoseph/assignmentconsumer/service/ValueServices.java:32And in local environment these warning are not shown... Why is that? Is there some issue with classpath i.e. spring classes (CrudRepository for example) is not available for Spoon when running in Docker? How can I fix it? |
|
Next try. How can I include jars that are contained in that directory? |
|
Made another try with setting InputClassloader like this ...
URLClassLoader urlClassLoader = URLClassLoader.newInstance(
existingDirs.stream()
.map(this::toUrl)
.toArray(URL[]::new)
);
spoonAPI.getEnvironment().setInputClassLoader(urlClassLoader);
...where In logs I see that input classloader was set (classpath is initialized which is done by StandardEnvironment when I set InputClassloader) spoon.support.StandardEnvironment@400e5d30[errorCount=0,processingStopped=false,prettyPrintingMode=FULLYQUALIFIED,warningCount=0, But no effect - again Spoon can't find necessary jars and\or include it to classpath. BTW - I already switched version of spoon to 9.0.0. because in earlier versions InputClassLoader was not set if it's a URLClassLoader (fixed here - #3818) Well, I'm stuck... ((( |
|
At last I found a solution
URLClassLoader urlClassLoader = URLClassLoader.newInstance(
jarUrls.toArray(new URL[0])
,spoonAPI.getClass().getClassLoader()
);
spoonAPI.getEnvironment().setInputClassLoader(urlClassLoader);I strongly advice you to put this in docs for those like me are struggling with using spoon packaged in a jar application (for example Spring Boot) |
|
Hi @sok82, Thanks for the detailed summary of your findings, and sorry about the slowpoke response to your troubles. You should not have to set a custom classloader just to resolve types. General classpath resolution is explained here under First of all, to ensure that Spoon has all types in your project, you'll want to run Spoon in classpath mode. It will cause Spoon to scream bloody murder if there's an unresolved type. Launcher launcher = new Launcher();
launcher.getEnvironment().setNoclasspath(false);Then, you'll need to set the classpath. This can be done manually with String mavenProjectRoot = ...; // path to the directory with the pom.xml
MavenLauncher launcher = new MavenLauncher(mavenProjectRoot, MavenLauncher.SOURCE_TYPE.APP_SOURCE);
CtModel model = launcher.buildModel();See Hope that helps! |
|
Hi @slarse. Thanks a lot for your comments. Fortunately I managed to fix all that was necessary by myself - the way that I described above with custom classloader. I'm sure, that solution that you provided suits some 'standard' scenario, but not my case. I tried to use setNoClasspath(false), but that didn't help to resolve issue with dependencies in external jars. It's here in StandardEnvironment.java (as of version 9) private void verifySourceClasspath(String[] sourceClasspath) throws InvalidClassPathException {
String[] var2 = sourceClasspath;
int var3 = sourceClasspath.length;
for(int var4 = 0; var4 < var3; ++var4) {
String classPathElem = var2[var4];
File classOrJarFolder = new File(classPathElem);
if (!classOrJarFolder.exists()) {
throw new InvalidClassPathException(classPathElem + " does not exist, it is not a valid folder");
}
if (classOrJarFolder.isDirectory()) {
SpoonFolder tmp = new FileSystemFolder(classOrJarFolder);
List<SpoonFile> javaFiles = tmp.getAllJavaFiles();
if (!javaFiles.isEmpty()) {
this.print("You're trying to give source code in the classpath, this should be given to addInputSource " + javaFiles, Level.WARN);
}
this.print("You specified the directory " + classOrJarFolder.getPath() + " in source classpath, please note that only class files will be considered. Jars and subdirectories will be ignored.", Level.WARN);
} else if (classOrJarFolder.getName().endsWith(".class")) {
throw new InvalidClassPathException(".class files are not accepted in source classpath.");
}
}
}In any case my solution with classloader works, so I'm happy with it) Maybe be there is some caveat with it of which I do not know? Thanks a lot for your collaboration |
Then I would use
This is a safety measure, it doesn't find the classpath for you. It's just to ensure that all types are actually available. If you require all types to be resolved, you should always run in classpath mode, because then you'll be alerted if a type is not resolved. In the default noclasspath mode, you'll just sporadically find
You've taken the warning out of context: it will only appear if you add the path to a directory, that code path cannot be entered given the path directly to a JAR file. If you have a look at the docs, they say the following: spoon/src/main/java/spoon/compiler/Environment.java Lines 297 to 300 in 7912f0e In particular:
Which says that if you add the path to a directory containing jar files, those jar files are ignored. Jar files are only included if you add the full path to that jar. It's the same way providing the classpath to e.g.
You're free to keep using it, of course! I just wanted to add for completeness that setting the source classpath is the intended solution to your problem. I.e. this should work fine: spoonAPI.getEnvironment().setSourceClasspath(urls.stream().map(URL::toString).toArray(String[]::new));
Classloaders are kind of complicated, and I don't claim to have a full grasp on how the classloader inheritance works in practice (you're setting the default classloader as a parent), but there shouldn't be any problems with your solution. |
|
Thanks for clarification @slarse. Now I see where I lost context and what I could do wrong. I don't like playing with classloaders also) This seems like some magic for me sometimes - need to keep fingers crossed.. Thanks a lot and have a nice coding |
|
You're welcome, hope it all works out to your liking! |
Good day.
I encountered very strange issue using Spoon analysis library in my Java project.
I get different parsing results when running same code, that uses Spoon Launcher in different environments
In locally run Spring Boot project - it's all OK, but when I run the same code in Docker - I get
nullvalues inCtExecutableReference.getType()andCtExecutableReference.getDeclaredType()Here are the details
My Spoon version is 8.2.0. (from maven repo)
I am trying to parse (build AST) of code from this GitHub repository
And I have troubles parsing this class
There are following lines here
When I run analysis and try to parse
CtExecutableReferenceforfindAll()method ofthis.valuesRepository.findAll().forEach(values::add)statement I getnullvalues forgetType()andgetDeclaredType()when running my project in Docker.When running locally both
getType()andgetDeclaredType()have non-null valuesThe same issue happens when parsing other similar code blocks in other projects.
For example here
Statement with
this.betRepository.findAll()has null in bothgetTypeandgetDeclaredTypewhen running in Docker, but ok in local environment.At the same time following code is parsed well in both environments
and statement
betRepository.findByCustomerId()is parsed ok and has necessary type info both in Docker and n local Spring Boot run.I double checked local tests - and all is ok - when running code in tests from IDE or starting Spring Boot project from IDE and initialising analysis by calling service from Web UI - it OK and works as expected.
But when I build Docker image - I get null in both type and declaredType.
I'm running Spoon analysis with the following code
Before running Launcher I tried to print it's settings. Here is what i got
I'm running project on Java8 - in both environments (details following).
To build docker I use following commands
FROM java:8 COPY maven /maven/ ENTRYPOINT java -Xverify:none -XX:TieredStopAtLevel=1 -XX:+TieredCompilation -XX:+UseSerialGC -XX:MinHeapFreeRatio=20 -XX:MaxHeapFreeRatio=40 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -Djava.security.egd=file:/dev/./urandom -Dspring.profiles.active=${PROFILE:-docker-dev} -jar /maven/skillcounters-sca-service-1.0-SNAPSHOT.jarI tried to switch to different Docker base images (openjdk \ alpine etc.), but nothing helped.
I tried to exclude all java run options listed above (i.e. like -XXblabla) - didn't helped either.
To get ideas about what may go wrong i print all environment (including java) data on application start.
Here what is printed for local environment
And here is what printed in Docker environment
Any help would be appreciated
The text was updated successfully, but these errors were encountered: