Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gradletest can't download dependency #256

Closed
scphantm opened this issue Jun 21, 2018 · 7 comments
Closed

gradletest can't download dependency #256

scphantm opened this issue Jun 21, 2018 · 7 comments
Labels

Comments

@scphantm
Copy link

using the very basic example in my enterprise plugin system. whenever i try to activate any of the groovy features, my gradletests spit this error.

Why is this trying to dynamically download this dependency? Why isn't it built into the normal dependencies?

we run a custom version of gradle that has its own init with these repos in it, but it looks like your system doesn't pick those repos up when run inside a gradletest.

project.plugins.apply(SyzygyJavascriptPlugin)
|       |       |
|       |       org.gradle.api.internal.plugins.PluginApplicationException: Failed to apply plugin [class 'com.hmhs.frameworks.syzygy.gradle.core.quality.plugins.SyzygyQualityPlugin']
|       [org.gradle.language.base.plugins.LifecycleBasePlugin@7a83823a, org.gradle.api.plugins.BasePlugin@47dcb632, org.gradle.api.plugins.ReportingBasePlugin@5d4a4daa, org.gradle.platform.base.plugins.ComponentBasePlugin@6ad4ddd1, org.gradle.language.base.plugins.LanguageBasePlugin@11ed620c, org.gradle.platform.base.plugins.BinaryBasePlugin@3227834b, org.gradle.api.plugins.JavaBasePlugin@6f433986, org.gradle.api.plugins.JavaPlugin@4937face, org.gradle.api.plugins.GroovyBasePlugin@36e0d656, org.gradle.api.plugins.GroovyPlugin@69f3b241, org.gradle.plugins.ide.idea.IdeaPlugin@36330d88, org.gradle.plugins.ide.eclipse.EclipsePlugin@1542f688, org.gradle.api.plugins.MavenPlugin@522eb329, org.gradle.api.plugins.ProjectReportsPlugin@4a9c7ca0, com.hmhs.frameworks.syzygy.gradle.core.plugins.SyzygyTasktreePlugin@3ed10838, org.gradle.api.plugins.quality.CodeNarcPlugin@7c7c28f3, org.gradle.api.plugins.quality.CheckstylePlugin@595d49, com.diffplug.gradle.spotless.SpotlessPlugin@69c17f7a]
root project 'test'
    at com.hmhs.frameworks.syzygy.gradle.js.tasks.WebjarTaskTest.can apply javascript plugin(WebjarTaskTest.groovy:13)

    Caused by:
    org.gradle.api.internal.plugins.PluginApplicationException: Failed to apply plugin [class 'com.hmhs.frameworks.syzygy.gradle.core.quality.plugins.SyzygyQualityPlugin']
        at org.gradle.api.internal.plugins.DefaultPluginManager.doApply(DefaultPluginManager.java:150)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addImperativePlugin(DefaultPluginManager.java:80)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addImperativePlugin(DefaultPluginManager.java:86)
        at org.gradle.api.internal.plugins.DefaultPluginContainer.apply(DefaultPluginContainer.java:92)
        at com.hmhs.frameworks.syzygy.gradle.lib.ProjectTaskCreaterMethods.addPluginLocal(ProjectTaskCreaterMethods.groovy:58)
        at com.hmhs.frameworks.syzygy.gradle.core.plugins.SyzygyCorePlugin.apply(SyzygyCorePlugin.groovy:60)
        at org.gradle.api.internal.plugins.ImperativeOnlyPluginTarget.applyImperative(ImperativeOnlyPluginTarget.java:42)
        at org.gradle.api.internal.plugins.RuleBasedPluginTarget.applyImperative(RuleBasedPluginTarget.java:50)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addPlugin(DefaultPluginManager.java:164)
        at org.gradle.api.internal.plugins.DefaultPluginManager.access$200(DefaultPluginManager.java:47)
        at org.gradle.api.internal.plugins.DefaultPluginManager$AddPluginBuildOperation.run(DefaultPluginManager.java:252)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.api.internal.plugins.DefaultPluginManager.doApply(DefaultPluginManager.java:144)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addImperativePlugin(DefaultPluginManager.java:80)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addImperativePlugin(DefaultPluginManager.java:86)
        at org.gradle.api.internal.plugins.DefaultPluginContainer.apply(DefaultPluginContainer.java:92)
        at com.hmhs.frameworks.syzygy.gradle.lib.ProjectTaskCreaterMethods.addPluginLocal(ProjectTaskCreaterMethods.groovy:58)
        at com.hmhs.frameworks.syzygy.gradle.js.plugins.SyzygyJavascriptPlugin.apply(SyzygyJavascriptPlugin.groovy:49)
        at org.gradle.api.internal.plugins.ImperativeOnlyPluginTarget.applyImperative(ImperativeOnlyPluginTarget.java:42)
        at org.gradle.api.internal.plugins.RuleBasedPluginTarget.applyImperative(RuleBasedPluginTarget.java:50)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addPlugin(DefaultPluginManager.java:164)
        at org.gradle.api.internal.plugins.DefaultPluginManager.access$200(DefaultPluginManager.java:47)
        at org.gradle.api.internal.plugins.DefaultPluginManager$AddPluginBuildOperation.run(DefaultPluginManager.java:252)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.api.internal.plugins.DefaultPluginManager.doApply(DefaultPluginManager.java:144)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addImperativePlugin(DefaultPluginManager.java:80)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addImperativePlugin(DefaultPluginManager.java:86)
        at org.gradle.api.internal.plugins.DefaultPluginContainer.apply(DefaultPluginContainer.java:92)
        ... 1 more

        Caused by:
        org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration$ArtifactResolveException: Could not resolve all files for configuration ':detachedConfiguration1'.
            at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.rethrowFailure(DefaultConfiguration.java:995)
            at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.access$1700(DefaultConfiguration.java:121)
            at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$ConfigurationFileCollection.getFiles(DefaultConfiguration.java:969)
            at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.getFiles(DefaultConfiguration.java:428)
            at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.resolve(DefaultConfiguration.java:424)
            at com.diffplug.gradle.spotless.GradleProvisioner.lambda$fromProject$1(GradleProvisioner.java:42)
            at com.diffplug.spotless.Provisioner.provisionWithDependencies(Provisioner.java:39)
            at com.diffplug.spotless.JarState.from(JarState.java:62)
            at com.diffplug.spotless.extra.groovy.GrEclipseFormatterStep.lambda$create$0(GrEclipseFormatterStep.java:49)
            at com.diffplug.spotless.FormatterStepImpl.calculateState(FormatterStepImpl.java:56)
            at com.diffplug.spotless.LazyForwardingEquality.state(LazyForwardingEquality.java:56)
            at com.diffplug.spotless.LazyForwardingEquality.toBytes(LazyForwardingEquality.java:85)
            at com.diffplug.spotless.LazyForwardingEquality.equals(LazyForwardingEquality.java:94)
            at java.util.ArrayList.indexOf(ArrayList.java:321)
            at com.diffplug.gradle.spotless.FormatExtension.replaceStep(FormatExtension.java:235)
            at com.diffplug.gradle.spotless.GroovyExtension$GrEclipseConfig.configFile(GroovyExtension.java:112)
            at com.hmhs.frameworks.syzygy.gradle.core.quality.plugins.SyzygyQualityPlugin$_apply_closure1.closure4$_closure11(SyzygyQualityPlugin.groovy:73)
            at groovy.lang.Closure.call(Closure.java:414)
            at groovy.lang.Closure.call(Closure.java:430)
            at org.gradle.api.internal.ClosureBackedAction.execute(ClosureBackedAction.java:71)
            at org.gradle.util.ConfigureUtil.configureTarget(ConfigureUtil.java:155)
            at org.gradle.util.ConfigureUtil.configure(ConfigureUtil.java:106)
            at org.gradle.util.ConfigureUtil$WrappedConfigureAction.execute(ConfigureUtil.java:167)
            at com.diffplug.gradle.spotless.SpotlessExtension.configure(SpotlessExtension.java:156)
            at com.diffplug.gradle.spotless.SpotlessExtension.groovy(SpotlessExtension.java:108)
            at org.gradle.internal.metaobject.BeanDynamicObject$MetaClassAdapter.invokeMethod(BeanDynamicObject.java:479)
            at org.gradle.internal.metaobject.BeanDynamicObject.tryInvokeMethod(BeanDynamicObject.java:191)
            at org.gradle.internal.metaobject.CompositeDynamicObject.tryInvokeMethod(CompositeDynamicObject.java:98)
            at org.gradle.internal.metaobject.MixInClosurePropertiesAsMethodsDynamicObject.tryInvokeMethod(MixInClosurePropertiesAsMethodsDynamicObject.java:30)
            at org.gradle.internal.metaobject.ConfigureDelegate.invokeMethod(ConfigureDelegate.java:57)
            at com.hmhs.frameworks.syzygy.gradle.core.quality.plugins.SyzygyQualityPlugin.apply_closure1$_closure4(SyzygyQualityPlugin.groovy:68)
            at groovy.lang.Closure.call(Closure.java:414)
            at groovy.lang.Closure.call(Closure.java:430)
            at org.gradle.api.internal.ClosureBackedAction.execute(ClosureBackedAction.java:71)
            at org.gradle.util.ConfigureUtil.configureTarget(ConfigureUtil.java:155)
            at org.gradle.util.ConfigureUtil.configure(ConfigureUtil.java:106)
            at org.gradle.util.ConfigureUtil$WrappedConfigureAction.execute(ConfigureUtil.java:167)
            at org.gradle.api.internal.plugins.ExtensionsStorage$ExtensionHolder.configure(ExtensionsStorage.java:183)
            at org.gradle.api.internal.plugins.ExtensionsStorage.configureExtension(ExtensionsStorage.java:67)
            at org.gradle.api.internal.plugins.DefaultConvention.configureExtension(DefaultConvention.java:399)
            at org.gradle.api.internal.plugins.DefaultConvention.access$500(DefaultConvention.java:45)
            at org.gradle.api.internal.plugins.DefaultConvention$ExtensionsDynamicObject.tryInvokeMethod(DefaultConvention.java:336)
            at org.gradle.internal.metaobject.CompositeDynamicObject.tryInvokeMethod(CompositeDynamicObject.java:98)
            at org.gradle.internal.metaobject.MixInClosurePropertiesAsMethodsDynamicObject.tryInvokeMethod(MixInClosurePropertiesAsMethodsDynamicObject.java:30)
            at org.gradle.internal.metaobject.AbstractDynamicObject.invokeMethod(AbstractDynamicObject.java:160)
            at com.hmhs.frameworks.syzygy.gradle.core.quality.plugins.SyzygyQualityPlugin.apply_closure1(SyzygyQualityPlugin.groovy:63)
            at groovy.lang.Closure.call(Closure.java:414)
            at groovy.lang.Closure.call(Closure.java:430)
            at com.hmhs.frameworks.syzygy.gradle.core.quality.plugins.SyzygyQualityPlugin.apply(SyzygyQualityPlugin.groovy:40)
            at org.gradle.api.internal.plugins.ImperativeOnlyPluginTarget.applyImperative(ImperativeOnlyPluginTarget.java:42)
            at org.gradle.api.internal.plugins.RuleBasedPluginTarget.applyImperative(RuleBasedPluginTarget.java:50)
            at org.gradle.api.internal.plugins.DefaultPluginManager.addPlugin(DefaultPluginManager.java:164)
            at org.gradle.api.internal.plugins.DefaultPluginManager.access$200(DefaultPluginManager.java:47)
            at org.gradle.api.internal.plugins.DefaultPluginManager$AddPluginBuildOperation.run(DefaultPluginManager.java:252)
            at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
            at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
            at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
            at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
            at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
            at org.gradle.api.internal.plugins.DefaultPluginManager.doApply(DefaultPluginManager.java:144)
            ... 36 more

            Caused by:
            org.gradle.internal.resolve.ModuleVersionNotFoundException: Cannot resolve external dependency com.diffplug.spotless:spotless-ext-greclipse:2.3.0 because no repositories are defined.
            Required by:
                project :
                at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.NoRepositoriesResolver.resolve(NoRepositoriesResolver.java:69)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.ComponentResolversChain$DependencyToComponentIdResolverChain.resolve(ComponentResolversChain.java:150)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.SelectorState.resolve(SelectorState.java:125)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.performSelection(DependencyGraphBuilder.java:224)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.performSelectionSerially(DependencyGraphBuilder.java:205)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.resolveEdges(DependencyGraphBuilder.java:194)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.traverseGraph(DependencyGraphBuilder.java:149)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.resolve(DependencyGraphBuilder.java:118)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.DefaultArtifactDependencyResolver.resolve(DefaultArtifactDependencyResolver.java:92)
                at org.gradle.api.internal.artifacts.ivyservice.DefaultConfigurationResolver.resolveGraph(DefaultConfigurationResolver.java:150)
                at org.gradle.api.internal.artifacts.ivyservice.ShortCircuitEmptyConfigurationResolver.resolveGraph(ShortCircuitEmptyConfigurationResolver.java:76)
                at org.gradle.api.internal.artifacts.ivyservice.ErrorHandlingConfigurationResolver.resolveGraph(ErrorHandlingConfigurationResolver.java:66)
                at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$4.run(DefaultConfiguration.java:507)
                at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
                at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
                at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
                at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
                at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
                at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.resolveGraphIfRequired(DefaultConfiguration.java:498)
                at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.resolveToStateOrLater(DefaultConfiguration.java:483)
                at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.access$1800(DefaultConfiguration.java:121)
                at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$ConfigurationFileCollection.getSelectedArtifacts(DefaultConfiguration.java:978)
                at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$ConfigurationFileCollection.getFiles(DefaultConfiguration.java:966)
                ... 93 more

If you're just submitting a feature request or question, no need for the above.

@jbduncan
Copy link
Member

jbduncan commented Jun 21, 2018

Good question! I've never fully understood this myself.

@scphantm For your information, what the dynamic dependency download is doing is it's trying to download the formatter library used by greclipse to do its Groovy code formatting.

Currently, behind-the-scenes, this dynamic downloading is achieved with a single detached configuration per dependency. I wonder if each of them could be replaced with a named configuration e.g. "spotlessGreclipse", without forcing each dependency to be downloaded earlier than strictly necessary.

@nedtwigg Thoughts?

@nedtwigg
Copy link
Member

it looks like your system doesn't pick those repos

Are those repos project repos or buildscript repos? Spotless downloads from the buildscript repos, which can cause some confusion. The history and consequences of this started in #99.

I wonder if each of them could be replaced with a named configuration e.g. "spotlessGreclipse"

They definitely could, but it'd be pretty big change, and probably require a major-version bump and should be discussed in a new issue, iff somebody wants to take that on. I think that approach has some sizable downsides w.r.t. to documentation and portability.

@scphantm
Copy link
Author

oh the master init file has both buildscript and normal dependency blocks. im just kinda confused why its not picking it up in the gradletest script. but ive never seen anyone do a dependency like you guys did, so im still a little confused. i will tinker with it today

@scphantm
Copy link
Author

i see whats going on now. gradleTest has a lot in it to increase performance of the tests. For example, my plugin has over 300 gradletests in it. So to increase performance, the gradleTest runtime goes out and downloads the dependencies into the .gradle cache at the beginning of the run, then it rewrites a new init.gradle file that it injects into the test run. this init.gradle file eliminates all the repositories and sets everything to file from the cache, that way you don't waste time with the tests searching mvncentral 300 times.

What that does for this plugin is since this plugin uses a rather non-standard way of downloading dependencies, since gradletest nukes the repos, it can't pull down the secondary dependencies you are looking for.

Pretty sure this is going to take a while to fix. I am probably going to have to put a patch into gradletest so that i can add repositories to that init file. But that will be a hard sell.

@nedtwigg
Copy link
Member

Spotless works with many formatters that each require conflicting versions of various dependencies. By provisioning jars for each formatter individually, and running them in their own classloader, we can support many versions of many formatters all in one plugin. We're not doing it just to be difficult :)

If you'd like to submit a PR to improve Spotless to use named configurations rather than improve gradleTest, we'd be happy to merge. The improvement to spotless would be a fair amount of work, but if gradletest is hard to patch then maybe it'd be worth your time.

@scphantm
Copy link
Author

Oh I get it. One design change I made blew up for trying to load groovy 2.3, gradle runs 2.14. Since it works in its own thing, gradle wasn’t able to resolve the dependency with resolution rules. I get why you did it. Just excercising my right to not be happy about it. ;)

I’m a core dev for gradletest and know that project well. I can knock that change out fairly quickly over the weekend. And I don’t have to learn another code base.

You should check out gradletest. It allows you to write gradle test kit tests as basic build files. It’s pretty awesome.

https://gitlab.com/ysb33rOrg/gradleTest

@scphantm
Copy link
Author

scphantm commented Jul 3, 2018

in case anyone is wondering, i made the changes to gradletest to do this, we are working on building it now. but i had setup spotless to be disabled when the CI server runs the tests. What happened is with this thing dynamically downloading files, each time, it added over an hour to our unit test run and was blowing up circuit breakers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants