Error: Java heap space

[NassirFfx]

Comparing the screenshot step is failing with an error: Java heap space
Further, look into gives:

• When org.gradle.jvmargs=-Xmx1024m is set , the java heap error was consistent
• When increased org.gradle.jvmargs=-Xmx4608m is set , the frequency of error has dropped down to almost 20%.

Comparing screenshots with previous ones.
[02:14:07]: ▸ > Task :sample:phoneDebugExecuteScreenshotTests FAILED
[02:14:07]: ▸ FAILURE: Build failed with an exception.
[02:14:07]: ▸ * What went wrong:
[02:14:07]: ▸ Execution failed for task ':sample:phoneDebugExecuteScreenshotTests'.
[02:14:07]: ▸ > Java heap space
[02:14:07]: ▸ * Try:
[02:14:07]: ▸ Run with --info or --debug option to get more log output. Run with --scan to get full insights.
[02:14:07]: ▸ * Exception is:
[02:14:07]: ▸ org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':sample:phoneDebugExecuteScreenshotTests'.
[02:14:07]: ▸ at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:207)
[02:14:07]: ▸ at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:263)
[02:14:07]: ▸ at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:205)
[02:14:07]: ▸ at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:186)
[02:14:07]: ▸ at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:114)
[02:14:07]: ▸ at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
[02:14:07]: ▸ at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:62)
[02:14:07]: ▸ at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
[02:14:07]: ▸ at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
[02:14:07]: ▸ at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
[02:14:07]: ▸ at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
[02:14:07]: ▸ at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
[02:14:07]: ▸ at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
[02:14:07]: ▸ at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:409)
[02:14:07]: ▸ at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:399)
[02:14:07]: ▸ at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:157)
[02:14:07]: ▸ at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:242)
[02:14:07]: ▸ at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:150)
[02:14:07]: ▸ at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:94)
[02:14:07]: ▸ at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
[02:14:07]: ▸ at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
[02:14:07]: ▸ at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:41)
[02:14:07]: ▸ at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:356)
[02:14:07]: ▸ at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:343)
[02:14:07]: ▸ at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:336)
[02:14:07]: ▸ at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:322)
[02:14:07]: ▸ at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127)
[02:14:07]: ▸ at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191)
[02:14:07]: ▸ at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182)
[02:14:07]: ▸ at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
[02:14:07]: ▸ at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
[02:14:07]: ▸ at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
[02:14:07]: ▸ at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
[02:14:07]: ▸ Caused by: java.lang.OutOfMemoryError: Java heap space
[02:14:07]: ▸ at scala.collection.mutable.ResizableArray.ensureSize(ResizableArray.scala:106)
[02:14:07]: ▸ at scala.collection.mutable.ResizableArray.ensureSize$(ResizableArray.scala:96)
[02:14:07]: ▸ at scala.collection.mutable.ArrayBuffer.ensureSize(ArrayBuffer.scala:49)
[02:14:07]: ▸ at scala.collection.mutable.ArrayBuffer.$plus$eq(ArrayBuffer.scala:85)
[02:14:07]: ▸ at scala.collection.mutable.ArrayBuffer.$plus$eq(ArrayBuffer.scala:49)
[02:14:07]: ▸ at scala.collection.generic.Growable.$anonfun$$plus$plus$eq$1(Growable.scala:62)
[02:14:07]: ▸ at scala.collection.generic.Growable$$Lambda$2154/0x0000000801378040.apply(Unknown Source)
[02:14:07]: ▸ at scala.collection.Iterator.foreach(Iterator.scala:943)
[02:14:07]: ▸ at scala.collection.Iterator.foreach$(Iterator.scala:943)
[02:14:07]: ▸ at com.sksamuel.scrimage.AwtImage$$anon$1.foreach(AwtImage.scala:52)
[02:14:07]: ▸ at scala.collection.generic.Growable.$plus$plus$eq(Growable.scala:62)
[02:14:07]: ▸ at scala.collection.generic.Growable.$plus$plus$eq$(Growable.scala:53)
[02:14:07]: ▸ at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:105)
[02:14:07]: ▸ at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:49)
[02:14:07]: ▸ at scala.collection.TraversableOnce.to(TraversableOnce.scala:366)
[02:14:07]: ▸ at scala.collection.TraversableOnce.to$(TraversableOnce.scala:364)
[02:14:07]: ▸ at com.sksamuel.scrimage.AwtImage$$anon$1.to(AwtImage.scala:52)
[02:14:07]: ▸ at scala.collection.TraversableOnce.toBuffer(TraversableOnce.scala:358)
[02:14:07]: ▸ at scala.collection.TraversableOnce.toBuffer$(TraversableOnce.scala:358)
[02:14:07]: ▸ at com.sksamuel.scrimage.AwtImage$$anon$1.toBuffer(AwtImage.scala:52)
[02:14:07]: ▸ at scala.collection.TraversableOnce.toArray(TraversableOnce.scala:345)
[02:14:07]: ▸ at scala.collection.TraversableOnce.toArray$(TraversableOnce.scala:339)
[02:14:07]: ▸ at com.sksamuel.scrimage.AwtImage$$anon$1.toArray(AwtImage.scala:52)
[02:14:07]: ▸ at com.sksamuel.scrimage.AwtImage.pixels(AwtImage.scala:43)
[02:14:07]: ▸ at com.karumi.shot.screenshots.ScreenshotsComparator.imagesAreDifferent(ScreenshotsComparator.scala:51)
[02:14:07]: ▸ at com.karumi.shot.screenshots.ScreenshotsComparator.compareScreenshot(ScreenshotsComparator.scala:33)
[02:14:07]: ▸ at com.karumi.shot.screenshots.ScreenshotsComparator.$anonfun$compare$1(ScreenshotsComparator.scala:13)
[02:14:07]: ▸ at com.karumi.shot.screenshots.ScreenshotsComparator$$Lambda$2254/0x000000080141a840.apply(Unknown Source)
[02:14:07]: ▸ at scala.collection.parallel.AugmentedIterableIterator.flatmap2combiner(RemainsIterator.scala:133)
[02:14:07]: ▸ at scala.collection.parallel.AugmentedIterableIterator.flatmap2combiner$(RemainsIterator.scala:130)
[02:14:07]: ▸ at scala.collection.parallel.immutable.ParVector$ParVectorIterator.flatmap2combiner(ParVector.scala:66)
[02:14:07]: ▸ at scala.collection.parallel.ParIterableLike$FlatMap.leaf(ParIterableLike.scala:1074)
[02:14:07]: ▸ * Get more help at https://help.gradle.org/
[02:14:07]: ▸ Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
[02:14:07]: ▸ Use '--warning-mode all' to show the individual deprecation warnings.
[02:14:07]: ▸ See https://docs.gradle.org/6.5/userguide/command_line_interface.html#sec:command_line_warnings
[02:14:07]: ▸ BUILD FAILED in 14m 14s
[02:14:07]: ▸ 79 actionable tasks: 79 executed

Snapshot tests on Phone

Version: 5.13.0

[pedrovgs]

🤔 @NassirFfx the sample projects we have are not able to reproduce this error. can you please upload a sample project we can use to reproduce this bug? I'm also wondering if this is related to the number of images you have or the size of some images.

[bartsg]

I'm having the same issue. It seems to happen only when a certain amount of differences are detected (e.g. 250+ changed screenshots or so).

[adesentenac]

I'm also reproducing the issue with Shot 5.14.1.
I have 22 screenshots which have a resolution of 1080x2400. If I run them on a different device with a tolerance of 1.0, so they all pass with a warning, I have an OOM exception if Xmx is set at 2G, but pass if I change it to 4G.

[karsie]

I get this when I have a significant number of differences. (And I know tolerance is applied after gathering differences). From looking at the code of the image comparison library, the AWT images are never flushed. Maybe adding this will help?

[karsie]

For me PR #314 helps

[mhiew]

I think the issue is due to parallelism when comparing images and not really a garbage collection issue. (While debugging locally I could see that the total used memory was properly being recycled during execution)

screenshots.par.flatMap(compareScreenshot(_, tolerance)).toList

I am not too familiar with scala, but I believe converting the collection to a parallel collection with par will by default set the number of threads to Runtime.AvailableProcesses. I think this matches the number of available cores on your machine.

When there are a lot of image differences the imagesAreDifferent function can potentially take up quite a bit of memory due to allocating an array of pixels for each set of screenshots to compare against.

These two factors can lead to random out of memory issues depending on which set of screenshots are being processed at the same time. If we have 4 threads then we are loading 4 sets of images at the same time. The sets of images that get processed per run is somewhat random as par doesn't guarantee ordering. So my hunch is the inconsistent failures happen due to bad luck of processing a set of multiple images that doesn't fit into memory.

---

I tried to set the maximum concurrency using -Dscala.concurrent.context.maxThreads=1 but unfortunately that didn't seem to work for me.

---

To test I cloned the project and locally tested setting a maximum thread pool size which seems to have alleviated my issues.

//set the maximum number of threads to 2
private val taskSupport = new ForkJoinTaskSupport(new ForkJoinPool(2))

def compare(screenshots: ScreenshotsSuite, tolerance: Double): ScreenshotsComparisionResult = {
  val screenshotsParallel = screenshots.par
  screenshotsParallel.tasksupport = taskSupport

  val errors =
    screenshotsParallel.flatMap(compareScreenshot(_, tolerance)).toList
  ScreenshotsComparisionResult(errors, screenshots)
}

Perhaps one solution would be add an optional max threads to the plugin extension so that we can control the parallelism on memory constrained machines (CI executors)?

I don't think this will ultimately solve all issues as even with 2 threads we could run into a situation where the set of images does not fit into memory. However, being able to control it or even set it to 1 max thread would likely reduce the likelihood of this issue happening.

---

Alternatively, I tried removing parallelism altogether and it fixed my issue as well. Anecdotally the comparison time for my project (1.5k screenshots) took about 6 seconds instead of 2 seconds. This isn't too bad for my use case so I may go with this option as the bulk of the time is spent generating the comparison screenshots.

screenshots.flatMap(compareScreenshot(_, tolerance)).toList

[AlmightyCZ]

For me PR #314 helps

I noticed that you closed it due to it not being considered a "complete solution." Could you please clarify what aspects are still missing?

Currently, I'm encountering "heap space" and "java.lang.IllegalArgumentException: Self-suppression not permitted" exceptions after transitioning to a higher Android version on my testing AVDs. I had to set tolerance = 0.01 due to variations in a few pixels and I usually get one of the exceptions for test runs with 200+ shots. (shorter test are fine) I've already set org.gradle.jvmargs=-Xmx8192M long time ago.

[karsie]

I only managed to fix all my issues by converting the entire thing to Kotlin (mainly due to the lack of Scala knowledge on my part). What I also encountered why converting it, is that there is 1 action where all recorded screenshots are copied using Scrimage (and thus loading every file into an awt image), where this could easily be achieved using regular file copy.