OOM on save/refactor

[dsyer]

Not sure how to trigger this. I'm getting UI hangs and eventual OOM on save/refactor changes. Uninstalling this feature "fixes" the issue.

Example stack trace:

java.lang.reflect.InvocationTargetException
    at org.eclipse.jface.operation.ModalContext.runInCurrentThread(ModalContext.java:449)
    at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:354)
    at org.eclipse.ui.internal.WorkbenchWindow.lambda$5(WorkbenchWindow.java:2364)
    at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:72)
    at org.eclipse.ui.internal.WorkbenchWindow.run(WorkbenchWindow.java:2362)
    at org.eclipse.jdt.internal.ui.refactoring.RefactoringExecutionHelper.perform(RefactoringExecutionHelper.java:199)
    at org.eclipse.jdt.internal.ui.refactoring.RefactoringExecutionHelper.perform(RefactoringExecutionHelper.java:156)
    at org.eclipse.jdt.ui.refactoring.RenameSupport.perform(RenameSupport.java:201)
    at org.eclipse.jdt.internal.ui.refactoring.reorg.RenameLinkedMode.doRename(RenameLinkedMode.java:371)
    at org.eclipse.jdt.internal.ui.refactoring.reorg.RenameLinkedMode$EditorSynchronizer.left(RenameLinkedMode.java:125)
    at org.eclipse.jface.text.link.LinkedModeModel.exit(LinkedModeModel.java:338)
    at org.eclipse.jface.text.link.LinkedModeUI.lambda$2(LinkedModeUI.java:1174)
    at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
    at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:185)
    at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:5113)
    at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4666)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$5.run(PartRenderingEngine.java:1173)
    at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1062)
    at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:155)
    at org.eclipse.ui.internal.Workbench.lambda$3(Workbench.java:635)
    at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
    at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:559)
    at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:150)
    at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:155)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:137)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:107)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:400)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:660)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:597)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1468)
    at org.eclipse.equinox.launcher.Main.main(Main.java:1441)
Caused by: java.lang.OutOfMemoryError: Java heap space

[wilkinsona]

Could you run with -XX:+HeapDumpOnOutOfMemoryError and share the resulting dump so that we can see what's consuming all the heap?

[dsyer]

Is it a file (the dump)? Any idea where the file is created? I only see the same as above in the Eclipse error view?

[wilkinsona]

Yeah, it should be a file written to the JVM's working directory. You can control the location use -XX:HeapDumpPath.

[dsyer]

With -XX:HeapDumpPath=/tmp I see these files:

$ find /tmp/hsperfdata_dsyer/
/tmp/hsperfdata_dsyer/
/tmp/hsperfdata_dsyer/21356
/tmp/hsperfdata_dsyer/21441

and 21356 is the PID of my Eclipse process. But none of them looks like a heap dump.

[wilkinsona]

As far as I know hsperfdata_<username>/<pid> is unrelated to dumping the heap on OOME and is created by default.

[dsyer]

Maybe Eclipse is catching the OOM error and the JVM never sees it?

[wilkinsona]

Sounds plausible (and annoying). Can you get anything with jmap or the like immediately after Eclipse barfs?

[martinlippert]

I tried to analyze this a bit more. It seems to be easy to reproduce:

• use latest STS4 on Eclipse 2019-09 (maybe this also works with latest Eclipse 2019-09 milestone builds)
• install latest java-formatter version (I got 0.0.15)
• go to Preferences -> Java -> Code Style -> Formatter
• switch the implementation to Spring
• the IDE immediately runs out of heap

Looking under the hood, I can see this allocating more than 250MB:

DefaultCodeFormatter.java:207 io.spring.javaformat.formatter.eclipse.DefaultCodeFormatter.tokenizeSource(int)

This is already when formatting the preview for the preferences dialog (so nothing seriously large). Maybe something changed in the internals of Eclipse that causes the java formatter to blow up.

Is there an easy way to setup a dev environment for this that would allow be to debug this? I can't even find this type in the GitHub repo, but maybe I am looking at the wrong place?

[wilkinsona]

Thanks for taking a look, @martinlippert.

DefaultCodeFormatter is repackaged from org.eclipse.jdt.internal.formatter.DefaultCodeFormatter. It builds against Eclipse 2019-03 at the moment. Perhaps there's a bug/change in Eclipse 2019-09 that's adversely affected the behaviour of tokenizeSource(int)?

The contributing guidelines contain some instruction on how to set up a dev environment using Eclipse.

[martinlippert]

There is indeed a change in the DefaultCodeFormatter in the way it creates the scanner: https://github.com/eclipse/eclipse.jdt.core/commit/5d58390bf2fb018a53c69e075d1416a113301107#diff-e4b25e2704c57f755dad0d2673d243c0

Haven't looked into the details yet though...

[BoykoAlex]

Upgrading to Eclipse 2019-09 (4.13) makes it work for STS based on that Eclipse version (and higher). However, Eclipse 2019-06 (4.12) doesn't work with the Spring formatter...
(The same is true if upgraded to 2019-06 (4.12). Then 4.13 won't work with IndexOutOfBoundsException during scanning)
Is that a problem?
If yes, any chance different versions of Spring Java Formatter can be provided for different Eclipse versions?
Otherwise, I can create a PR for the upgrade to 2019-09 + some Gradle Buildship API usage correction.

[magJ]

I am also getting a memory leak when running the gradle plugin.

The issue only presents itself with versions of org.eclipse.jdt:org.eclipse.jdt.core 3.18.0 or above.
Version 3.17.0 seems to work fine, but there is a lot of changes between the two versions: https://github.com/eclipse/eclipse.jdt.core/compare/c6d7ef991f723f8b37cf85f1d368979f6e5f0485...3d670fe48cd3ec2fe4d3f7a2fca5830a4057291b

Looking at the dominator tree of the heap dump via eclipse MAT shows:

Class Name                                                                         | Shallow Heap | Retained Heap | Percentage
-------------------------------------------------------------------------------------------------------------------------------
io.spring.javaformat.formatter.Formatter$DelegateCodeFormatter @ 0x7f71c7700       |           56 |   165,206,040 |     80.74%
|- java.util.ArrayList @ 0x7f62ea570                                               |           24 |   165,204,128 |     80.74%
|  '- java.lang.Object[2734845] @ 0x7ff200000                                      |   10,939,400 |   165,204,104 |     80.74%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c5dc8                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c5e08                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c5e48                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c5e88                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c5ec8                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c5f08                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c5f48                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c5f88                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c5fc8                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c6008                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c6048                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c6088                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c60c8                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c6108                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c6148                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c6188                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c61c8                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c6208                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c6248                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c6288                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c62c8                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c6308                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c6348                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c6388                |           64 |            64 |      0.00%
|     |- io.spring.javaformat.formatter.eclipse.Token @ 0x7f62c63c8                |           64 |            64 |      0.00%
|     '- Total: 25 of 2,410,386 entries; 2,410,361 more                            |              |               |           
|- io.spring.javaformat.formatter.eclipse.DefaultCodeFormatterOptions @ 0x7f62ea5e0|          640 |           736 |      0.00%
|- io.spring.javaformat.formatter.eclipse.DefaultCodeFormatterOptions @ 0x7f62eaa60|          640 |           736 |      0.00%
|- io.spring.javaformat.formatter.eclipse.TokenManager @ 0x7f62ea538               |           56 |           248 |      0.00%
|- java.util.ArrayList @ 0x7f62ea3c8                                               |           24 |           112 |      0.00%
|- java.util.Arrays$ArrayList @ 0x7f7374c90                                        |           24 |            24 |      0.00%
'- Total: 6 entries                                                                |              |               |           
-------------------------------------------------------------------------------------------------------------------------------

[wilkinsona]

@magJ Interesting, thank you. I suspect that this is a different issue as we believe that the problem reported here is due to an incompatibility between this project's Eclipse plugin (that is built against Eclipse 2019-03) and Eclipse 2019-09.

Could you please open a new issue for the memory leak with the Gradle plugin so that it doesn't get forgotten? It'd be much appreciated if you could attach a small sample project with some steps to reproduce it.