Set cache dir to user's home directory #2862

hangshao0 · 2018-09-13T19:01:04Z

Set cache dir to user's home directory if groupAccess does not
presents on Java 11
Add options bootClassesOnly and fatal
Set J9SHR_RUNTIMEFLAG_ENABLE_CACHE_NON_BOOT_CLASSES by default,
remove this flag if option bootClassesOnly presents
Change the second parameter of j9shmem_getDir().
Change the first parameter of getCacheDir().

Doc change eclipse-openj9/openj9-docs#103.

Signed-off-by: hangshao [email protected]

pshipton · 2018-09-13T21:56:52Z

For consistency, I expect the shared cache location should always default to the home directory. If we can agree on this, we should also discuss a migration strategy since existing VMs use the temp directory.

hangshao0 · 2018-09-14T14:47:45Z

I expect the shared cache location should always default to the home directory.

The home directory might not be accessible to other users. This might break users using "groupAccess". Probably we can use home directory if "groupAccess" is not in use.

hangshao0 · 2018-09-21T14:59:45Z

@pshipton, Do you agree to set cache directory to user's home only when "groupAccess" is not in use ?

pshipton · 2018-09-21T17:52:39Z

Do you agree to set cache directory to user's home only when "groupAccess" is not in use ?

Yes, for compatibility it seems necessary.

I think what we need is a list of scenarios and what will occur in each. This will help ensure everything is covered properly, and the documentation is updated appropriately.

Here is a starter list.

what happens when there is an existing cache in /tmp and you get a new JVM which puts the cache in the home directory? How do caches in /tmp get cleaned up?
do bootstrap caches and caches created via -Xshareclasses use the same file name?
how do you delete a bootstrap cache?
what happens if one JVM creates and uses a bootstrap cache, and another JVM enables -Xshareclasses using the default name?

hangshao0 · 2018-09-21T18:04:00Z

what happens when there is an existing cache in /tmp and you get a new JVM which puts the cache in the home directory? How do caches in /tmp get cleaned up?

Then the new JVM will use the cache in the home directory. Users can always delete the caches in /tmp using "cacheDir=/tmp,destroy".

do bootstrap caches and caches created via -Xshareclasses use the same file name?

Currently yes.

how do you delete a bootstrap cache?

We can use -Xshareclasses:destroy.

what happens if one JVM creates and uses a bootstrap cache, and another JVM enables -Xshareclasses using the default name?

They will share the same cache.

pshipton · 2018-09-21T18:11:22Z

They will share the same cache.

Will JVMs which haven't enabled -Xshareclasses load non-bootstrap classes from the cache?

what is the default size of a bootstrap cache?

pshipton · 2018-09-21T18:27:35Z

how do you change the default location of a bootstrap cache, or apply other options
seems nonfatal or the equivalent should automatically be in effect for bootstrap caches, how does this interact with the above
how do you turn off the bootstrap cache
I assume changing the location of the cache affects the location of snapshots

hangshao0 · 2018-09-21T19:28:47Z

Will JVMs which haven't enabled -Xshareclasses load non-bootstrap classes from the cache?

No

what is the default size of a bootstrap cache?

It is the same as the default cache size. 300MB with 64softmx on 64-bit platforms and 16MB on 32 bit platforms.

how do you change the default location of a bootstrap cache, or apply other options

By changing the setting of user home, or environment variable "HOME". (There is another way to do this , by using -Xshareclasses:CacheDir=/dir/,bootClassesOnly.)

how do you turn off the bootstrap cache

By using -Xshareclasses:none

seems nonfatal or the equivalent should automatically be in effect for bootstrap caches, how does this interact with the above

The default class sharing is equivalent to -Xshareclasses:cacheDir=userhome,nonfatal,bootClassesOnly. Nonfatal can be turn on by using -Xshareclasses:cacheDir=userhome,bootClassesOnly.

I assume changing the location of the cache affects the location of snapshots

Yes.

There will be a trace point if the default shared cache failed to start up. And the size of the default shared cache can be set by -XX:SharedCacheHardLimit= and -Xscmx

hangshao0 · 2018-09-21T19:35:17Z

Discussed with Peter, I will try on some real applications with -Xshareclasses:bootClassesOnly to see if the bootstrap shared cache gradually grows in size.

pshipton · 2018-09-21T19:36:25Z

I suggest bootClassesOnly implies nonfatal. A new "fatal" option (although I don't like this name) can be added so -Xshareclasses:bootClassesOnly is equivalent to the default, and -Xshareclasses:bootClassesOnly,fatal can disable it if desired.

pshipton · 2018-09-21T19:38:30Z

Another point we discussed

what happens if the shared cache cannot be created in the home directory, due to it being on a network drive, or home doesn't exist, or there isn't enough disk space.

Default bootClassesOnly is disabled.

hangshao0 · 2018-10-01T20:53:37Z

The change is ready to be reviewed again.

pshipton · 2018-10-03T21:07:55Z

Please open the doc issue, this change cannot be merged without the documentation being ready to go as well.

runtime/port/sysvipc/j9shmem.c

pshipton · 2018-10-03T21:26:16Z

runtime/port/sysvipc/j9shmem.c

+			Assert_PRT_true(TRUE == appendBaseDir);
+			uintptr_t baseDirLen = strlen(J9SH_BASEDIR);
+			/* From https://linux.die.net/man/3/getpwuid:
+			 * "An application that wants to determine its user's home directory should inspect the value of HOME (rather than the value getpwuid(getuid())->pw_dir)


Please don't copy the text from the Linux man pages. You can paraphrase.

hangshao0 · 2018-10-04T14:51:27Z

OpenJ9 doc issue created: eclipse-openj9/openj9-docs#103.

We never documented the debug options from -Xshareclasses:morehelp, I guess we do not need to document -Xshareclasses:bootClassesOnly and -Xshareclasses:fatal either ? (Though "bootClassesOnly" and "fatal" are in the doc change request for now)

pshipton · 2018-10-04T15:53:54Z

bootClassesOnly and fatal should be first class options shown via -Xshareclasses:help

1. Set cache dir to user's home directory if groupAccess is not presents on Java 11 2. Add a new debug option bootClassesOnly 3. Set J9SHR_RUNTIMEFLAG_ENABLE_CACHE_NON_BOOT_CLASSES by default, remove this flag if option bootClassesOnly presents 4. Change the second parameter of j9shmem_getDir(). 5. Change the first parameter of getCacheDir(). Signed-off-by: hangshao <[email protected]>

hangshao0 · 2018-10-04T16:00:03Z

Moved bootClassesOnly and fatal to -Xshareclasses:help

pshipton · 2018-10-05T15:22:26Z

jenkins test all plinux jdk11

pshipton · 2018-10-05T15:39:48Z

jenkins test sanity plinux jdk11

pshipton · 2018-10-05T15:40:07Z

jenkins test extended plinux jdk11

pshipton · 2018-10-05T15:40:34Z

jenkins test sanity plinux jdk11

pshipton · 2018-10-05T15:49:05Z

jenkins compile win,zlinux jdk8

pshipton · 2018-10-05T16:15:56Z

jenkins compile aix jdk8

pshipton · 2018-10-05T19:13:06Z

The compile failures are a result of #3158 and can be ignored.

pshipton · 2018-10-05T19:27:25Z

@SueChaplain I've added a review on this for you so you can indicate when the doc changes are ready to go.

pshipton

Do not merge before doc changes are ready.

pshipton · 2018-10-05T19:32:51Z

jenkins compile aix,zlinux jdk8

pshipton · 2018-10-05T19:49:26Z

jenkins compile xlinux,plinux jdk8

pshipton · 2018-10-05T19:51:17Z

jenkins compile xlinux,plinux jdk8

pshipton · 2018-10-05T19:57:50Z

jenkins compile plinux jdk8

pshipton · 2018-10-05T20:05:52Z

jenkins compile plinux jdk8

pshipton · 2018-10-05T20:06:17Z

jenkins test sanity plinux jdk8

Doc changes can come later

hangshao0 force-pushed the sharedCache branch 3 times, most recently from 210b2f2 to 22ef22a Compare September 13, 2018 19:12

hangshao0 mentioned this pull request Sep 13, 2018

Enable bootstrap class sharing by default on OpenJ9 #2449

Merged

pshipton added the comp:vm label Sep 13, 2018

hangshao0 force-pushed the sharedCache branch from 22ef22a to c820dde Compare October 1, 2018 20:44

hangshao0 changed the title ~~Add a new debug option -Xshareclasses:bootClassesOnly~~ Set cache dir to user's home directory Oct 1, 2018

hangshao0 force-pushed the sharedCache branch from c820dde to 79676a5 Compare October 1, 2018 20:58

pshipton added the depends:doc label Oct 3, 2018

pshipton reviewed Oct 3, 2018

View reviewed changes

hangshao0 mentioned this pull request Oct 4, 2018

Change default cache directory to user's home eclipse-openj9/openj9-docs#103

Closed

hangshao0 force-pushed the sharedCache branch from 79676a5 to 857b660 Compare October 4, 2018 14:58

hangshao0 force-pushed the sharedCache branch from 857b660 to 411f19e Compare October 4, 2018 15:59

pshipton requested a review from SueChaplain October 5, 2018 19:26

pshipton previously requested changes Oct 5, 2018

View reviewed changes

pshipton approved these changes Oct 9, 2018

View reviewed changes

pshipton merged commit 9084442 into eclipse-openj9:master Oct 9, 2018

pshipton added the jdk11 label Oct 9, 2018

pshipton mentioned this pull request Nov 29, 2018

Set shared cache dir to user's home directory jdk8 #3882

Closed

scalpel4k mentioned this pull request Mar 20, 2021

use ~/.cache/ for caching purposes #12255

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set cache dir to user's home directory #2862

Set cache dir to user's home directory #2862

hangshao0 commented Sep 13, 2018 •

edited by pshipton

Loading

pshipton commented Sep 13, 2018

hangshao0 commented Sep 14, 2018

hangshao0 commented Sep 21, 2018

pshipton commented Sep 21, 2018

hangshao0 commented Sep 21, 2018

pshipton commented Sep 21, 2018

pshipton commented Sep 21, 2018 •

edited

Loading

hangshao0 commented Sep 21, 2018 •

edited

Loading

hangshao0 commented Sep 21, 2018

pshipton commented Sep 21, 2018

pshipton commented Sep 21, 2018

hangshao0 commented Oct 1, 2018

pshipton commented Oct 3, 2018

pshipton Oct 3, 2018

hangshao0 Oct 4, 2018

hangshao0 commented Oct 4, 2018

pshipton commented Oct 4, 2018

hangshao0 commented Oct 4, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton left a comment •

edited

Loading

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

Set cache dir to user's home directory #2862

Set cache dir to user's home directory #2862

Conversation

hangshao0 commented Sep 13, 2018 • edited by pshipton Loading

pshipton commented Sep 13, 2018

hangshao0 commented Sep 14, 2018

hangshao0 commented Sep 21, 2018

pshipton commented Sep 21, 2018

hangshao0 commented Sep 21, 2018

pshipton commented Sep 21, 2018

pshipton commented Sep 21, 2018 • edited Loading

hangshao0 commented Sep 21, 2018 • edited Loading

hangshao0 commented Sep 21, 2018

pshipton commented Sep 21, 2018

pshipton commented Sep 21, 2018

hangshao0 commented Oct 1, 2018

pshipton commented Oct 3, 2018

pshipton Oct 3, 2018

Choose a reason for hiding this comment

hangshao0 Oct 4, 2018

Choose a reason for hiding this comment

hangshao0 commented Oct 4, 2018

pshipton commented Oct 4, 2018

hangshao0 commented Oct 4, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton left a comment • edited Loading

Choose a reason for hiding this comment

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

hangshao0 commented Sep 13, 2018 •

edited by pshipton

Loading

pshipton commented Sep 21, 2018 •

edited

Loading

hangshao0 commented Sep 21, 2018 •

edited

Loading

pshipton left a comment •

edited

Loading