Support user-specified paths for conf-file in the eclipse-link metastore manager

[eric-maynard]

Description

Previously only resources bundled in the JAR could be used as the EclipseLink configuration file. This change adds support for arbitrary locations specified as the conf-file without breaking support for files included in the JAR.

It will first check for a file included in the JAR matching the path, and it will fall back to trying to load the file directly if that fails.

Fixes #45

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Confirmed I'm able to load a persistence.xml file at a new location with the change, where previously it gave:

Cannot find or parse the configuration file...

Checklist:

Please delete options that are not relevant.

• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[MonkeyCanCode]

Same updated on #45.

Here is the setup I have:

metaStoreManager:
  type: eclipse-link
  conf-file: /home/xxxx/polaris/persistence-local.xml
  persistence-unit: polaris-dev

Content of persistence-local.xml (same as default one but on diff path and diff file name):

<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">

  <persistence-unit name="polaris-dev" transaction-type="RESOURCE_LOCAL">
    <provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
    <class>io.polaris.core.persistence.models.ModelEntity</class>
    <class>io.polaris.core.persistence.models.ModelEntityActive</class>
    <class>io.polaris.core.persistence.models.ModelEntityChangeTracking</class>
    <class>io.polaris.core.persistence.models.ModelEntityDropped</class>
    <class>io.polaris.core.persistence.models.ModelGrantRecord</class>
    <class>io.polaris.core.persistence.models.ModelPrincipalSecrets</class>
    <class>io.polaris.core.persistence.models.ModelSequenceId</class>
    <shared-cache-mode>NONE</shared-cache-mode>
    <properties>
      <property name="jakarta.persistence.jdbc.url"
        value="jdbc:h2:file:/tmp/eclipseLink/{realm}"/>
      <property name="jakarta.persistence.jdbc.user" value="sa"/>
      <property name="jakarta.persistence.jdbc.password" value=""/>
      <property name="jakarta.persistence.schema-generation.database.action" value="create"/>
      <property name="eclipselink.logging.level.sql" value="FINE"/>
      <property name="eclipselink.logging.parameters" value="true"/>
      <property name="eclipselink.persistence-context.flush-mode" value="auto"/>
    </properties>
  </persistence-unit>
</persistence>

With the provided code, it will be able to find the file and create the properties obj with the provided settings, however, it will later on failed on

emf = Persistence.createEntityManagerFactory(persistenceUnitName, properties);

Here is the trace:

INFO  [2024-08-02 19:29:02,230 - 1096  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load io.polaris.extension.persistence.impl.hibernate.HibernatePolarisMetaStoreManagerFactory
INFO  [2024-08-02 19:29:02,231 - 1097  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load com.snowflake.polaris.persistence.impl.remote.RemotePolarisMetaStoreManagerFactory
INFO  [2024-08-02 19:29:02,231 - 1097  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load io.polaris.extension.persistence.impl.hibernate.HibernatePolarisMetaStoreManagerFactory
INFO  [2024-08-02 19:29:02,231 - 1097  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load com.snowflake.polaris.persistence.impl.remote.RemotePolarisMetaStoreManagerFactory
INFO  [2024-08-02 19:29:02,232 - 1098  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load io.polaris.extension.persistence.impl.hibernate.HibernatePolarisMetaStoreManagerFactory
INFO  [2024-08-02 19:29:02,232 - 1098  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load com.snowflake.polaris.persistence.impl.remote.RemotePolarisMetaStoreManagerFactory
INFO  [2024-08-02 19:29:02,232 - 1098  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load io.polaris.extension.persistence.impl.hibernate.HibernatePolarisMetaStoreManagerFactory
INFO  [2024-08-02 19:29:02,232 - 1098  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load com.snowflake.polaris.persistence.impl.remote.RemotePolarisMetaStoreManagerFactory
WARN  [2024-08-02 19:29:02,267 - 1133  ] [main] [] i.p.e.p.i.e.PolarisEclipseLinkMetaStoreSessionImpl: Cannot find or parse the configuration file /home/xxxx/polaris/persistence-local.xml for persistence-unit polaris-dev
jakarta.persistence.PersistenceException: No Persistence provider for EntityManager named polaris-dev
        at jakarta.persistence.Persistence.createEntityManagerFactory(Persistence.java:86)
        at io.polaris.extension.persistence.impl.eclipselink.PolarisEclipseLinkMetaStoreSessionImpl.<init>(PolarisEclipseLinkMetaStoreSessionImpl.java:111)
        at io.polaris.extension.persistence.impl.eclipselink.EclipseLinkPolarisMetaStoreManagerFactory.createMetaStoreSession(EclipseLinkPolarisMetaStoreManagerFactory.java:48)
        at io.polaris.extension.persistence.impl.eclipselink.EclipseLinkPolarisMetaStoreManagerFactory.createMetaStoreSession(EclipseLinkPolarisMetaStoreManagerFactory.java:32)
        at io.polaris.core.persistence.LocalPolarisMetaStoreManagerFactory.lambda$initializeForRealm$0(LocalPolarisMetaStoreManagerFactory.java:68)
        at io.polaris.core.persistence.LocalPolarisMetaStoreManagerFactory.bootstrapServiceAndCreatePolarisPrincipalForRealm(LocalPolarisMetaStoreManagerFactory.java:151)
        at io.polaris.core.persistence.LocalPolarisMetaStoreManagerFactory.bootstrapRealms(LocalPolarisMetaStoreManagerFactory.java:84)
        at io.polaris.service.BootstrapRealmsCommand.run(BootstrapRealmsCommand.java:58)
        at io.polaris.service.BootstrapRealmsCommand.run(BootstrapRealmsCommand.java:33)
        at io.dropwizard.core.cli.ConfiguredCommand.run(ConfiguredCommand.java:98)
        at io.dropwizard.core.cli.Cli.run(Cli.java:78)
        at io.dropwizard.core.Application.run(Application.java:94)
        at io.polaris.service.PolarisApplication.main(PolarisApplication.java:112)

I am not a Java expert and I am assuming this has something to do with the used java framework?

Here is the basic debugging I did couple days back after verified the provided file path is getting loaded but failing on the next part:

Dummy debugging code to print out content of properties:

    properties.put(ECLIPSELINK_PERSISTENCE_XML, confFile);
    System.out.println("=====");
    System.out.println(properties);
    System.out.println("=====");
    emf = Persistence.createEntityManagerFactory(persistenceUnitName, properties);

Trigger bootstrap to read the config and try to configure the backend:

 java -jar polaris-service/build/libs/polaris-service-1.0.0-all.jar bootstrap polaris-server.yml

Here is the full trace as above but with the extra dump of content of properties variable:

INFO  [2024-08-02 19:38:46,367 - 1018  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load io.polaris.extension.persistence.impl.hibernate.HibernatePolarisMetaStoreManagerFactory
INFO  [2024-08-02 19:38:46,367 - 1018  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load com.snowflake.polaris.persistence.impl.remote.RemotePolarisMetaStoreManagerFactory
INFO  [2024-08-02 19:38:46,367 - 1018  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load io.polaris.extension.persistence.impl.hibernate.HibernatePolarisMetaStoreManagerFactory
INFO  [2024-08-02 19:38:46,367 - 1018  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load com.snowflake.polaris.persistence.impl.remote.RemotePolarisMetaStoreManagerFactory
INFO  [2024-08-02 19:38:46,368 - 1019  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load io.polaris.extension.persistence.impl.hibernate.HibernatePolarisMetaStoreManagerFactory
INFO  [2024-08-02 19:38:46,368 - 1019  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load com.snowflake.polaris.persistence.impl.remote.RemotePolarisMetaStoreManagerFactory
INFO  [2024-08-02 19:38:46,368 - 1019  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load io.polaris.extension.persistence.impl.hibernate.HibernatePolarisMetaStoreManagerFactory
INFO  [2024-08-02 19:38:46,368 - 1019  ] [main] [] i.d.j.DiscoverableSubtypeResolver: Unable to load com.snowflake.polaris.persistence.impl.remote.RemotePolarisMetaStoreManagerFactory
=====
{jakarta.persistence.jdbc.url=jdbc:h2:file:/tmp/eclipseLink/default-realm, jakarta.persistence.jdbc.user=sa, eclipselink.logging.parameters=true, eclipselink.logging.level.sql=FINE, jakarta.persistence.schema-generation.database.action=create, eclipselink.persistence-context.flush-mode=auto, jakarta.persistence.jdbc.password=, eclipselink.persistencexml=/home/xxxx/polaris/persistence-local.xml}
=====
jakarta.persistence.PersistenceException: No Persistence provider for EntityManager named polaris-dev
        at jakarta.persistence.Persistence.createEntityManagerFactory(Persistence.java:86)
        at io.polaris.extension.persistence.impl.eclipselink.PolarisEclipseLinkMetaStoreSessionImpl.<init>(PolarisEclipseLinkMetaStoreSessionImpl.java:115)
        at io.polaris.extension.persistence.impl.eclipselink.EclipseLinkPolarisMetaStoreManagerFactory.createMetaStoreSession(EclipseLinkPolarisMetaStoreManagerFactory.java:48)
        at io.polaris.extension.persistence.impl.eclipselink.EclipseLinkPolarisMetaStoreManagerFactory.createMetaStoreSession(EclipseLinkPolarisMetaStoreManagerFactory.java:32)
        at io.polaris.core.persistence.LocalPolarisMetaStoreManagerFactory.lambda$initializeForRealm$0(LocalPolarisMetaStoreManagerFactory.java:68)
        at io.polaris.core.persistence.LocalPolarisMetaStoreManagerFactory.bootstrapServiceAndCreatePolarisPrincipalForRealm(LocalPolarisMetaStoreManagerFactory.java:151)
        at io.polaris.core.persistence.LocalPolarisMetaStoreManagerFactory.bootstrapRealms(LocalPolarisMetaStoreManagerFactory.java:84)
        at io.polaris.service.BootstrapRealmsCommand.run(BootstrapRealmsCommand.java:58)
        at io.polaris.service.BootstrapRealmsCommand.run(BootstrapRealmsCommand.java:33)
        at io.dropwizard.core.cli.ConfiguredCommand.run(ConfiguredCommand.java:98)
        at io.dropwizard.core.cli.Cli.run(Cli.java:78)
        at io.dropwizard.core.Application.run(Application.java:94)
        at io.polaris.service.PolarisApplication.main(PolarisApplication.java:112)

[eric-maynard]

I see, thanks @MonkeyCanCode -- let me close this for now and try to reproduce

[sfc-gh-aixu]

@eric-maynard and @MonkeyCanCode Let me take a look if some changes got lost or it's the limitation of Persistence.createEntityManagerFactory().

[sfc-gh-aixu]

Seems that's the limitation from Persistence.createEntityManagerFactory() which uses Enumeration<URL> resources = loader.getResources(descriptorPath) to load the resources. @eric-maynard is investigating a solution for this.

[guitcastro]

@sfcs-gh-aixu I was able to fix the issue by removing properties.put(ECLIPSELINK_PERSISTENCE_XML, confFile);. This line was causing createEntityManagerFactory to try loading the config file from resources. Since the config properties are already being set manually, it's safe to remove it. Here's the relevant code:

      for (int i = 0; i < nodeList.getLength(); i++) {
        NamedNodeMap nodeMap = nodeList.item(i).getAttributes();
        properties.put(
            nodeMap.getNamedItem("name").getNodeValue(),
            nodeMap.getNamedItem("value").getNodeValue());
      }

[aihuaxu]

@guitcastro Actually it's not because of the properties. Of course we need the properties because we want to replace realm in the database name - each realm will have its own database.

Persistence.createEntityManagerFactory() will still load configuration file from resource even without properties. Let's follow up on #95.