Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@
import org.apache.polaris.core.persistence.dao.entity.PrincipalSecretsResult;
import org.apache.polaris.core.storage.PolarisStorageIntegrationProvider;
import org.apache.polaris.core.storage.cache.StorageCredentialCache;
import org.apache.polaris.core.utils.CachedSupplier;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand Down Expand Up @@ -97,12 +98,16 @@ private void initializeForRealm(
DatasourceOperations databaseOperations = getDatasourceOperations(isBootstrap);
sessionSupplierMap.put(
realmContext.getRealmIdentifier(),
() ->
new JdbcBasePersistenceImpl(
databaseOperations,
secretsGenerator(realmContext, rootCredentialsSet),
storageIntegrationProvider,
realmContext.getRealmIdentifier()));
new CachedSupplier<>(
() ->
new JdbcBasePersistenceImpl(
databaseOperations,
secretsGenerator(realmContext, rootCredentialsSet),
storageIntegrationProvider,
realmContext.getRealmIdentifier())));

// Ensure the supplier caches the first time
var unused = sessionSupplierMap.get(realmContext.getRealmIdentifier()).get();

PolarisMetaStoreManager metaStoreManager = createNewMetaStoreManager();
metaStoreManagerMap.put(realmContext.getRealmIdentifier(), metaStoreManager);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@
import org.apache.polaris.core.persistence.transactional.TransactionalMetaStoreManagerImpl;
import org.apache.polaris.core.persistence.transactional.TransactionalPersistence;
import org.apache.polaris.core.storage.cache.StorageCredentialCache;
import org.apache.polaris.core.utils.CachedSupplier;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand Down Expand Up @@ -100,7 +101,12 @@ private void initializeForRealm(
final StoreType backingStore = createBackingStore(diagnostics);
sessionSupplierMap.put(
realmContext.getRealmIdentifier(),
() -> createMetaStoreSession(backingStore, realmContext, rootCredentialsSet, diagnostics));
new CachedSupplier<>(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this change conforms with the behavior intended by the sessionSupplierMap.
Currently, each call to the supplier yields a new instance - this PR updates the behavior to provide a lazily initialized Persistence instance per realm-ID.
Maybe @collado-mike or @dennishuo could chime in here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point. This makes me wonder - would my original idea of materializing the RealmContext prior to the creation of the Supplier also become an issue? For instance, would it be possible that the RealmContext is also be computed lazily in some instances?

Let me follow up with @collado-mike and @dennishuo on this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, each instance of the session was intended to be scoped to a single request. Though, it seems now that all the current implementations are stateless, but the TransactionalPersistence interface methods kind of imply a stateful implementation - e.g., lookupEntityInCurrentTxn assumes that there is a current transaction that has already been started and will be committed at some point.

Copy link
Contributor Author

@adnanhemani adnanhemani Jun 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, so the way I see it is we have really 2 options to fix this issue:

  1. Keep this approach and change the semantics for the TransactionalPersistence interface to be stateless (either now or in the future). OR
  2. Take the approach to materialize the Realm ID and create a new realmContext to pass into the supplier that breaks the dependency on the realmContext that originally came from the function signature. While this is a less invasive change, I do not have an easy way to test this behavior.

I'm leaning towards option 2 simply because it is less invasive - but is everyone else okay without there being hard testing for this few-line change? It would look something like this:

private void initializeForRealm(
      RealmContext realmContext, RootCredentialsSet rootCredentialsSet) {
    final StoreType backingStore = createBackingStore(diagnostics);
    String materializedRealmId = realmContext.getRealmIdentifier();
    RealmContext materializedRealmContext = () -> materializedRealmId;
    sessionSupplierMap.put(
        realmContext.getRealmIdentifier(),
        () -> createMetaStoreSession(backingStore, materializedRealmContext, rootCredentialsSet, diagnostics));

    PolarisMetaStoreManager metaStoreManager = createNewMetaStoreManager();
    metaStoreManagerMap.put(realmContext.getRealmIdentifier(), metaStoreManager);
  }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good question and it's not as straightforward as it might seem. The RealmContext interface only defines a getName method, but there are concrete implementations that may contain extra information about the realm (we have our own custom impl). Simply materializing the RealmContext in this way could break functionality if the underlying Session/MetaStoreManager depend on the concrete implementation.

I think the proper long-term fix is to make the BasePersistence itself a CDI-managed bean so that the RealmContext can be injected by the context rather than us materializing it manually. It also means we have to make the task execution framework CDI-managed, which is a bigger task that we've been putting off for a while

() ->
createMetaStoreSession(
backingStore, realmContext, rootCredentialsSet, diagnostics)));
// Ensure the supplier caches the first time
var unused = sessionSupplierMap.get(realmContext.getRealmIdentifier()).get();

PolarisMetaStoreManager metaStoreManager = createNewMetaStoreManager();
metaStoreManagerMap.put(realmContext.getRealmIdentifier(), metaStoreManager);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

package org.apache.polaris.core.utils;

import java.util.function.Supplier;

public class CachedSupplier<T> implements Supplier<T> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC this functionality already exists in Guava.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL - I believe you're talking about Suppliers#memoize. Thanks! Will convert to using this if we continue with this approach!

private final Supplier<T> delegate;
private T value;
private boolean initialized = false;

public CachedSupplier(Supplier<T> delegate) {
this.delegate = delegate;
}

@Override
public synchronized T get() {
if (!initialized) {
value = delegate.get();
initialized = true;
}
return value;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

package org.apache.polaris.core.utils;

import java.util.function.Supplier;
import org.apache.polaris.core.context.RealmContext;
import org.assertj.core.api.Assertions;
import org.junit.jupiter.api.Test;

public class CachedSupplierTest {
private String realmName = "test";
private int timesCalled = 0;
private final RealmContext realmContext =
new RealmContext() {
@Override
public String getRealmIdentifier() {
if (++timesCalled == 1) {
return realmName;
}
throw new IllegalStateException();
}
};

private static class ContainerRealmIdentifier {
private String realmIdentifier;

public ContainerRealmIdentifier(RealmContext realmContext) {
this.realmIdentifier = realmContext.getRealmIdentifier();
}

public String getRealmIdentifier() {
return realmIdentifier;
}
}

@Test
public void testCachedSupplier() {
Supplier<ContainerRealmIdentifier> realmIdentifierSupplier =
() -> new ContainerRealmIdentifier(realmContext);
Assertions.assertThat(realmName.equals(realmIdentifierSupplier.get().getRealmIdentifier()))
.isTrue(); // This will work
Assertions.assertThatThrownBy(() -> realmIdentifierSupplier.get().getRealmIdentifier())
.isInstanceOf(IllegalStateException.class);

timesCalled = 0;
CachedSupplier<ContainerRealmIdentifier> cachedSupplier =
new CachedSupplier<>(() -> new ContainerRealmIdentifier(realmContext));
Assertions.assertThat(realmName.equals(cachedSupplier.get().getRealmIdentifier()))
.isTrue(); // This will work
Assertions.assertThat(realmName.equals(cachedSupplier.get().getRealmIdentifier()))
.isTrue(); // This will work
}
}
Loading