Skip to content

Configure Phoenix server scan page timeout #24689

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 15, 2025
Merged

Configure Phoenix server scan page timeout #24689

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions docs/src/main/sphinx/connector/phoenix.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,12 @@ to load custom Phoenix client connection properties.

The following Phoenix-specific configuration properties are available:

| Property name | Required | Description |
| ----------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `phoenix.connection-url` | Yes | `jdbc:phoenix[:zk_quorum][:zk_port][:zk_hbase_path]`. The `zk_quorum` is a comma separated list of ZooKeeper servers. The `zk_port` is the ZooKeeper port. The `zk_hbase_path` is the HBase root znode path, that is configurable using `hbase-site.xml`. By default the location is `/hbase` |
| `phoenix.config.resources` | No | Comma-separated list of configuration files (e.g. `hbase-site.xml`) to use for connection properties. These files must exist on the machines running Trino. |
| `phoenix.max-scans-per-split` | No | Maximum number of HBase scans that will be performed in a single split. Default is 20. Lower values will lead to more splits in Trino. Can also be set via session propery `max_scans_per_split`. For details see: [https://phoenix.apache.org/update_statistics.html](https://phoenix.apache.org/update_statistics.html). (This setting has no effect when guideposts are disabled in Phoenix.) |
| Property name | Required | Description |
|------------------------------------| -------- |--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `phoenix.connection-url` | Yes | `jdbc:phoenix[:zk_quorum][:zk_port][:zk_hbase_path]`. The `zk_quorum` is a comma separated list of ZooKeeper servers. The `zk_port` is the ZooKeeper port. The `zk_hbase_path` is the HBase root znode path, that is configurable using `hbase-site.xml`. By default the location is `/hbase` |
| `phoenix.config.resources` | No | Comma-separated list of configuration files (e.g. `hbase-site.xml`) to use for connection properties. These files must exist on the machines running Trino. |
| `phoenix.max-scans-per-split` | No | Maximum number of HBase scans that will be performed in a single split. Default is 20. Lower values will lead to more splits in Trino. Can also be set via session propery `max_scans_per_split`. For details see: [https://phoenix.apache.org/update_statistics.html](https://phoenix.apache.org/update_statistics.html). (This setting has no effect when guideposts are disabled in Phoenix.) |
| `phoenix.server-scan-page-timeout` | No | The time limit on the amount of work single RPC request can do before it times out. Type: [](prop-type-duration). |

```{include} jdbc-common-configurations.fragment
```
Expand Down
5 changes: 5 additions & 0 deletions plugin/trino-phoenix5/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,11 @@
<artifactId>log</artifactId>
</dependency>

<dependency>
<groupId>io.airlift</groupId>
<artifactId>units</artifactId>
</dependency>

<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-base-jdbc</artifactId>
Expand Down
2 changes: 2 additions & 0 deletions plugin/trino-phoenix5/src/main/java/io/trino/plugin/phoenix5/PhoenixClientModule.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@
import static io.trino.plugin.phoenix5.PhoenixClient.DEFAULT_DOMAIN_COMPACTION_THRESHOLD;
import static io.trino.plugin.phoenix5.PhoenixErrorCode.PHOENIX_CONFIG_ERROR;
import static java.util.Objects.requireNonNull;
import static org.apache.phoenix.query.QueryServices.PHOENIX_SERVER_PAGE_SIZE_MS;
import static org.apache.phoenix.util.ReadOnlyProps.EMPTY_PROPS;
import static org.weakref.jmx.guice.ExportBinder.newExporter;

Expand Down Expand Up @@ -209,6 +210,7 @@ public static Properties getConnectionProperties(PhoenixConfig config)

ConnectionInfo connectionInfo = ConnectionInfo.create(config.getConnectionUrl(), EMPTY_PROPS, new Properties());
connectionInfo.asProps().asMap().forEach(connectionProperties::setProperty);
config.getServerScanPageTimeout().ifPresent(duration -> connectionProperties.setProperty(PHOENIX_SERVER_PAGE_SIZE_MS, String.valueOf(duration.toMillis())));
return connectionProperties;
}

Expand Down
20 changes: 20 additions & 0 deletions plugin/trino-phoenix5/src/main/java/io/trino/plugin/phoenix5/PhoenixConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,13 @@
import io.airlift.configuration.Config;
import io.airlift.configuration.ConfigDescription;
import io.airlift.configuration.validation.FileExists;
import io.airlift.units.Duration;
import jakarta.validation.constraints.Max;
import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.NotNull;

import java.util.List;
import java.util.Optional;

public class PhoenixConfig
{
Expand All @@ -41,6 +43,11 @@ public class PhoenixConfig
private int maxScansPerSplit = 20;
private boolean reuseConnection = true;

/**
* By default, let Phoenix client derive Page size from HBase RPC timeout configs.
*/
private Optional<Duration> serverScanPageTimeout = Optional.empty();

@NotNull
public String getConnectionUrl()
{
Expand Down Expand Up @@ -94,4 +101,17 @@ public PhoenixConfig setReuseConnection(boolean reuseConnection)
this.reuseConnection = reuseConnection;
return this;
}

public Optional<Duration> getServerScanPageTimeout()
{
return serverScanPageTimeout;
}

@Config("phoenix.server-scan-page-timeout")
@ConfigDescription("Phoenix scan page timeout to reflect the time limit on the amount of work single Scan RPC request can do")
public PhoenixConfig setServerScanPageTimeout(Duration serverScanPageTimeout)
{
this.serverScanPageTimeout = Optional.ofNullable(serverScanPageTimeout);
return this;
}
}
7 changes: 6 additions & 1 deletion plugin/trino-phoenix5/src/test/java/io/trino/plugin/phoenix5/TestPhoenixConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@

import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import io.airlift.units.Duration;
import org.junit.jupiter.api.Test;

import java.io.IOException;
Expand All @@ -25,6 +26,7 @@
import static io.airlift.configuration.testing.ConfigAssertions.assertFullMapping;
import static io.airlift.configuration.testing.ConfigAssertions.assertRecordedDefaults;
import static io.airlift.configuration.testing.ConfigAssertions.recordDefaults;
import static java.util.concurrent.TimeUnit.HOURS;

public class TestPhoenixConfig
{
Expand All @@ -35,7 +37,8 @@ public void testDefaults()
.setConnectionUrl(null)
.setResourceConfigFiles(ImmutableList.of())
.setMaxScansPerSplit(20)
.setReuseConnection(true));
.setReuseConnection(true)
.setServerScanPageTimeout(null));
}

@Test
Expand All @@ -49,12 +52,14 @@ public void testExplicitPropertyMappings()
.put("phoenix.config.resources", configFile.toString())
.put("phoenix.max-scans-per-split", "1")
.put("query.reuse-connection", "false")
.put("phoenix.server-scan-page-timeout", "11h")
.buildOrThrow();

PhoenixConfig expected = new PhoenixConfig()
.setConnectionUrl("jdbc:phoenix:localhost:2181:/hbase")
.setResourceConfigFiles(ImmutableList.of(configFile.toString()))
.setMaxScansPerSplit(1)
.setServerScanPageTimeout(new Duration(11, HOURS))
.setReuseConnection(false);

assertFullMapping(properties, expected);
Expand Down
Loading