[#8891] feat(lance-rest-server): Add standalone and auxiliary mode support for Lance REST service#8903
Merged
jerryshao merged 5 commits intoapache:branch-lance-namepspace-devfrom Oct 24, 2025
Conversation
Contributor
|
LGTM. Could you plz resolve the conflicts? thx |
e78757f to
ca98524
Compare
f6d49d9
into
apache:branch-lance-namepspace-dev
49 of 50 checks passed
jerryshao
pushed a commit
to jerryshao/gravitino
that referenced
this pull request
Nov 11, 2025
…ode support for Lance REST service (apache#8903) ### What changes were proposed in this pull request? 1. Created service discovery file (META-INF/services/org.apache.gravitino.auxiliary.GravitinoAuxiliaryService) 2. Added unit tests (TestLanceConfig.java) ### Why are the changes needed? Fix: apache#8891 To enable flexible deployment of Lance REST service: Standalone mode (independent JVM): - Lance needs to connect to Gravitino server via HTTP to access catalog/schema/table metadata - GRAVITINO_URI and GRAVITINO_METALAKE configs specify the connection parameters - Use case: Deploy Lance REST separately for isolation or scaling Auxiliary mode (embedded in Gravitino server): - Lance runs in the same JVM as Gravitino, enabling direct memory access (zero overhead) - META-INF/services file enables automatic service discovery via Java ServiceLoader - Use case: Simplified deployment with shared resources and unified monitoring ### Does this PR introduce _any_ user-facing change? Yes - New configuration properties For Standalone Mode: ``` # Required: Gravitino server URI gravitino.lance-rest.gravitino-uri=http://gravitino-server:8090 # Required: Metalake name gravitino.lance-rest.gravitino-metalake=production ``` For Auxiliary Mode: ``` # Enable Lance REST as auxiliary service gravitino.auxService.names=iceberg-rest,lance-rest # Classpath for Lance REST JAR gravitino.lance-rest.classpath=lance/lance-rest-server/build/libs ``` Deployment modes supported: Standalone: Lance REST runs independently, connects to Gravitino via HTTP Auxiliary: Lance REST runs embedded in Gravitino, uses direct memory access No breaking changes - all changes are additive and backward compatible. ### How was this patch tested? 1. unit test 2. manual test **Standalone mode test:** ``` junda@java(junda.devpod-us-or) ~/gravitino/distribution/package/bin % ./gravitino.sh start Gravitino Server is running[PID:3907790] junda@java(junda.devpod-us-or) ~ % curl http://localhost:8090/api/version {"code":0,"version":{"version":"1.1.0-SNAPSHOT","compileDate":"23/10/2025 21:03:17","gitCommit":"35bd819975badea2493b4dbd9d0de68425229945"}}% junda@java(junda.devpod-us-or) ~/gravitino/distribution/package/bin % ./gravitino-lance-rest-server.sh start GravitinoLanceRESTServer is running[PID:3930096] junda@java(junda.devpod-us-or) ~ % curl "http://localhost:9101/lance/v1/namespace/root/list" {"id":"root","delimiter":"$","namespaces":["default"]}% ``` **Auxiliary mode test:** ``` junda@java(junda.devpod-us-or) ~/gravitino/distribution/package % cd /home/user/gravitino && cat >> distribution/package/conf/gravitino.conf << 'EOF' # ===== Lance REST Auxiliary Service ===== gravitino.auxService.names = iceberg-rest,lance-rest gravitino.lance-rest.classpath = /home/user/gravitino/lance/lance-rest-server/build/libs gravitino.lance-rest.catalog-name = lance_catalog EOF junda@java(junda.devpod-us-or) ~/gravitino/distribution/package % ./bin/gravitino.sh start Gravitino Server is running[PID:274396] junda@java(junda.devpod-us-or) ~/gravitino/distribution/package % tail -50 logs/gravitino-server.log | grep -i lance | grep -i started 2025-10-23 21:34:41.495 INFO [main] [org.apache.gravitino.server.web.JettyServer.start(JettyServer.java:202)] - lance-rest web server started on host 0.0.0.0 port 9101. 2025-10-23 21:34:41.495 INFO [main] [org.apache.gravitino.lance.LanceRESTService.serviceStart(LanceRESTService.java:77)] - Lance REST service started ```
youngyjd
added a commit
to youngyjd/gravitino
that referenced
this pull request
Nov 18, 2025
…ode support for Lance REST service (apache#8903) ### What changes were proposed in this pull request? 1. Created service discovery file (META-INF/services/org.apache.gravitino.auxiliary.GravitinoAuxiliaryService) 2. Added unit tests (TestLanceConfig.java) ### Why are the changes needed? Fix: apache#8891 To enable flexible deployment of Lance REST service: Standalone mode (independent JVM): - Lance needs to connect to Gravitino server via HTTP to access catalog/schema/table metadata - GRAVITINO_URI and GRAVITINO_METALAKE configs specify the connection parameters - Use case: Deploy Lance REST separately for isolation or scaling Auxiliary mode (embedded in Gravitino server): - Lance runs in the same JVM as Gravitino, enabling direct memory access (zero overhead) - META-INF/services file enables automatic service discovery via Java ServiceLoader - Use case: Simplified deployment with shared resources and unified monitoring ### Does this PR introduce _any_ user-facing change? Yes - New configuration properties For Standalone Mode: ``` # Required: Gravitino server URI gravitino.lance-rest.gravitino-uri=http://gravitino-server:8090 # Required: Metalake name gravitino.lance-rest.gravitino-metalake=production ``` For Auxiliary Mode: ``` # Enable Lance REST as auxiliary service gravitino.auxService.names=iceberg-rest,lance-rest # Classpath for Lance REST JAR gravitino.lance-rest.classpath=lance/lance-rest-server/build/libs ``` Deployment modes supported: Standalone: Lance REST runs independently, connects to Gravitino via HTTP Auxiliary: Lance REST runs embedded in Gravitino, uses direct memory access No breaking changes - all changes are additive and backward compatible. ### How was this patch tested? 1. unit test 2. manual test **Standalone mode test:** ``` junda@java(junda.devpod-us-or) ~/gravitino/distribution/package/bin % ./gravitino.sh start Gravitino Server is running[PID:3907790] junda@java(junda.devpod-us-or) ~ % curl http://localhost:8090/api/version {"code":0,"version":{"version":"1.1.0-SNAPSHOT","compileDate":"23/10/2025 21:03:17","gitCommit":"35bd819975badea2493b4dbd9d0de68425229945"}}% junda@java(junda.devpod-us-or) ~/gravitino/distribution/package/bin % ./gravitino-lance-rest-server.sh start GravitinoLanceRESTServer is running[PID:3930096] junda@java(junda.devpod-us-or) ~ % curl "http://localhost:9101/lance/v1/namespace/root/list" {"id":"root","delimiter":"$","namespaces":["default"]}% ``` **Auxiliary mode test:** ``` junda@java(junda.devpod-us-or) ~/gravitino/distribution/package % cd /home/user/gravitino && cat >> distribution/package/conf/gravitino.conf << 'EOF' # ===== Lance REST Auxiliary Service ===== gravitino.auxService.names = iceberg-rest,lance-rest gravitino.lance-rest.classpath = /home/user/gravitino/lance/lance-rest-server/build/libs gravitino.lance-rest.catalog-name = lance_catalog EOF junda@java(junda.devpod-us-or) ~/gravitino/distribution/package % ./bin/gravitino.sh start Gravitino Server is running[PID:274396] junda@java(junda.devpod-us-or) ~/gravitino/distribution/package % tail -50 logs/gravitino-server.log | grep -i lance | grep -i started 2025-10-23 21:34:41.495 INFO [main] [org.apache.gravitino.server.web.JettyServer.start(JettyServer.java:202)] - lance-rest web server started on host 0.0.0.0 port 9101. 2025-10-23 21:34:41.495 INFO [main] [org.apache.gravitino.lance.LanceRESTService.serviceStart(LanceRESTService.java:77)] - Lance REST service started ```
youngyjd
added a commit
to youngyjd/gravitino
that referenced
this pull request
Nov 19, 2025
…ode support for Lance REST service (apache#8903) ### What changes were proposed in this pull request? 1. Created service discovery file (META-INF/services/org.apache.gravitino.auxiliary.GravitinoAuxiliaryService) 2. Added unit tests (TestLanceConfig.java) ### Why are the changes needed? Fix: apache#8891 To enable flexible deployment of Lance REST service: Standalone mode (independent JVM): - Lance needs to connect to Gravitino server via HTTP to access catalog/schema/table metadata - GRAVITINO_URI and GRAVITINO_METALAKE configs specify the connection parameters - Use case: Deploy Lance REST separately for isolation or scaling Auxiliary mode (embedded in Gravitino server): - Lance runs in the same JVM as Gravitino, enabling direct memory access (zero overhead) - META-INF/services file enables automatic service discovery via Java ServiceLoader - Use case: Simplified deployment with shared resources and unified monitoring ### Does this PR introduce _any_ user-facing change? Yes - New configuration properties For Standalone Mode: ``` # Required: Gravitino server URI gravitino.lance-rest.gravitino-uri=http://gravitino-server:8090 # Required: Metalake name gravitino.lance-rest.gravitino-metalake=production ``` For Auxiliary Mode: ``` # Enable Lance REST as auxiliary service gravitino.auxService.names=iceberg-rest,lance-rest # Classpath for Lance REST JAR gravitino.lance-rest.classpath=lance/lance-rest-server/build/libs ``` Deployment modes supported: Standalone: Lance REST runs independently, connects to Gravitino via HTTP Auxiliary: Lance REST runs embedded in Gravitino, uses direct memory access No breaking changes - all changes are additive and backward compatible. ### How was this patch tested? 1. unit test 2. manual test **Standalone mode test:** ``` junda@java(junda.devpod-us-or) ~/gravitino/distribution/package/bin % ./gravitino.sh start Gravitino Server is running[PID:3907790] junda@java(junda.devpod-us-or) ~ % curl http://localhost:8090/api/version {"code":0,"version":{"version":"1.1.0-SNAPSHOT","compileDate":"23/10/2025 21:03:17","gitCommit":"35bd819975badea2493b4dbd9d0de68425229945"}}% junda@java(junda.devpod-us-or) ~/gravitino/distribution/package/bin % ./gravitino-lance-rest-server.sh start GravitinoLanceRESTServer is running[PID:3930096] junda@java(junda.devpod-us-or) ~ % curl "http://localhost:9101/lance/v1/namespace/root/list" {"id":"root","delimiter":"$","namespaces":["default"]}% ``` **Auxiliary mode test:** ``` junda@java(junda.devpod-us-or) ~/gravitino/distribution/package % cd /home/user/gravitino && cat >> distribution/package/conf/gravitino.conf << 'EOF' # ===== Lance REST Auxiliary Service ===== gravitino.auxService.names = iceberg-rest,lance-rest gravitino.lance-rest.classpath = /home/user/gravitino/lance/lance-rest-server/build/libs gravitino.lance-rest.catalog-name = lance_catalog EOF junda@java(junda.devpod-us-or) ~/gravitino/distribution/package % ./bin/gravitino.sh start Gravitino Server is running[PID:274396] junda@java(junda.devpod-us-or) ~/gravitino/distribution/package % tail -50 logs/gravitino-server.log | grep -i lance | grep -i started 2025-10-23 21:34:41.495 INFO [main] [org.apache.gravitino.server.web.JettyServer.start(JettyServer.java:202)] - lance-rest web server started on host 0.0.0.0 port 9101. 2025-10-23 21:34:41.495 INFO [main] [org.apache.gravitino.lance.LanceRESTService.serviceStart(LanceRESTService.java:77)] - Lance REST service started ```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Why are the changes needed?
Fix: #8891
To enable flexible deployment of Lance REST service:
Standalone mode (independent JVM):
Auxiliary mode (embedded in Gravitino server):
Does this PR introduce any user-facing change?
Yes - New configuration properties
For Standalone Mode:
For Auxiliary Mode:
Deployment modes supported:
Standalone: Lance REST runs independently, connects to Gravitino via HTTP
Auxiliary: Lance REST runs embedded in Gravitino, uses direct memory access
No breaking changes - all changes are additive and backward compatible.
How was this patch tested?
Standalone mode test:
Auxiliary mode test: