-
-
Notifications
You must be signed in to change notification settings - Fork 586
feat(nebulagraph): add NebulaGraph module #3266
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
4b9458c
feat(nebulagraph): add NebulaGraph module
egasimov a816a18
Resolve comments 1
egasimov cf567ad
Update document
egasimov 1ab1dc4
Remove static container name
egasimov 63e4df0
Resolve comments 2
egasimov b5ccd1a
docs: refine
mdelapenya 0758d24
docs: add nebulagraph to mkdocs nav entry
mdelapenya 508bdf0
chore: fix lint (auto)
mdelapenya f186557
chore: append errors to the same slice
mdelapenya b04544b
chore: handle state error
mdelapenya a86e71d
chore: format errors
mdelapenya 1d80c05
chore: do not hide container package
mdelapenya e2c19c5
chore: handle IPv6
mdelapenya 94ad525
chore: simplify
mdelapenya 12371e9
fix: remove activator on error
mdelapenya 033b5ce
Fix tests 1
egasimov fb1baf5
Fix tests 2
egasimov File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,111 @@ | ||
| # NebulaGraph | ||
|
|
||
| Not available until the next release <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||
|
|
||
| ## Introduction | ||
|
|
||
| The Testcontainers module for [NebulaGraph](https://nebula-graph.io/), a distributed, scalable, and lightning-fast graph database. This module manages a complete NebulaGraph cluster including Meta Service, Storage Service, and Graph Service components. | ||
|
|
||
| ## Adding this module to your project dependencies | ||
|
|
||
| Add the NebulaGraph module to your Go dependencies: | ||
|
|
||
| ```go | ||
| go get github.com/testcontainers/testcontainers-go/modules/nebulagraph | ||
| ``` | ||
|
|
||
| ## Usage example | ||
|
|
||
| <!--codeinclude--> | ||
| [Creating a NebulaGraph container](../../modules/nebulagraph/nebulagraph_test.go) inside_block:TestNebulaGraphContainer | ||
| <!--/codeinclude--> | ||
|
|
||
| ## Module Reference | ||
|
|
||
| ### RunCluster function | ||
|
|
||
| - Not available until the next release <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||
|
|
||
| The NebulaGraph module provides a function to create a complete NebulaGraph cluster within a Docker network: | ||
|
|
||
| ```golang | ||
| func RunCluster(ctx context.Context, | ||
| graphdImg string, graphdCustomizers []testcontainers.ContainerCustomizer, | ||
| storagedImg string, storagedCustomizers []testcontainers.ContainerCustomizer, | ||
| metadImg string, metadCustomizers []testcontainers.ContainerCustomizer, | ||
| ) (*Cluster, error) | ||
| ``` | ||
|
|
||
| This function creates a complete NebulaGraph cluster with customizable settings. It returns a `Cluster` struct that contains references to all four components: | ||
| - Meta Service (metad) | ||
| - Storage Service (storaged) | ||
| - Graph Service (graphd) | ||
|
|
||
| ### Default Configuration | ||
|
|
||
| The module uses the following default configurations: | ||
|
|
||
| #### Default Images | ||
| - Graph Service: `vesoft/nebula-graphd:v3.8.0` | ||
| - Meta Service: `vesoft/nebula-metad:v3.8.0` | ||
| - Storage Service: `vesoft/nebula-storaged:v3.8.0` | ||
|
|
||
| #### Exposed Ports | ||
| - Graph Service: 9669 (TCP), 19669 (HTTP) | ||
| - Meta Service: 9559 (TCP), 19559 (HTTP) | ||
| - Storage Service: 9779 (TCP), 19779 (HTTP) | ||
|
|
||
| #### Health Checks | ||
|
|
||
| The module implements health checks for all services: | ||
|
|
||
| - Meta Service: HTTP health check on `/status` endpoint (port 19559) | ||
| - Graph Service: HTTP health check on `/status` endpoint (port 19669) | ||
| - Storage Service: Log-based health check for initialization | ||
| - Activator Service: Log-based health check and exit status for storage registration | ||
|
|
||
| A cluster is considered ready when: | ||
|
|
||
| 1. Meta service is healthy and accessible | ||
| 2. Graph service is healthy and accessible | ||
| 3. Storage service is initialized and running | ||
| 4. Storage service is successfully registered with the meta service via the activator | ||
|
|
||
| ### Container Options | ||
|
|
||
| When starting the NebulaGraph container, you can pass options in a variadic way to configure it. | ||
|
|
||
| The module supports customization for each service container (Meta, Storage, Graph, and Activator) through ContainerCustomizer options. Common customizations include: | ||
|
|
||
| - Custom images for each service | ||
| - Environment variables | ||
| - Resource limits | ||
| - Network settings | ||
| - Volume mounts | ||
| - Wait strategies | ||
|
|
||
| {% include "../features/common_functional_options_list.md" %} | ||
|
|
||
| ### Container Methods | ||
|
|
||
| The `Cluster` struct provides the following methods: | ||
|
|
||
| #### ConnectionString | ||
|
|
||
| - Not available until the next release <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||
|
|
||
| Returns the host:port string for connecting to the NebulaGraph graph service (graphd). | ||
|
|
||
| ```golang | ||
| func (c *Cluster) ConnectionString(ctx context.Context) (string, error) | ||
| ``` | ||
|
|
||
| #### Terminate | ||
|
|
||
| - Not available until the next release <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||
|
|
||
| Stops and removes all containers in the NebulaGraph cluster (Meta, Storage, Graph, and Activator services) and cleans up the associated Docker network. | ||
|
|
||
| ```golang | ||
| func (c *Cluster) Terminate(ctx context.Context) error | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| include ../../commons-test.mk | ||
|
|
||
| .PHONY: test | ||
| test: | ||
| $(MAKE) test-nebulagraph |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| for i in $(seq 1 ${ACTIVATOR_RETRY}); do | ||
| echo "nebula" | nebula-console -addr graphd0 -port 9669 -u root -e 'ADD HOSTS "storaged0":9779' 1>/dev/null 2>/dev/null | ||
| if [ $? -eq 0 ]; then | ||
| echo "✔️ Storage activated successfully." | ||
| exit 0 | ||
| else | ||
| output=$(echo "nebula" | nebula-console -addr graphd0 -port 9669 -u root -e 'ADD HOSTS "storaged0":9779' 2>&1) | ||
| if echo "$output" | grep -q "Existed"; then | ||
| echo "✔️ Storage activated already , Exiting..." | ||
| exit 0 | ||
| fi | ||
| fi | ||
| if [ $i -lt ${ACTIVATOR_RETRY} ]; then | ||
| echo "⏳ Attempting to activate storaged, attempt $i/${ACTIVATOR_RETRY}... It's normal to take some attempts before storaged is ready. Please wait." | ||
| else | ||
| echo "❌ Failed to activate storaged after ${ACTIVATOR_RETRY} attempts. Please check MetaD, StorageD logs." | ||
| echo "ℹ️ Error during storage activation:" | ||
| echo "==============================================================" | ||
| echo "$output" | ||
| echo "==============================================================" | ||
| exit 1 | ||
| fi | ||
| sleep 5 | ||
| done && tail -f /dev/null |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| module github.com/testcontainers/testcontainers-go/modules/nebulagraph | ||
|
|
||
| go 1.23.6 | ||
|
|
||
| require ( | ||
| github.com/jolestar/go-commons-pool v2.0.0+incompatible | ||
| github.com/nebula-contrib/nebula-sirius v1.0.0-rc2 | ||
| github.com/stretchr/testify v1.10.0 | ||
| github.com/testcontainers/testcontainers-go v0.38.0 | ||
| ) | ||
|
|
||
| require ( | ||
| dario.cat/mergo v1.0.1 // indirect | ||
| github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect | ||
| github.com/Microsoft/go-winio v0.6.2 // indirect | ||
| github.com/apache/thrift v0.21.0 // indirect | ||
| github.com/cenkalti/backoff/v4 v4.2.1 // indirect | ||
| github.com/containerd/errdefs v1.0.0 // indirect | ||
| github.com/containerd/errdefs/pkg v0.3.0 // indirect | ||
| github.com/containerd/log v0.1.0 // indirect | ||
| github.com/containerd/platforms v0.2.1 // indirect | ||
| github.com/cpuguy83/dockercfg v0.3.2 // indirect | ||
| github.com/davecgh/go-spew v1.1.1 // indirect | ||
| github.com/distribution/reference v0.6.0 // indirect | ||
| github.com/docker/docker v28.2.2+incompatible // indirect | ||
| github.com/docker/go-connections v0.5.0 // indirect | ||
| github.com/docker/go-units v0.5.0 // indirect | ||
| github.com/ebitengine/purego v0.8.4 // indirect | ||
| github.com/felixge/httpsnoop v1.0.4 // indirect | ||
| github.com/go-logr/logr v1.4.3 // indirect | ||
| github.com/go-logr/stdr v1.2.2 // indirect | ||
| github.com/go-ole/go-ole v1.2.6 // indirect | ||
| github.com/gogo/protobuf v1.3.2 // indirect | ||
| github.com/google/uuid v1.6.0 // indirect | ||
| github.com/klauspost/compress v1.18.0 // indirect | ||
| github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect | ||
| github.com/magiconair/properties v1.8.10 // indirect | ||
| github.com/moby/docker-image-spec v1.3.1 // indirect | ||
| github.com/moby/go-archive v0.1.0 // indirect | ||
| github.com/moby/patternmatcher v0.6.0 // indirect | ||
| github.com/moby/sys/sequential v0.6.0 // indirect | ||
| github.com/moby/sys/user v0.4.0 // indirect | ||
| github.com/moby/sys/userns v0.1.0 // indirect | ||
| github.com/moby/term v0.5.0 // indirect | ||
| github.com/morikuni/aec v1.0.0 // indirect | ||
| github.com/opencontainers/go-digest v1.0.0 // indirect | ||
| github.com/opencontainers/image-spec v1.1.1 // indirect | ||
| github.com/pkg/errors v0.9.1 // indirect | ||
| github.com/pmezard/go-difflib v1.0.0 // indirect | ||
| github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect | ||
| github.com/shirou/gopsutil/v4 v4.25.5 // indirect | ||
| github.com/sirupsen/logrus v1.9.3 // indirect | ||
| github.com/tklauser/go-sysconf v0.3.12 // indirect | ||
| github.com/tklauser/numcpus v0.6.1 // indirect | ||
| github.com/yusufpapurcu/wmi v1.2.4 // indirect | ||
| go.opentelemetry.io/auto/sdk v1.1.0 // indirect | ||
| go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 // indirect | ||
| go.opentelemetry.io/otel v1.37.0 // indirect | ||
| go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.37.0 // indirect | ||
| go.opentelemetry.io/otel/metric v1.37.0 // indirect | ||
| go.opentelemetry.io/otel/sdk v1.37.0 // indirect | ||
| go.opentelemetry.io/otel/trace v1.37.0 // indirect | ||
| go.opentelemetry.io/proto/otlp v1.7.1 // indirect | ||
| golang.org/x/crypto v0.40.0 // indirect | ||
| golang.org/x/net v0.42.0 // indirect | ||
| golang.org/x/sys v0.34.0 // indirect | ||
| golang.org/x/text v0.27.0 // indirect | ||
| google.golang.org/protobuf v1.36.6 // indirect | ||
| gopkg.in/yaml.v3 v3.0.1 // indirect | ||
| ) | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.