wasm: Add wasm support #976

tmc · 2025-06-10T05:27:00Z

Add WebAssembly (WASM) Support to bbolt

This PR adds comprehensive support for WebAssembly targets (js and wasip1) to bolt, enabling the database to run in browsers and WASI-compliant runtimes.

What's Changed

Memory-mapped I/O emulation: Since WASM doesn't support real mmap, we implement a buffer-based approach that reads the database file into memory
Transaction synchronization: Added a txIniter interface to handle platform-specific transaction initialization, ensuring data consistency in WASM environments where memory buffers
need manual refresh
Platform-specific implementations: Created bolt_wasm.go with WASM-specific implementations of mmap, munmap, madvise, and file locking operations

Testing Infrastructure

Added WASM test targets in the Makefile:
- make test-wasm for browser-based JavaScript WASM (GOOS=js)
- make test-wasip1 for WASI Preview 1 runtime testing
Added GitHub Actions workflows for automated WASM testing in CI

Test Adaptations

Skip memory-intensive tests on WASM platforms due to runtime constraints
Added appropriate test guards for WASM-specific limitations

Testing

All tests pass on WASM platforms:

Browser WASM (GOOS=js GOARCH=wasm)
WASI Preview 1 (GOOS=wasip1 GOARCH=wasm)

The implementation has been tested with both wazero and wasmtime runtimes.

k8s-ci-robot · 2025-06-10T05:27:03Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: tmc
Once this PR has been reviewed and has the lgtm label, please assign ahrtr for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2025-06-10T05:27:09Z

Hi @tmc. Thanks for your PR.

I'm waiting for a etcd-io member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

internal/common/pagesize.go

internal/common/bolt_wasm.go

ahrtr · 2025-06-12T09:59:50Z

cc @Elbehery @fuweid @ivanvc @jmhbnz @tjungblu

db_test.go

db.go

ivanvc · 2025-06-12T17:59:40Z

/ok-to-test

Elbehery · 2025-06-12T18:14:45Z

db.go


+	// Allow WASM-specific transaction initialization
+	if runtime.GOARCH == "wasm" {
+		if initer, ok := any(db).(txIniter); ok {


Thanks @tmc for your prompt follow up 🚀 👍🏽

Suggested change

if initer, ok := any(db).(txIniter); ok {

if initer, ok := db.(txIniter); ok {

wdyt about this ?

Any reason to wrap db within any ?

type assertions must take an interface value and db is *DB which means it can't be used in a type assertion.

cc @fuweid @ahrtr

ivanvc · 2025-06-13T23:40:53Z

I suggest holding this pull request until we make #986 work and find a solution to cross-compile in other architectures, so we're able to catch issues in new ones early/automated, rather than waiting for someone to find out compile/build issues.

Elbehery · 2025-06-14T07:31:13Z

I suggest holding this pull request until we make #986 work and find a solution to cross-compile in other architectures, so we're able to catch issues in new ones early/automated, rather than waiting for someone to find out compile/build issues.

cc @ahrtr @fuweid

ivanvc · 2025-07-18T06:41:00Z

Now we have merged #986, which lints the files in different architectures. Would it be possible to run the linter against the js GOOS? I have no experience with this, so I'm unsure. @tmc, would you be able to add running the linter for this OS/ARCH?

Add support for running bbolt in WebAssembly environments (js/wasm and wasip1): - Implement memory management for WASM platforms without mmap - Add platform-specific transaction initialization - Configure smaller page size and memory limits for WASM - Add GitHub workflow for WASM testing (js and wasip1) - Adjust tests to handle WASM memory constraints - Support both browser (js) and WASI environments Signed-off-by: Travis Cline <[email protected]>

Replace os.Getpagesize() calls with common.DefaultPageSize constant and remove redundant pagesize.go implementation. Simplify WASM-specific code by removing custom initialization and page size handling. Update tests to use the new constant. Signed-off-by: Travis Cline <[email protected]>

Update the db_test.go file to accommodate the recent WASM-related changes, including simplifying platform-specific page size handling and adjusting tests to handle WASM memory constraints. Signed-off-by: Travis Cline <[email protected]>

Elbehery · 2025-10-21T10:41:23Z

Thanks @tmc for your contribution 🙏🏽

Elbehery

LGTM

the implementation of bolt_wasm.go is straight forward.

But I am in favor of using defer for releasing the lock

cc @ahrtr @ivanvc

Elbehery · 2025-10-21T10:48:24Z

db.go

+	if runtime.GOARCH == "wasm" {
+		if initer, ok := any(db).(txIniter); ok {
+			if err := initer.txInit(); err != nil {
+				db.metalock.Unlock()


Any reason not to precede this call with defer ?

ideally it should be move above, after acquiring the lock

Elbehery · 2025-10-21T10:48:41Z

db.go

+	if runtime.GOARCH == "wasm" {
+		if initer, ok := any(db).(txIniter); ok {
+			if err := initer.txInit(); err != nil {
+				db.rwlock.Unlock()


ahrtr · 2025-10-21T13:46:22Z

Do not get time to dig into the details, but my immediately feeling is that this PR isn't well integrated into the existing implementations,

can you add built job for wasm into https://github.com/etcd-io/bbolt/blob/main/.github/workflows/cross-arch-test.yaml?
can you add test job for wasm similar to https://github.com/etcd-io/bbolt/blob/main/.github/workflows/tests_arm64.yaml
Adding if runtime.GOARCH == "wasm" into db.go seems not a nice way, is it possible to get all wasm related details included in bolt_wasm.go?

Pls do not get me wrong, I am supportive to add wasm support.

k8s-ci-robot added size/L needs-ok-to-test labels Jun 10, 2025

tmc force-pushed the wasm-support branch from a7d8412 to 55aec40 Compare June 10, 2025 05:28

tmc changed the title ~~bolt: Add wasm support~~ wasm: Add wasm support Jun 10, 2025

ahrtr reviewed Jun 12, 2025

View reviewed changes

internal/common/pagesize.go Outdated Show resolved Hide resolved

internal/common/bolt_wasm.go Outdated Show resolved Hide resolved

Elbehery reviewed Jun 12, 2025

View reviewed changes

db_test.go Outdated Show resolved Hide resolved

Elbehery reviewed Jun 12, 2025

View reviewed changes

db.go Outdated Show resolved Hide resolved

Elbehery reviewed Jun 12, 2025

View reviewed changes

db.go Outdated Show resolved Hide resolved

tmc force-pushed the wasm-support branch from f1b686c to 0ace59e Compare June 12, 2025 17:44

k8s-ci-robot added ok-to-test and removed needs-ok-to-test labels Jun 12, 2025

fuweid self-requested a review June 12, 2025 18:04

Elbehery reviewed Jun 12, 2025

View reviewed changes

github-actions bot added the stale label Oct 17, 2025

tmc added 3 commits October 21, 2025 00:46

wasm: update db_test.go for WASM support changes

aeededc

Update the db_test.go file to accommodate the recent WASM-related changes, including simplifying platform-specific page size handling and adjusting tests to handle WASM memory constraints. Signed-off-by: Travis Cline <[email protected]>

tmc force-pushed the wasm-support branch from 0ace59e to aeededc Compare October 21, 2025 10:33

ahrtr removed the stale label Oct 21, 2025

Elbehery reviewed Oct 21, 2025

View reviewed changes

	if initer, ok := any(db).(txIniter); ok {
	if initer, ok := db.(txIniter); ok {

wasm: Add wasm support #976

Are you sure you want to change the base?

wasm: Add wasm support #976

Uh oh!

Conversation

tmc commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Jun 10, 2025

Uh oh!

k8s-ci-robot commented Jun 10, 2025

Uh oh!

Uh oh!

Uh oh!

ahrtr commented Jun 12, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ivanvc commented Jun 12, 2025

Uh oh!

Elbehery Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Elbehery Jun 14, 2025

Choose a reason for hiding this comment

Uh oh!

tmc Jun 17, 2025

Choose a reason for hiding this comment

Uh oh!

Elbehery Jun 17, 2025

Choose a reason for hiding this comment

Uh oh!

ivanvc commented Jun 13, 2025

Uh oh!

Elbehery commented Jun 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ivanvc commented Jul 18, 2025

Uh oh!

Elbehery commented Oct 21, 2025

Uh oh!

Elbehery left a comment

Choose a reason for hiding this comment

Uh oh!

Elbehery Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Elbehery Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ahrtr commented Oct 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

5 participants

tmc commented Jun 10, 2025 •

edited

Loading

Elbehery Jun 12, 2025 •

edited

Loading

Elbehery commented Jun 14, 2025 •

edited

Loading

Elbehery Oct 21, 2025 •

edited

Loading

Elbehery Oct 21, 2025 •

edited

Loading