Refactor the D1 binding to use D1DatabaseSession for all actions

[lambrospetrou]

Overview

This change prepares the D1 binding for the upcoming Sessions API (relevant https://blog.cloudflare.com/building-d1-a-global-database). Most of the changes is moving things around, but the highlights are as below:

1. A new D1DatabaseSession class is introduced that maintains a commitTokenOrConstraint property, and also takes ownership of the fetch+_send/_sendOrThrow methods.
   • This class proxies all calls to the D1 eyeball worker, and automatically adds the commit token as an HTTP header to the requests and extracts it from the response headers.
   • Our eyeball worker and the D1 DO will extract the commit token and accordingly decide which replica should route the requests to.
2. There is a new D1DatabaseSessionAlwaysPrimary that extends D1DatabaseSession and essentially always sends the commit token constraint first-primary.
   • This session subclass also provides the existing dump, exec functions that the new Sessions API does not have.
3. As specified by the SPEC, the default if no commit token is provided to the withSession() function is "first-unconstrained" (to run the first query (which may be a read or a write) with no constraints, and then use causal consistency for subsequent queries).

Notes

I decided to design it such that D1DatabaseSession proxies all the calls and maintains the commit token so that the other classes don't really care about which session they use, or having to know about it.

This way, all the complex logic about the commit token and how the interaction with the D1 DOs is going to be inside our eyeball worker, which we control fully, it's easier to deploy and fix, and is the first shared entrypoint with the REST API too, so once the REST API can pass the commit token to use, the rest of the logic is exactly the same.

I am not sure if using HTTP headers is common in Cloudflare to pass this kind of information to our APIs, so if that's not allowed/suggested, then we will need to pass it as query parameter instead.

Suggestions for naming of the header or the query parameter welcome.

Compatibility flag

There is a new experimental compatibility flag enable_d1_with_sessions_api to expose this new withSessions() method. The default binding still uses the D1DatabaseSession object though, so this PR's changes will actually be running after merge, always sending the first-primary constraint.

The d1-api-test.js is used by the D1 eyeball worker E2E tests so I added the new tests in a new test suite with the compatibility flag enabled only for that.

Tests

bazel run //src/cloudflare/internal/test/d1:d1-api-test
...
Executing tests from //src/cloudflare/internal/test/d1:d1-api-test
-----------------------------------------------------------------------------
workerd/server/server.c++:3847: debug: [ TEST ] d1-api-test:testWithoutSessions
workerd/server/server.c++:3855: debug: [ PASS ] d1-api-test:testWithoutSessions


bazel run //src/cloudflare/internal/test/d1:d1-api-test-with-sessions
...
Executing tests from //src/cloudflare/internal/test/d1:d1-api-test-with-sessions
-----------------------------------------------------------------------------
workerd/server/server.c++:3847: debug: [ TEST ] d1-api-test-with-sessions:test_d1_api_happy_path
workerd/server/server.c++:3855: debug: [ PASS ] d1-api-test-with-sessions:test_d1_api_happy_path
workerd/server/server.c++:3847: debug: [ TEST ] d1-api-test-with-sessions:test_d1_api_happy_path_withsessions_default
workerd/server/server.c++:3855: debug: [ PASS ] d1-api-test-with-sessions:test_d1_api_happy_path_withsessions_default
workerd/server/server.c++:3847: debug: [ TEST ] d1-api-test-with-sessions:test_d1_api_happy_path_withsessions_first_primary
workerd/server/server.c++:3855: debug: [ PASS ] d1-api-test-with-sessions:test_d1_api_happy_path_withsessions_first_primary
workerd/server/server.c++:3847: debug: [ TEST ] d1-api-test-with-sessions:test_d1_api_happy_path_withsessions_first_unconstrained
workerd/server/server.c++:3855: debug: [ PASS ] d1-api-test-with-sessions:test_d1_api_happy_path_withsessions_first_unconstrained
workerd/server/server.c++:3847: debug: [ TEST ] d1-api-test-with-sessions:test_d1_api_happy_path_withsessions_some_ranomd_token
workerd/server/server.c++:3855: debug: [ PASS ] d1-api-test-with-sessions:test_d1_api_happy_path_withsessions_some_ranomd_token
workerd/server/server.c++:3847: debug: [ TEST ] d1-api-test-with-sessions:test_d1_api_withsessions_old_token_skipped
workerd/server/server.c++:3855: debug: [ PASS ] d1-api-test-with-sessions:test_d1_api_withsessions_old_token_skipped
workerd/server/server.c++:3847: debug: [ TEST ] d1-api-test-with-sessions:test_d1_api_withsessions_token_handling
workerd/server/server.c++:3855: debug: [ PASS ] d1-api-test-with-sessions:test_d1_api_withsessions_token_handling

The first target runs the default database tests, without using the sessions API, and without the compatibility flag enabled.

The second target runs all the test suites with the compatibility flag enabled.

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[lambrospetrou]

I have read the CLA Document and I hereby sign the CLA

[justin-mp]

I won't comment on the details of the code, but the class structure seems reasonable, especially how the statements use the session class.

[geelen]

Looks good! A bit of feedback, but nothing major, so consider this approved.

[lambrospetrou]

This was cut-pasted as-is, without any changes apart from adding the last section of DELETE statements, so that the database is cleaned up.