feat(nextjs,backend,integration): Introduce dynamic keys from clerkMiddleware

[LauraBeatris]

Description

Resolves SDK-1253, SDK-1811, SDK-1770

This PR introduces dynamic keys from clerkMiddleware, solving the following problems:

• auth() cannot assert the session token signature due to an undefined secret key, leading to a runtime error.
• auth.createRedirect doesn't redirect to signInUrl or signUpUrl since those aren't propagated as well.

The propagation is done by encrypting an x-clerk-request-data header and sending it from the middleware runtime to the application runtime, where it's going to be decrypted using a key. The key used for encryption is resolved based on the following conditions:

• If security-sensitive options, such as secretKey are provided, then CLERK_ENCRYPTION_KEY is required
• Otherwise, fallbacks to CLERK_SECRET_KEY, which is going to have backward compatibility.

Checklist

• npm test runs as expected.
• npm run build runs as expected.
• (If applicable) JSDoc comments have been added or updated for any package exports
• (If applicable) Documentation has been updated
  • Merge Docs PR with this PR

Type of change

• 🐛 Bug fix
• 🌟 New feature
• 🔨 Breaking change
• 📖 Refactoring / dependency upgrade / documentation
• other:

[changeset-bot]

🦋 Changeset detected

Latest commit: c458ab7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 7 packages

Name	Type
@clerk/backend	Minor
@clerk/nextjs	Minor
@clerk/express	Patch
@clerk/fastify	Patch
@clerk/remix	Patch
@clerk/clerk-sdk-node	Patch
@clerk/tanstack-start	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[nikosdouvlis]

I know that this is still a draft PR, so don't worry about this comment yet. Once everything's is in place, let's also add an e2e test covering this flow. Feel free to ping anyone from team-sdk about this :)

[dimkl]

❓ Why did we choose the approach of the clerkClient() Proxy instead of exposing a getClerkClient()?
Even though we have already introduced the Proxy in the @clerk/clerk-sdk-node to load dynamically the env variables, I think that we could avoid using it here, and introduce a more specific API to avoid any confusion over the clerkClient.

cc: @LauraBeatris , @nikosdouvlis , @BRKalow