[PerUserAuth] Add EARS auth type for Connectors V2

[lorenabalan]

Summary

Closes https://github.com/elastic/search-team/issues/12949

Changes:

• introduced getStoredTokenWithRefresh to avoid duplicate logic between ears and non-ears authz; the only real difference was how we refreshed
• introduced "strategies" for axios instances to avoid too many if ears branches in how we instantiate the instance
• added xpack.actions.ears.url config that points to the relevant base EARS url; ⚠️ this needs to be populated in ES3 (and hosted) deployments, like we did https://github.com/elastic/serverless-gitops/blob/f51db9f789fe9ce1ea668e27970a4468b37ef7c2/services/kibana-controller/values/qa/default.yaml#L63
• added 2 new folders lib/ears and lib/axios_auth_strategies but some files import from outer folder lib. I just compromised for that over moving files and making the diff larger, though I'm still not sure if it's the best folder split. The lib folder is quite unwieldy. 😅
• Remove leftover capitalize - see thread
• Key lock by connector ID and profile UID #257968

++ Thank you CodeRabbit:

• fixed interceptor logic for authz code grant & ears flow, so that on retry & successful refresh, we also update the headers on the axiosInstance, for future requests (Update axiosInstance headers for future requests)
• fixed useBasicAuth logic in authz code grant in Fix useBasicAuth propagation; connectors that specify tokenEndpointAuthMethod: 'client_secret_post' will now correctly send credentials in the request body instead of always defaulting to HTTP Basic Auth when refreshing tokens via getToken.

Other considerations

• 🚫 revoke workflow kept out of scope of this PR ↔️ see [DO NOT CLOSE PR] Prepare demo project #255701 (review)
• ✨ We could maybe add nicer UI component to render for ears authz type in the form (connector flyout); not a huge fan of the fact that scope are treated like secrets and they reset to default after clicking Save
• 👉 We should raise a follow-up PR to set EARS URL in ES3 deployments in the xpack.actions.ears.url config (if everyone agrees on the name)
  • Serverless: https://github.com/elastic/serverless-gitops/pull/81614
  • ECH: https://github.com/elastic/cloud/pull/153459
• ❓ We have NO separate rate limits between EARS and normal Authz Grant flow. Should we? I would say no because ultimately we reach 3rd party provider so we can't have higher rate limits than vanilla flow. The name of the current config auth.oauth_authorization_code.rate_limits kind of implies there would be more rate limits per auth types.

Testing

Enable EARS auth on Google Drive connector by adding the following lines to auth.type in src/platform/packages/shared/kbn-connector-specs/src/specs/google_drive/google_drive.ts:

{
    type: 'ears',
    defaults: {
        provider: 'google',
        scope: 'https://www.googleapis.com/auth/drive.readonly',
    },
},

Update your kibana.dev.yaml to contain the following lines:

server.publicBaseUrl: http://localhost:5601
xpack.actions.ears.url: https://elastic-auth-redirect-service.eu-west-1.aws.svc.qa.elastic.cloud

uiSettings:
  overrides:
    'workflows:ui:enabled': true

logging:
  loggers:
    - name: plugins.actions
      level: debug
      appenders: [default]

Start ES

yarn es serverless --projectType elasticsearch_search --kill

Start Kibana

yarn start --serverless=es

Try to create & authorise a Google Drive connector. Then run a workflow such as

version: '1'
name: 'sources.google_drive.search'
description: Search for files in Google Drive using Google's query syntax
tags: ['agent-builder-tool']
enabled: true
triggers:
  - type: manual
inputs:
  - name: query
    type: string
    description: "Google Drive search query. Examples: name contains 'budget' and trashed=false | fullText contains 'quarterly report' and mimeType='application/pdf' | 'me' in owners and modifiedTime > '2024-01-01' | mimeType='application/vnd.google-apps.folder' and trashed=false. Operators: contains, =, !=, <, >, <=, >=. Combine with 'and'/'or'. String values use single quotes. Add 'and trashed=false' to exclude trashed files."
  - name: pageSize
    type: number
    required: false
    description: Number of results to return (default 250, max 1000)
  - name: pageToken
    type: string
    required: false
    description: "Pagination token. Pass the 'nextPageToken' value from a previous response to get the next page. When nextPageToken is absent in the response, there are no more results."
  - name: orderBy
    type: string
    required: false
    description: "Sort order: 'createdTime', 'createdTime desc', 'modifiedTime', 'modifiedTime desc', 'name', or 'name desc'"
steps:
  - name: search_files
    type: google_drive.searchFiles
    connector-id: <ENTER YOUR CONNECTOR ID HERE!>
    with:
      query: "${{inputs.query}}"
      pageSize: ${{inputs.pageSize}}
      pageToken: "${{inputs.pageToken}}"
      orderBy: "${{inputs.orderBy}}"

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials - we have separate https://github.com/elastic/search-team/issues/13469 to track this
• Unit or functional tests were updated or added to match the most common scenarios
• The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
• Review the backport guidelines and apply applicable backport:* labels.

[coderabbitai]

Caution

Review failed

The head commit changed during the review from bd3da1a to 268368a.

📝 Walkthrough

Walkthrough

This pull request adds OAuth 2.0 Authorization Code flow and EARS (Elastic Authentication Redirect Service) support to the Actions plugin. Changes include new auth type specifications for OAuth Authorization Code and EARS (supporting Google, Microsoft, Slack providers), refactored Axios authentication via a strategy pattern, OAuth authorization and callback routes, EARS URL/token management utilities, token refresh with per-connector concurrency control, OAuth rate limiting configuration, and updated connector specs with new authentication modes. Code ownership assignments for EARS components are also added.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch lb/poc/ears-auth

• 🛠️ Update Documentation: Commit on current branch
• 🛠️ Update Documentation: Create PR

📝 Coding Plan

• Generate coding plan for human review comments

Warning

Tools execution failed with the following error:

Failed to run tools: 13 INTERNAL: Received RST_STREAM with code 2 (Internal server error)

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Tip

You can make CodeRabbit's review stricter and more nitpicky using the `assertive` profile, if that's what you prefer.

Change the reviews.profile setting to assertive to make CodeRabbit's nitpick more issues in your PRs.

[jeramysoucy]

Approving to unblock - no longer owner of any changes here

[seanstory]

👏
🚢

[jcger]

I'm assuming the content here was copied over, right? nothing that was added/removed?

[lorenabalan]

There are changes, the ones highlighted in the PR description as CodeRabbit finds:

• Update axiosInstance headers for future requests
• Fix useBasicAuth propagation

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: f3d24c3

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
stackConnectors	1003	1004	+1
workflowsManagement	1553	1554	+1
total			+2

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
@kbn/connector-specs	149	156	+7

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
stackConnectors	1.7MB	1.7MB	+1.2KB
triggersActionsUi	1.9MB	1.9MB	+5.0B
workflowsManagement	2.4MB	2.4MB	+231.0B
total			+1.5KB

Unknown metric groups

API count

id	before	after	diff
@kbn/connector-specs	158	165	+7