Skip to content

feat(azure_blob sink): Add proxy support#24256

Merged
thomasqueirozb merged 6 commits intovectordotdev:masterfrom
Automattic:feature/azure_blob_proxy
Feb 14, 2026
Merged

feat(azure_blob sink): Add proxy support#24256
thomasqueirozb merged 6 commits intovectordotdev:masterfrom
Automattic:feature/azure_blob_proxy

Conversation

@joshcoughlan
Copy link
Contributor

@joshcoughlan joshcoughlan commented Nov 17, 2025

Summary

This PR adds per-sink proxy support to the azure_blob sink, matching the behavior already available for AWS S3 and GCP sinks.

Highlights:

  • Each Azure Blob sink instance now uses its own proxy settings from SinkContext (ProxyConfig), not environment variables.
  • The Azure SDK transport is configured via azure_core::http::ClientOptions and a dedicated, aliased reqwest 0.12 client (reqwest_0_12_24), so it does not impact the project-wide reqwest usage.
  • Supports HTTP and HTTPS proxies, including credentials embedded in the proxy URL (handled by reqwest::Proxy).
  • Honors no_proxy patterns at the sink level to bypass the proxy for matching Azure endpoints.

Vector configuration

Example of per-sink proxy settings for Azure Blob:

[sinks.my_azure_blob]
type = "azure_blob"
connection_string = "DefaultEndpointsProtocol=https;AccountName=<account>;AccountKey=<key>;EndpointSuffix=core.windows.net"
container_name = "my-logs"
healthcheck.enabled = true

[sinks.my_azure_blob.proxy]
enabled = true
http = "http://user:pass@proxy.local:3128"
https = "http://user:pass@proxy.local:3128"
# Bypass proxy for these hosts/domains
no_proxy = "localhost,.internal.example.com"

Notes:

  • Credentials in the proxy URL are supported by reqwest.
  • If the Azure endpoint matches no_proxy or the proxy is disabled, the sink makes direct connections and disables any system proxy auto-detection for this client.

How did you test this PR?

  • Build and lint:
    • Ensured the updated call sites compile (e.g., integration helpers that construct the client).
  • Functional checks:
    • Verified that the Azure client is constructed with a reqwest 0.12 transport and that proxies are applied only for this sink.
    • Sanity-tested no_proxy handling by targeting a host in the exclusion list to ensure direct connections are used.
    • Am running a compiled binary of this PR in a real-world environment.
  • Suggested reviewer test plan (optional):
    • Run Azurite locally and point an azure_blob sink at it with:
      • No proxy (baseline).
      • HTTP proxy set.
      • HTTPS proxy set.
      • A no_proxy entry matching the Azurite host to confirm bypass.
    • Confirm healthcheck behavior with valid/invalid credentials and unknown containers.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
    • Suggested changelog: enhancement(azure_blob sink): Add per-sink proxy support via reqwest + azure_core::http::ClientOptions (HTTP/HTTPS proxies, honors no_proxy, supports proxy credentials).
  • No. A maintainer will apply the no-changelog label to this PR.

References

Notes

  • Implementation uses an aliased reqwest 0.12 client (reqwest_0_12_24) to avoid affecting other components that depend on a different reqwest version.
  • No environment variables are used; proxy configuration is per-sink via ProxyConfig.
  • If this PR modifies dependencies (Cargo.lock), I’ll run make build-licenses to refresh the license inventory.
  • This (or the parent SDK PR) fixes a bug introduced by Azure SDK version bump introduced regression in token refresh #23036 in which the documented SAS connection_string ("BlobEndpoint=https://mylogstorage.blob.core.windows.net/;SharedAccessSignature=generatedsastoken") doesn't work and fails with an error indicating it requires AccountName:
    ERROR vector::topology::builder: Configuration error. error=Sink "testing-azure_blob": Account name missing in connection string

@github-actions github-actions bot added the domain: sinks Anything related to the Vector's sinks label Nov 17, 2025
@joshcoughlan joshcoughlan marked this pull request as ready for review November 17, 2025 18:28
@joshcoughlan joshcoughlan requested a review from a team as a code owner November 17, 2025 18:28
@thomasqueirozb
Copy link
Contributor

Hi @joshcoughlan, I think this one should go in after #24255 right?

@thomasqueirozb thomasqueirozb added provider: azure Anything `azure` service provider related sink: azure_blob Anything `azure_blob` sink related meta: awaiting author Pull requests that are awaiting their author. labels Dec 10, 2025
@joshcoughlan
Copy link
Contributor Author

Hi @joshcoughlan, I think this one should go in after #24255 right?

Correct. #24255 will need to be merged first. After that's done, this PR will likely need a little attention.

@joshcoughlan
Copy link
Contributor Author

I have read the CLA Document and I hereby sign the CLA

@joshcoughlan joshcoughlan changed the title feat(azure_blob): Add proxy support feat(azure_blob sink): Add proxy support Jan 8, 2026
@joshcoughlan joshcoughlan force-pushed the feature/azure_blob_proxy branch from 2c682fe to 808fb44 Compare February 2, 2026 22:14
@github-actions github-actions bot removed the meta: awaiting author Pull requests that are awaiting their author. label Feb 3, 2026
@joshcoughlan
Copy link
Contributor Author

joshcoughlan commented Feb 3, 2026

Hey @thomasqueirozb I've updated this PR now that #24255 has been merged. I've run the checks in .github/workflows/test.yml and there are failing make test-vrl checks, but everything else seems fine. I'm pretty sure the test-vrl checks were failing when I ran them for #24255. I also compiled the binary and tried it on a live workload both using the proxy and not. Seems to be working fine.

@joshcoughlan
Copy link
Contributor Author

Added changelog entry and updated the sink's documentation.

@joshcoughlan
Copy link
Contributor Author

recheck

Copy link
Contributor

@thomasqueirozb thomasqueirozb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @joshcoughlan, the changelog entry is a file placed inside the changelog.d/ directory. You can take a look at other files already in there as examples. I can merge this in once this is added.

Additionally do you think that it is viable to add an integration test for this?

Either way, theses changes look good to me. Thanks for your contribution!

@thomasqueirozb thomasqueirozb added the meta: awaiting author Pull requests that are awaiting their author. label Feb 11, 2026
@thomasqueirozb thomasqueirozb self-assigned this Feb 11, 2026
@joshcoughlan joshcoughlan requested a review from a team as a code owner February 11, 2026 18:52
@github-actions github-actions bot added domain: external docs Anything related to Vector's external, public documentation and removed meta: awaiting author Pull requests that are awaiting their author. labels Feb 11, 2026
@joshcoughlan
Copy link
Contributor Author

I had made these changes, but was on a child branch that gets built in our CI for testing the binary and container image. Oops.

Copy link
Contributor

@thomasqueirozb thomasqueirozb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Co-authored-by: Thomas <thomasqueirozb@gmail.com>
auto-merge was automatically disabled February 13, 2026 22:20

Head branch was pushed to by a user without write access

@thomasqueirozb thomasqueirozb added this pull request to the merge queue Feb 13, 2026
Merged via the queue into vectordotdev:master with commit 3a73af4 Feb 14, 2026
50 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Feb 14, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

domain: external docs Anything related to Vector's external, public documentation domain: sinks Anything related to the Vector's sinks provider: azure Anything `azure` service provider related sink: azure_blob Anything `azure_blob` sink related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants