feat(kubernetes): add kubernetes_watch input for real-time resource monitoring

[aronchick]

Summary

Add a new kubernetes_watch input that uses the Kubernetes Watch API to stream real-time events for cluster resources. This enables event-driven pipelines that react to Kubernetes state changes.

Features

• Watch any resource type via dynamic discovery (pods, deployments, CRDs, etc.)
• Per-namespace or cluster-wide monitoring
• Label and field selector filtering
• Automatic reconnection with exponential backoff
• Proper handling of 410 Gone errors (expected watch API behavior)

Implementation Highlights

• Uses DeferredDiscoveryRESTMapper for dynamic GVR resolution (no static resource list)
• Supports in-cluster auth, kubeconfig, or explicit credentials
• Resource version tracking for resumable watches

Example Configuration

input:
  kubernetes_watch:
    resource: pods
    namespaces:
      - default
    label_selector: "app=myapp"

Test Plan

• Unit tests for config parsing
• Unit tests for auth configuration
• Lint passes (0 issues)
• All tests pass
• Manual testing with local cluster

Files Changed (~1,060 lines)

• internal/impl/kubernetes/ - New kubernetes input package
• go.mod, go.sum - k8s client-go dependencies

[aronchick]

I've built some simple testing scripts as well.

https://github.com/aronchick/bento/tree/kubernetes-integration-tests/testing/kubernetes

[aronchick]

I think i got to everything!

[aronchick]

@gregfurman I've addressed your feedback - think i got to everything. Just LMK if you need anything else!

[gregfurman]

Thanks for addressing all my feedback. The only blocking things left to address IMO is the Close method and use of caching. Looking forward to getting this in!

[aronchick]

@gregfurman All feedback addressed and tests passing locally - ready for re-review!

[aronchick]

@gregfurman tag you're it. (i thinki got everything)

[gregfurman]

@aronchick Will give this a final look today! Thanks for addressing all my feedback 🙌

[gregfurman]

Looks good! Last, you just need to add the kubernetes implementation to public/components in order to register this plugin to the global bento environment.

Here's an example of this for the mqtt component:

bento/public/components/mqtt/package.go

Lines 1 to 6 in 85332f0

	package mqtt
	import (
	// Bring in the internal plugin definitions.
	_ "github.com/warpstreamlabs/bento/internal/impl/mqtt"
	)

then we can add this to public/components/all so that importing in github.com/warpstreamlabs/bento/public/components/all will automatically register the component to the global environment i.e

bento/public/components/all/package.go

Line 37 in 85332f0

_ "github.com/warpstreamlabs/bento/public/components/mqtt"

Lastly, run make docs and commit the generated .md files for the kubernetes component that will be used in the bento doc site.

Once done, think it's good to go!

[gregfurman]

@jem-davies The K8s API adds an extra 24.4 MB to the bento binary size (at least on Darwin) which ends up constituing like 11% of the size. Should we add this package behind the x_bento_extra build tag? Or perhaps a custom kubernetes build tag could work as well.

See gsa analysis output below of bento binary 👇

┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ bento                                                                                                              │
├─────────┬─────────────────────────────────────────────────────────────────────────────────────┬────────┬───────────┤
│ PERCENT │ NAME                                                                                │ SIZE   │ TYPE      │
├─────────┼─────────────────────────────────────────────────────────────────────────────────────┼────────┼───────────┤
│ 12.10%  │ __rodata __TEXT                                                                     │ 27 MB  │ section   │
│ 12.03%  │ __rodata __DATA_CONST                                                               │ 27 MB  │ section   │
│ 6.91%   │ k8s.io/api                                                                          │ 15 MB  │ vendor    │
│ 5.85%   │ github.com/aws/aws-sdk-go-v2                                                        │ 13 MB  │ vendor    │
│ 5.30%   │ github.com/elastic/go-elasticsearch/v9                                              │ 12 MB  │ vendor    │
│ 5.14%   │ github.com/apache/arrow-go/v18                                                      │ 11 MB  │ vendor    │
│ 4.25%   │ k8s.io/client-go                                                                    │ 9.4 MB │ vendor    │
├─────────┼─────────────────────────────────────────────────────────────────────────────────────┼────────┼───────────┤
│ 100.00% │ Known                                                                               │ 222 MB │           │
│ 100%    │ Total                                                                               │ 222 MB │           │
└─────────┴─────────────────────────────────────────────────────────────────────────────────────┴────────┴───────────┘

[jem-davies]

@jem-davies The K8s API adds an extra 24.4 MB to the bento binary size (at least on Darwin) which ends up constituing like 11% of the size. Should we add this package behind the x_bento_extra build tag? Or perhaps a custom kubernetes build tag could work as well.

My take is that if a user is worried about binary size or say, CVE's in components they don't use - then we should advise them to create their own distribution where they import select components.

So therefore I don't think we should use a build tag requiring users to specifically 'opt-in' to a k8s component.