Skip to content

fix: scheduler ignore KUBECONFIG env even if this environment variable is set - #681

Merged
archlitchi merged 1 commit into
Project-HAMi:masterfrom
Shouren:fix/k8s-client-for-scheduler
Feb 21, 2025
Merged

fix: scheduler ignore KUBECONFIG env even if this environment variable is set#681
archlitchi merged 1 commit into
Project-HAMi:masterfrom
Shouren:fix/k8s-client-for-scheduler

Conversation

@Shouren

@Shouren Shouren commented Dec 5, 2024 •
edited
Loading

Copy link
Copy Markdown
Member

What type of PR is this?
/kind bug

What this PR does / why we need it:
The implementation of NewClient function in pkg/k8sutil/client.go gets *restclient.Config from InClusterConfig api first even if the KUBECONFIG env has been set which is out of expect. This PR fix it.

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

@codecov

codecov Bot commented Dec 5, 2024 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pkg/scheduler/scheduler.go 0.00% 1 Missing ⚠️
Flag Coverage Δ
unittests 32.69% <0.00%> (+0.30%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
pkg/scheduler/scheduler.go 33.61% <0.00%> (+1.37%) ⬆️

@Shouren
Shouren force-pushed the fix/k8s-client-for-scheduler branch 3 times, most recently from ae54f90 to dda05c5 Compare December 5, 2024 10:04
@Shouren Shouren changed the title fix: scheduler force using InClusterConfig when KUBECONFIG is set fix: scheduler ignore KUBECONFIG env Dec 5, 2024
@Shouren
Shouren force-pushed the fix/k8s-client-for-scheduler branch 4 times, most recently from 57fee3a to 4aae789 Compare December 5, 2024 11:18
lengrongfu
lengrongfu reviewed Dec 6, 2024
View reviewed changes
Comment thread pkg/scheduler/scheduler.go Outdated

func (s *Scheduler) Start() {
kubeClient, err := k8sutil.NewClient()
kubeClient, err := client.NewClient()

@lengrongfu lengrongfu Dec 6, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can use client.GetClient() to get kubeClient, because this kubeClient having init function Initialization completed.

@Shouren Shouren Dec 6, 2024 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. By doing so, some the unused code can also be removed

@lengrongfu

lengrongfu commented Dec 6, 2024

Copy link
Copy Markdown
Member

suggest title: fix: scheduler ignore KUBECONFIG env even if this environment variable is set

@Shouren Shouren changed the title fix: scheduler ignore KUBECONFIG env fix: scheduler ignore KUBECONFIG env even if this environment variable is set Dec 6, 2024
@Shouren
Shouren force-pushed the fix/k8s-client-for-scheduler branch 3 times, most recently from 7e05044 to 2637cf5 Compare December 6, 2024 03:39
@lengrongfu

lengrongfu commented Dec 6, 2024

Copy link
Copy Markdown
Member

@Shouren can you rebase main branch? because having resolve code lint question.

@Shouren
Shouren force-pushed the fix/k8s-client-for-scheduler branch from 2637cf5 to 09448b9 Compare December 6, 2024 07:27
@Shouren

Shouren commented Dec 6, 2024 •
edited
Loading

Copy link
Copy Markdown
Member Author

@Shouren can you rebase main branch? because having resolve code lint question.

@lengrongfu All right, rebase is done.

@lengrongfu

lengrongfu commented Dec 6, 2024

Copy link
Copy Markdown
Member

/lgtm

@chaunceyjiang chaunceyjiang mentioned this pull request Jan 2, 2025
Merged
@wawa0210 wawa0210 added the kind/bug Something isn't working label Jan 2, 2025
@Shouren
Shouren force-pushed the fix/k8s-client-for-scheduler branch 2 times, most recently from f57696c to a0eb9f6 Compare February 21, 2025 07:41
@Shouren
fix: Remove the pkg/k8sutil/client.go and replace it with HAMi/pkg/ut…
0e2a9f4 
…il/client in pkg/scheduler/scheduler.go

Signed-off-by: Shouren Yang <yangshouren@gmail.com>
@Shouren
Shouren force-pushed the fix/k8s-client-for-scheduler branch from a0eb9f6 to 0e2a9f4 Compare February 21, 2025 07:42
@archlitchi

archlitchi commented Feb 21, 2025

Copy link
Copy Markdown
Member

/approve

@archlitchi
archlitchi merged commit b512861 into Project-HAMi:master Feb 21, 2025
@yangshiqi yangshiqi mentioned this pull request Mar 13, 2025
Merged
@ouyangluwei163 ouyangluwei163 mentioned this pull request Apr 8, 2025
Closed
45 tasks
@Shouren
Shouren deleted the fix/k8s-client-for-scheduler branch April 15, 2025 03:08
@archlitchi archlitchi mentioned this pull request May 6, 2025
Merged
11 tasks
archlitchi added a commit that referenced this pull request May 6, 2025
@archlitchi @HJJ256
@Nimbus318 @Shouren @flpanbin @yxxhero @dependabot
Release v2.5 (#1034)
6efdbbc 
* Fix: Update handling of version strings in Helm template and helpers.tpl (#845)

* Update condition to include regexReplaceAll for outputting proper numbers from minor versions

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update condition

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update definition of strippedKubeVersion to handle variety of version numbering systems

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update job-createSecret.yaml

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update job-patchWebhook.yaml

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

---------

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update libvgpu.so (#876)

* update libvgpu

Signed-off-by: limengxuan <391013634@qq.com>

* fix: disable passDeviceSpecsEnabled by default (#872)

Due to potential pod startup issues in certain environments,
set passDeviceSpecsEnabled to false by default.
This configuration can still be enabled via helm values
for environments that need it to handle runtime GPU access issues.

Signed-off-by: Nimbus318 <136771156+Nimbus318@users.noreply.github.com>

* fix: Remove the pkg/k8sutil/client.go and replace it with HAMi/pkg/util/client in pkg/scheduler/scheduler.go (#681)

Signed-off-by: Shouren Yang <yangshouren@gmail.com>

* fix: correct device filter initialization order (#857)

Ensure DevicePluginFilterDevice is initialized before FilterDeviceToRegister.
This fixes the initialization sequence to make device filtering work.

Signed-off-by: Nimbus318 <136771156+Nimbus318@users.noreply.github.com>

* fix parseNvidiaNumaInfo index out of range (#889)

Signed-off-by: bin <bin.pan@daocloud.io>

* fix conflict

Signed-off-by: limengxuan <mengxuan.li@dynamia.ai>

* fix conflict

Signed-off-by: limengxuan <mengxuan.li@dynamia.ai>

* fix ubuntu base image (#944)

Signed-off-by: bin <bin.pan@daocloud.io>

* fix: Add error handling for nvml.Init in NvidiaDevicePlugin (#982)

Signed-off-by: yxxhero <aiopsclub@163.com>

* Bump golang.org/x/net from 0.26.0 to 0.33.0

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.26.0 to 0.33.0.
- [Commits](golang/net@v0.26.0...v0.33.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix Dockerfile to make CI pass (#846)

* update dockerfile

Signed-off-by: limengxuan <391013634@qq.com>

---------

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>
Signed-off-by: limengxuan <391013634@qq.com>
Signed-off-by: Nimbus318 <136771156+Nimbus318@users.noreply.github.com>
Signed-off-by: Shouren Yang <yangshouren@gmail.com>
Signed-off-by: bin <bin.pan@daocloud.io>
Signed-off-by: limengxuan <mengxuan.li@dynamia.ai>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Harsh Jaykumar Jalan <harshjalan27@yahoo.com>
Co-authored-by: 霓漠Nimbus <136771156+Nimbus318@users.noreply.github.com>
Co-authored-by: Shouren Yang <yangshouren@gmail.com>
Co-authored-by: bin.pan <bin.pan@daocloud.io>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
archlitchi added a commit that referenced this pull request May 6, 2025
@archlitchi @HJJ256
@Nimbus318 @Shouren @flpanbin @yxxhero @dependabot
Update tag to v2.5.1 (#1035)
b853527 
* Fix: Update handling of version strings in Helm template and helpers.tpl (#845)

* Update condition to include regexReplaceAll for outputting proper numbers from minor versions

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update condition

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update definition of strippedKubeVersion to handle variety of version numbering systems

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update job-createSecret.yaml

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update job-patchWebhook.yaml

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

---------

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update libvgpu.so (#876)

* update libvgpu

Signed-off-by: limengxuan <391013634@qq.com>

* fix: disable passDeviceSpecsEnabled by default (#872)

Due to potential pod startup issues in certain environments,
set passDeviceSpecsEnabled to false by default.
This configuration can still be enabled via helm values
for environments that need it to handle runtime GPU access issues.

Signed-off-by: Nimbus318 <136771156+Nimbus318@users.noreply.github.com>

* fix: Remove the pkg/k8sutil/client.go and replace it with HAMi/pkg/util/client in pkg/scheduler/scheduler.go (#681)

Signed-off-by: Shouren Yang <yangshouren@gmail.com>

* fix: correct device filter initialization order (#857)

Ensure DevicePluginFilterDevice is initialized before FilterDeviceToRegister.
This fixes the initialization sequence to make device filtering work.

Signed-off-by: Nimbus318 <136771156+Nimbus318@users.noreply.github.com>

* fix parseNvidiaNumaInfo index out of range (#889)

Signed-off-by: bin <bin.pan@daocloud.io>

* fix conflict

Signed-off-by: limengxuan <mengxuan.li@dynamia.ai>

* fix conflict

Signed-off-by: limengxuan <mengxuan.li@dynamia.ai>

* fix ubuntu base image (#944)

Signed-off-by: bin <bin.pan@daocloud.io>

* fix: Add error handling for nvml.Init in NvidiaDevicePlugin (#982)

Signed-off-by: yxxhero <aiopsclub@163.com>

* Bump golang.org/x/net from 0.26.0 to 0.33.0

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.26.0 to 0.33.0.
- [Commits](golang/net@v0.26.0...v0.33.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix Dockerfile to make CI pass (#846)

* update dockerfile

Signed-off-by: limengxuan <391013634@qq.com>

* update version

Signed-off-by: limengxuan <mengxuan.li@dynamia.ai>

---------

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>
Signed-off-by: limengxuan <391013634@qq.com>
Signed-off-by: Nimbus318 <136771156+Nimbus318@users.noreply.github.com>
Signed-off-by: Shouren Yang <yangshouren@gmail.com>
Signed-off-by: bin <bin.pan@daocloud.io>
Signed-off-by: limengxuan <mengxuan.li@dynamia.ai>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Harsh Jaykumar Jalan <harshjalan27@yahoo.com>
Co-authored-by: 霓漠Nimbus <136771156+Nimbus318@users.noreply.github.com>
Co-authored-by: Shouren Yang <yangshouren@gmail.com>
Co-authored-by: bin.pan <bin.pan@daocloud.io>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
archlitchi added a commit that referenced this pull request May 6, 2025
@archlitchi @HJJ256
@Nimbus318 @Shouren @flpanbin @yxxhero @dependabot @Rei1010
Release v2.5.1 - fix e2e workflow (#1037)
83b9b23 
* Fix: Update handling of version strings in Helm template and helpers.tpl (#845)

* Update condition to include regexReplaceAll for outputting proper numbers from minor versions

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update condition

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update definition of strippedKubeVersion to handle variety of version numbering systems

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update job-createSecret.yaml

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update job-patchWebhook.yaml

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

---------

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>

* Update libvgpu.so (#876)

* update libvgpu

Signed-off-by: limengxuan <391013634@qq.com>

* fix: disable passDeviceSpecsEnabled by default (#872)

Due to potential pod startup issues in certain environments,
set passDeviceSpecsEnabled to false by default.
This configuration can still be enabled via helm values
for environments that need it to handle runtime GPU access issues.

Signed-off-by: Nimbus318 <136771156+Nimbus318@users.noreply.github.com>

* fix: Remove the pkg/k8sutil/client.go and replace it with HAMi/pkg/util/client in pkg/scheduler/scheduler.go (#681)

Signed-off-by: Shouren Yang <yangshouren@gmail.com>

* fix: correct device filter initialization order (#857)

Ensure DevicePluginFilterDevice is initialized before FilterDeviceToRegister.
This fixes the initialization sequence to make device filtering work.

Signed-off-by: Nimbus318 <136771156+Nimbus318@users.noreply.github.com>

* fix parseNvidiaNumaInfo index out of range (#889)

Signed-off-by: bin <bin.pan@daocloud.io>

* fix conflict

Signed-off-by: limengxuan <mengxuan.li@dynamia.ai>

* fix conflict

Signed-off-by: limengxuan <mengxuan.li@dynamia.ai>

* fix ubuntu base image (#944)

Signed-off-by: bin <bin.pan@daocloud.io>

* fix: Add error handling for nvml.Init in NvidiaDevicePlugin (#982)

Signed-off-by: yxxhero <aiopsclub@163.com>

* Bump golang.org/x/net from 0.26.0 to 0.33.0

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.26.0 to 0.33.0.
- [Commits](golang/net@v0.26.0...v0.33.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix Dockerfile to make CI pass (#846)

* update dockerfile

Signed-off-by: limengxuan <391013634@qq.com>

* update version

Signed-off-by: limengxuan <mengxuan.li@dynamia.ai>

* Optimize E2E with pod status check (#847)

Signed-off-by: wen.rui <wen.rui@daocloud.io>

---------

Signed-off-by: HJJ256 <harshjalan27@yahoo.com>
Signed-off-by: limengxuan <391013634@qq.com>
Signed-off-by: Nimbus318 <136771156+Nimbus318@users.noreply.github.com>
Signed-off-by: Shouren Yang <yangshouren@gmail.com>
Signed-off-by: bin <bin.pan@daocloud.io>
Signed-off-by: limengxuan <mengxuan.li@dynamia.ai>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: wen.rui <wen.rui@daocloud.io>
Co-authored-by: Harsh Jaykumar Jalan <harshjalan27@yahoo.com>
Co-authored-by: 霓漠Nimbus <136771156+Nimbus318@users.noreply.github.com>
Co-authored-by: Shouren Yang <yangshouren@gmail.com>
Co-authored-by: bin.pan <bin.pan@daocloud.io>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Rei1010 <56469400+Rei1010@users.noreply.github.com>
@fishman fishman mentioned this pull request Jul 1, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@lengrongfu lengrongfu lengrongfu left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

kind/bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Shouren @lengrongfu @archlitchi @wawa0210