Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip getVfsStats when file does not exist #1847

Conversation

dims
Copy link
Collaborator

@dims dims commented Dec 27, 2017

There are a lot of spurious exceptions in the kubernetes kubelet logs
like:
E1018 21:03:09.616581 22780 fs.go:332] Stat fs failed. Error: no such
file or directory

Since we know that calling syscall.Statfs will just fail when the path
does not exist, we should just skip making the call.

Fix for kubernetes/kubernetes#35062

@pineking
Copy link

pineking commented Jan 2, 2018

@dims when this is merged, which version of k8s can be fixed, v1.9.1 or master branch of k8s?

@dims
Copy link
Collaborator Author

dims commented Jan 2, 2018

@pineking neither one ... we will need this to get in first, then propose a PR to pull in a fresh cadvisor (to fix master) and then backport it to 1.9 branch

@dashpole
Copy link
Collaborator

dashpole commented Jan 2, 2018

Can we add a glog.V(4).Infof message specifying which mountpoint was not found if utils.FileExists returns false? This will allow us to find issues during testing.

@dashpole dashpole self-assigned this Jan 2, 2018
@dims
Copy link
Collaborator Author

dims commented Jan 2, 2018

@dashpole yep, i can do that

@dims dims force-pushed the skip-get-vfs-stats-when-mountpoint-does-not-exist branch from 5ed1d63 to 2448178 Compare January 2, 2018 22:23
@dims
Copy link
Collaborator Author

dims commented Jan 2, 2018

/test pull-cadvisor-e2e

@dims
Copy link
Collaborator Author

dims commented Jan 2, 2018

@dashpole something wrong with the build? local make looks file (http://paste.openstack.org/show/636661/)

@dashpole
Copy link
Collaborator

dashpole commented Jan 2, 2018

looks like it has been failing for a few days: https://k8s-testgrid.appspot.com/sig-node-cadvisor#cadvisor-kubelet. Not sure why.

Copy link
Collaborator

@dashpole dashpole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@dims
Copy link
Collaborator Author

dims commented Jan 2, 2018

Thanks @dashpole

@dashpole
Copy link
Collaborator

dashpole commented Jan 2, 2018

Can you regenerate the assets? I think the change from 2017->2018 made the tests fail (happened in k/k as well :)
Running ./build/jenkins_e2e.sh seems to do the trick

@dashpole
Copy link
Collaborator

dashpole commented Jan 2, 2018

Classic new year's bugs

There are a lot of spurious exceptions in the kubernetes kubelet logs
like:
E1018 21:03:09.616581   22780 fs.go:332] Stat fs failed. Error: no such
file or directory

Since we know that calling syscall.Statfs will just fail when the path
does not exist, we should just skip making the call.

NOTE: fixing 2017->2018 problems in build by running `./build/jenkins_e2e.sh`
@dims dims force-pushed the skip-get-vfs-stats-when-mountpoint-does-not-exist branch from 2448178 to 5f8eea9 Compare January 2, 2018 22:53
@dims
Copy link
Collaborator Author

dims commented Jan 2, 2018

:)

@dashpole dashpole merged commit aacb48c into google:master Jan 2, 2018
openshift-merge-robot added a commit to openshift/origin that referenced this pull request Jan 20, 2018
Automatic merge from submit-queue.

UPSTREAM: google/cadvisor: 1847: Skip getVfsStats when file does not exist

google/cadvisor#1847

xref https://bugzilla.redhat.com/show_bug.cgi?id=1511576

@eparis @derekwaynecarr 

**Reviewer note**
Drops two copyright year changes included in the upstream commit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants