pkg/asset/filefetcher: Only load files on demand #497
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
openshift-ci-robot
added
size/L
wking
force-pushed
the
load-assets-on-demand
branch
from
29bad4e to
1301358
Compare
Avoid:
$ bin/openshift-install cluster
FATAL Error executing openshift-install: open tests/smoke/vendor/github.com/prometheus/procfs/fixtures/26231/fd/0: no such file or directory
as the old implementation attempts to walk the whole directory and
hits:
$ ls -l tests/smoke/vendor/github.com/prometheus/procfs/fixtures/26231/fd/
total 0
lrwxrwxrwx. 1 trking trking 24 Oct 5 01:26 0 -> ../../symlinktargets/abc
lrwxrwxrwx. 1 trking trking 24 Oct 5 01:26 1 -> ../../symlinktargets/def
lrwxrwxrwx. 1 trking trking 24 Oct 5 01:26 10 -> ../../symlinktargets/xyz
lrwxrwxrwx. 1 trking trking 24 Oct 5 01:26 2 -> ../../symlinktargets/ghi
lrwxrwxrwx. 1 trking trking 24 Oct 5 01:26 3 -> ../../symlinktargets/uvw
With this commit, we only load files from the disk when someone asks
for them.
I've adjusted the unit tests a bit because:
* ioutil.ReadFile returns errors like:
read /: is a directory
for directories. There does not appear to be an analog to
os.IsNotExist() for this condition, so instead of checking for it in
the tests, I've just dropped the empty-string input cases. If we
break something and call FetchByName on an empty string, we want to
error out, and that error message is appropriately descriptive
already.
* Globs are not as precise as regular expressions, so our glob would
match master-1x.ign and similar which the previous regexp excluded.
But loading a few extra files doesn't seem like that big a deal, and
folks adding files with names like that seems unlikely.
wking
force-pushed
the
load-assets-on-demand
branch
from
1301358 to
edcd705
Compare
Member
Author
|
I've pushed 1301358 -> edcd705 updating the unit tests to keep up. |
yifan-gu
reviewed
Oct 19, 2018
| // Load returns the master ignitions from disk. | ||
| func (a *Master) Load(f asset.FileFetcher) (found bool, err error) { | ||
| fileList := f.FetchByPattern(regexp.MustCompile(`^(master-(0|([1-9]\d*))\.ign)$`)) | ||
| fileList, err := f.FetchByPattern("master-[0-9]*.ign") |
Contributor
There was a problem hiding this comment.
There was a discussion that this matches more than we need, e.g. it matches master-0a.ign, but I guess that's fine.
Member
Author
There was a problem hiding this comment.
There was a discussion that this matches more than we need, e.g. it matches
master-0a.ign...
Yeah, I waved my hands around that at the end of the edcd705 commit message ;).
Contributor
|
/lgtm |
Contributor
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: abhinavdahiya, wking, yifan-gu The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Avoid:
as the old implementation attempts to walk the whole directory and hits:
With this commit, we only load files from the disk when someone asks for them.
closes #494