Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Fail fast option in ProcessAll, so that file write errors can be fatal. #906

Merged
merged 4 commits into from
Jul 18, 2020

Conversation

gfr10598
Copy link
Contributor

@gfr10598 gfr10598 commented Jul 15, 2020

When we move to GCS json output, we need to be able to terminate parsing if there is a write error.

This also migrates some APIs to use stiface.Client to allow future injection of fakes.


This change is Reviewable

@gfr10598 gfr10598 marked this pull request as draft July 15, 2020 16:01
@coveralls
Copy link
Collaborator

coveralls commented Jul 15, 2020

Pull Request Test Coverage Report for Build 5952

  • 1 of 3 (33.33%) changed or added relevant lines in 1 file are covered.
  • 5 unchanged lines in 2 files lost coverage.
  • Overall coverage decreased (-0.07%) to 66.816%

Changes Missing Coverage Covered Lines Changed/Added Lines %
task/task.go 1 3 33.33%
Files with Coverage Reduction New Missed Lines %
task/task.go 1 77.12%
active/active.go 4 87.64%
Totals Coverage Status
Change from base Build 5950: -0.07%
Covered Lines: 3129
Relevant Lines: 4683

💛 - Coveralls

@gfr10598 gfr10598 requested a review from stephen-soltesz July 17, 2020 20:53
@gfr10598 gfr10598 marked this pull request as ready for review July 17, 2020 20:54
Copy link
Contributor

@stephen-soltesz stephen-soltesz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: - with one blocking comment.

Reviewed 6 of 7 files at r1.
Reviewable status: :shipit: complete! 1 of 1 approvals obtained (waiting on @gfr10598 and @stephen-soltesz)


storage/storage.go, line 61 at r1 (raw file):

			return nil, false, err
		} else if strings.Contains(err.Error(), "unexpected EOF") {
			// We are seeing these metrics, but logs at 113

I don't understand this comment.


storage/storage.go, line 365 at r1 (raw file):

		return rdr, 0, err
	}
	attr, err := obj.Attrs(ctx)

I added this comment in the draft PR. #902

This is a bug. attr is not guaranteed to be valid when err != nil.

Copy link
Contributor Author

@gfr10598 gfr10598 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 1 approvals obtained (waiting on @gfr10598 and @stephen-soltesz)


storage/storage.go, line 61 at r1 (raw file):

Previously, stephen-soltesz (Stephen Soltesz) wrote…

I don't understand this comment.

I think I was mistaken. I've improved the comments near line 113, and change next to nextHeader here. I think I may have been confused because the metric labels weren't distinct before.

Copy link
Contributor Author

@gfr10598 gfr10598 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 1 approvals obtained (waiting on @stephen-soltesz)


storage/storage.go, line 365 at r1 (raw file):

Previously, stephen-soltesz (Stephen Soltesz) wrote…

I added this comment in the draft PR. #902

This is a bug. attr is not guaranteed to be valid when err != nil.

Done.

@gfr10598 gfr10598 merged commit 037a05e into master Jul 18, 2020
@gfr10598 gfr10598 deleted the failfast branch July 18, 2020 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants