Skip to content

Don't check fail on missing lineage cache entry #3861

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ericl merged 7 commits into from
Feb 5, 2019
Merged

Don't check fail on missing lineage cache entry #3861

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
2f0baa6
fix linea
ericl Jan 25, 2019
a5c66fd
update
ericl Jan 26, 2019
58904ee
Update lineage_cache.cc
ericl Jan 26, 2019
eb3cbff
wip
ericl Jan 28, 2019
2a3dc9d
fix
ericl Jan 28, 2019
c60def3
fix
ericl Jan 28, 2019
798e724
rename fatal
ericl Jan 28, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions src/ray/raylet/lineage_cache.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -335,8 +335,11 @@ Lineage LineageCache::GetUncommittedLineage(const TaskID &task_id,
// already explicitly forwarded to this node before.
if (uncommitted_lineage.GetEntries().empty()) {
auto entry = lineage_.GetEntry(task_id);
RAY_CHECK(entry);
RAY_CHECK(uncommitted_lineage.SetEntry(entry->TaskData(), entry->GetStatus()));
if (entry) {
RAY_CHECK(uncommitted_lineage.SetEntry(entry->TaskData(), entry->GetStatus()));
} else {
RAY_LOG(ERROR) << "No lineage cache entry found for task " << task_id;
Copy link
Contributor

@zhijunfu zhijunfu Jan 26, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to add a comment on under what conditions this entry doesn't exist.

Copy link
Contributor Author

@ericl ericl Jan 26, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Added a pointer back to the issue.

It's not super clear what is evicting the lineage: perhaps some race condition on a task getting rescheduled, or the task succeeding but the node failing after that.

}
}
return uncommitted_lineage;
}
Expand Down