Skip to content

Test cases for Fallbacks container #280

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
j-kuehn wants to merge 5 commits into from
Closed

Test cases for Fallbacks container #280

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f2b12f1
add unit tests for Fallbacks container
j-kuehn Jul 8, 2021
f47f9c2
fix ActiveChildReset test / make it fail
v4hn Jul 8, 2021
b98ee7f
Fix test Fallbacks.ComputeFirstSuccessfulStageOnly
rhaschke Jul 13, 2021
514021c
Fix test Fallback.ActiveChildReset
rhaschke Jul 13, 2021
414964a
disable ConnectStageInsideFallbacks
v4hn Jul 26, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions core/src/container.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -819,6 +819,9 @@ bool Fallbacks::canCompute() const {
if (child->canCompute())
return true;

if (!Stage::solutions().empty())
return false; // we are done as soon as a child has found solutions

Copy link
Contributor

@v4hn v4hn Jul 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This patch does not make any sense to me.

If the generator in ComputeFirstSuccessfulStageOnly would spawn multiple solutions, I would still expect a solution from the first child for each of the inputs! Otherwise you prune too many computation branches and loose solutions.

// active child failed, continue with next
auto next = child->it();
++next;
Expand Down
50 changes: 50 additions & 0 deletions core/test/test_container.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ void append(ContainerBase& c, const std::initializer_list<StageType>& types) {
}
}
}
constexpr double INF = std::numeric_limits<double>::infinity();

class NamedStage : public GeneratorMockup
{
Expand Down Expand Up @@ -685,3 +686,52 @@ TEST(Fallback, failing) {
EXPECT_FALSE(t.plan());
EXPECT_EQ(t.solutions().size(), 0u);
}

TEST(Fallback, ConnectStageInsideFallbacks) {
Copy link
Contributor

@rhaschke rhaschke Jul 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fails for the following reason:

  1. The Generator spawns a solution, which is correctly forwarded to the start pull interface of Connect.
  2. However, because there was no state pushed into the end pull interface of Connect, canCompute() returns false
    and the Fallbacks container forwards to the next child (which doesn't exist).

This can only be solved, if we drop the simple linear execution order and switch to topological execution order.

Copy link
Contributor Author

@j-kuehn j-kuehn Jul 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds reasonable to disable this test for now.

resetMockupIds();
Task t;
t.setRobotModel(getModel());

t.add(std::make_unique<GeneratorMockup>());

auto fallbacks = std::make_unique<Fallbacks>("Fallbacks");
fallbacks->add(std::make_unique<ConnectMockup>());
t.add(std::move(fallbacks));

t.add(std::make_unique<GeneratorMockup>());

EXPECT_TRUE(t.plan());
EXPECT_EQ(t.numSolutions(), 1u);
}

TEST(Fallback, ComputeFirstSuccessfulStageOnly) {
resetMockupIds();
Task t;
t.setRobotModel(getModel());

t.add(std::make_unique<GeneratorMockup>());

auto fallbacks = std::make_unique<Fallbacks>("Fallbacks");
fallbacks->add(std::make_unique<ForwardMockup>(PredefinedCosts::constant(0.0)));
fallbacks->add(std::make_unique<ForwardMockup>(PredefinedCosts::constant(0.0)));
t.add(std::move(fallbacks));

EXPECT_TRUE(t.plan());
EXPECT_EQ(t.numSolutions(), 1u);
}

TEST(Fallback, ActiveChildReset) {
resetMockupIds();
Task t;
t.setRobotModel(getModel());

t.add(std::make_unique<GeneratorMockup>(PredefinedCosts{ { 0.0, INF, 0.0 } }));

auto fallbacks = std::make_unique<Fallbacks>("Fallbacks");
fallbacks->add(std::make_unique<ForwardMockup>(PredefinedCosts::constant(0.0)));
fallbacks->add(std::make_unique<ForwardMockup>(PredefinedCosts::constant(0.0)));
t.add(std::move(fallbacks));

EXPECT_TRUE(t.plan());
EXPECT_EQ(t.numSolutions(), 4u);
Copy link
Contributor

@v4hn v4hn Jul 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depending on the preferred semantics (whether fallback alternatives should be computed at all), the test is still not correct: If no alternatives should be considered you would have to check for two solutions where both should come from the first ForwardMockup.

Let's wait for Robert's response before modifying this.

Copy link
Contributor

@rhaschke rhaschke Jul 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't yet fully understand the purpose of this test. I would expect two solutions here, contributed by the first child. However, this was not yet tested here. I pushed a corresponding fixup.

Copy link
Contributor

@rhaschke rhaschke Jul 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be another test case, where the external generator is failing first before succeeding.
Currently, I would expect this test to fail (with a solution of the wrong, second child).

Copy link
Contributor Author

@j-kuehn j-kuehn Jul 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test suggested that after computing solutions for the second spawned solution from the generator, the active_child_ would not be reset to the first child for computing the third solution. Before your two commits, this led to the first child being ignored for the third spawned generator solution.

As @v4hn pointed out, this test was not consistent with the proposed functionality in ComputeFirstSuccessfulStageOnly, but rather tried to test the status quo. But even after your fix for the ActiveChildReset test, I would have expected the test to fail, because as I understand it, the active_child_ is pointing to the second child after the second spawned generator solution. So for the third solution, I expected the container to use the second child for computation. However, the test passes, so it is actually using the first child. I'm not sure how your commit fixed this.

I just tried adding another test as you suggested, where the external generator is failing first before succeeding:

std::make_unique<GeneratorMockup>(PredefinedCosts{ { INF, 0.0 } }));

However, this yields 0 solutions and I'm not sure why.

}