One OrderedSet test fails

[felix91gr]

OS: Linux, Ubuntu 16.04

Commands ran:

felix@felix-X550LD ~/D/P/A/commandant> rm -rf .build/
felix@felix-X550LD ~/D/P/A/commandant> git checkout 0.13.0 
Note: checking out '0.13.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 7f29606... Release 0.13.0
felix@felix-X550LD ~/D/P/A/commandant> swift test

Failed test output:

Test Case 'XCTestCase.OrderedSet, should preserve the order of the given input' started at 2018-04-09 22:11:17.477
/home/felix/Documents/Programming/AwesomeSystemSwift/commandant/Tests/CommandantTests/OrderedSetSpec.swift:17: error: XCTestCase.OrderedSet, should preserve the order of the given input : failed - expected to not equal <acb>, got <acb>

Test Case 'XCTestCase.OrderedSet, should preserve the order of the given input' failed (0.0 seconds)

What's the output on that line under Mac OS? Maybe it's failing on both ends.

[felix91gr]

I've just done more testing:

I changed the failing test to this, to check if it failed in the same way for all Sets instanced with that array:

it("should preserve the order of the given input") {
	let input = ["a", "c", "b", "a"]
	expect(Set(input).joined()) != "acb"
	expect(Set(input).joined()) != "acb"
	expect(Set(input).joined()) != "acb"
	expect(Set(input).joined()) != "acb"
	expect(Set(input).joined()) != "acb"
	expect(Set(input).joined()) != "acb"
	expect(Set(input).joined()) != "acb"
	expect(Set(input).joined()) != "acb"
	expect(Set(input).joined()) != "acb"
	expect(Set(input).joined()) != "acb"
	expect(OrderedSet(input).joined()) == "acb"
}

Then I ran swift test repeatedly many times. The result was something like this:

• 0 failures
• 0 failures
• 0 failures
• 10 failures
• 0 failures

This shows that the test isn't deterministic. Since Set's implementation uses hash tables, some non-determinisitic hashtable attack protection must be in play.

I dunno what can be done about this test. The OrderedSet part will always be true, but the Set part depends on luck. Maybe the Set part isn't necessary? But I dunno, because I didn't make the test.

[ikesyo]

Thanks for filling the issue! Your explanation totally makes sense, so let's remove the test.

[felix91gr]

No problem ;)