Fix potential non-determinism in SabreSwap rust code#10740
Conversation
In the SabreSwap rust module there were 2 potential sources of non-determinism that could cause variation in results even with a fixed seed, the ExtendedSet struct's nodes attribute and the decremented tracking when populating the extended set. Both were caused by the same root cause iterating over a HashMap object. Iteration order on a HashMap (or a HashSet) is dependent on the internal hash seed of the hasher and iteration order is explicitly not guaranteed. In the case of the decrementer it's unlikely to have any effect even if the iteration order is not guaranteed but it is switched to using an indexmap regarldess just out of best practice. But for the nodes attribute in the ExtendedSet struct there is a potential issue there because when the total_score() method is called the nodes are iterated over, the distance is looked up for each swap and then summed. As the distances are all floating point values the iteration order could result in different values being output. In both cases the `hashbrown::HashMap<K, V>` is changed to be an `indexmap::IndexMap<K, V, ahash::RandomState>` which will have deterministic iteration order (it uses insertion order).
|
One or more of the the following people are requested to review this:
|
jakelishman
left a comment
There was a problem hiding this comment.
Do you know if this actually fixes the non-determinism in the Sabre trials we were seeing? The floating-point argument in ExtendedSet::total_score thing certainly sounds like something we should squash - I'm not sure why I didn't see it at the time.
At any rate, all the changes look fine and good to me.
|
It's still running now (I should have trimmed the data set) but looking at the streaming results it still looks like there are fluctuations in the output swap count as the trial counts increase. I was optimistically hoping it would be this simple, but it looks like we'll have to do more digging. |
TBH, it's probably pretty unlikely to be an issue in practice as the distance matrix should typically be small whole numbers as generated by rustworkx's |
In the SabreSwap rust module there were 2 potential sources of non-determinism that could cause variation in results even with a fixed seed, the ExtendedSet struct's nodes attribute and the decremented tracking when populating the extended set. Both were caused by the same root cause iterating over a HashMap object. Iteration order on a HashMap (or a HashSet) is dependent on the internal hash seed of the hasher and iteration order is explicitly not guaranteed. In the case of the decrementer it's unlikely to have any effect even if the iteration order is not guaranteed but it is switched to using an indexmap regarldess just out of best practice. But for the nodes attribute in the ExtendedSet struct there is a potential issue there because when the total_score() method is called the nodes are iterated over, the distance is looked up for each swap and then summed. As the distances are all floating point values the iteration order could result in different values being output. In both cases the `hashbrown::HashMap<K, V>` is changed to be an `indexmap::IndexMap<K, V, ahash::RandomState>` which will have deterministic iteration order (it uses insertion order).
In the SabreSwap rust module there were 2 potential sources of non-determinism that could cause variation in results even with a fixed seed, the ExtendedSet struct's nodes attribute and the decremented tracking when populating the extended set. Both were caused by the same root cause iterating over a HashMap object. Iteration order on a HashMap (or a HashSet) is dependent on the internal hash seed of the hasher and iteration order is explicitly not guaranteed. In the case of the decrementer it's unlikely to have any effect even if the iteration order is not guaranteed but it is switched to using an indexmap regarldess just out of best practice. But for the nodes attribute in the ExtendedSet struct there is a potential issue there because when the total_score() method is called the nodes are iterated over, the distance is looked up for each swap and then summed. As the distances are all floating point values the iteration order could result in different values being output. In both cases the `hashbrown::HashMap<K, V>` is changed to be an `indexmap::IndexMap<K, V, ahash::RandomState>` which will have deterministic iteration order (it uses insertion order).
Summary
In the SabreSwap rust module there were 2 potential sources of non-determinism that could cause variation in results even with a fixed seed, the ExtendedSet struct's nodes attribute and the decremented tracking when populating the extended set. Both were caused by the same root cause iterating over a HashMap object. Iteration order on a HashMap (or a HashSet) is dependent on the internal hash seed of the hasher and iteration order is explicitly not guaranteed. In the case of the decrementer it's unlikely to have any effect even if the iteration order is not guaranteed but it is switched to using an indexmap regarldess just out of best practice. But for the nodes attribute in the ExtendedSet struct there is a potential issue there because when the total_score() method is called the nodes are iterated over, the distance is looked up for each swap and then summed. As the distances are all floating point values the iteration order could result in different values being output. In both cases the
hashbrown::HashMap<K, V>is changed to be anindexmap::IndexMap<K, V, ahash::RandomState>which will have deterministic iteration order (it uses insertion order).Details and comments