Speed performance difference when interactions built with wildcard

[salayatana66]

I've observed speed differences when interactions are explicitly constructed wrt. to using a wildcard.

For example

time for v in {1..1000000} ; do printf "1.0 |a q=2\n"; done | vw -q ::
with vw 8.5.0 takes on my machine ~1.4mins

while

time for v in {1..1000000} ; do printf "1.0 |a q=2\n"; done | vw -q aa
takes about 25s.

Looking at vowpalwabbit/interactions.cc, if I understand correctly, under -q:: vw creates interactions also for namespaces it might not see in the data (I guess there are 92 such namespaces) and this creates an overhead in processing each example. I'm a recent user but some colleagues who have used vw for a few years were surprised, I guess they were operating under the assumption that :: is expanded only to namespaces seen in each example; would it make sense to state / warn a potential impact on training speed of :: in the documentation?

[JohnLangford]

This is a known issue that I'd like to fix. In essence, we need to shift from a globally defined set of interactions to a per-example set of interactions which are efficiently extracted from the globally defined set at parse time.

Go ahead and add a comment in the wiki.

[salayatana66]

Added a comment in the wiki in the section about interactions.

[olgavrou]

This is now fixed in master, interactions are generated as the namespaces are found instead of pre-calculated. I'll update the wiki comment