Detect overlapping detections

[sgiehl]

Description:

It seems like we have introduced some overlapping detections recently.
There are various detections that work correctly, when they are parsed by a specific client parser, but when parsed with all parsers another parser overwrites the results.
I've added some tests in this PR that use the fixtures used for specific parsers to parse them fully and check if the result is still the same.
There are currently 31 failures, where that doesn't match anymore.

I think we should try to get rid of overlapping detections. A user agent should either be detected as a browser or a mobile app. But not both.

One of the problematic detections seems to be this one:

device-detector/regexes/client/mobile_apps.yml

Lines 2480 to 2483 in c49f7e9

	# Electron generic apps
	- regex: ' (?!(?:AppleWebKit|brave))([a-z0-9]*)(?:-desktop|-electron-app)?/(\d+\.[\d.]+).*Electron/'
	name: '$1'
	version: '$2'

We might need to rethink the order in which the parsers are added, so such generic matches are done more later.
On the other side could every parser contain generic detections, which might overwrite other results...

Or would it make sense to somehow mark generic regexes so they are only used if nothing else matches?

@liviuconcioiu @sanchezzzhak Any thoughts / ideas on that ?

refs #7668

Review

• Functional review done
• Potential edge cases thought about (behavior of the code with strange input, with strange internal state or possible interactions with other Matomo subsystems)
• Usability review done (is anything maybe unclear or think about anything that would cause people to reach out to support)
• Security review done
• Wording review done
• Code review done
• Tests were added if useful/possible
• Reviewed for breaking changes
• Developer changelog updated if needed
• Documentation added if needed
• Existing documentation updated if needed

[sanchezzzhak]

I don't see any other way out but to add exceptions to the rule
fix for mobile-apps.yaml

# Electron generic apps
- regex:  ' (?!(?:AppleWebKit|brave|Franz|Mailspring|Notion|Basecamp|Evernote|catalyst|ramboxpro|BlueMail|BeakerBrowser|TweakStyle|Colibri|Polypane|VibeMate|(?:d|LT|Glass|Sushi|Flash|OhHai)Browser|Sizzy))([a-z0-9]*)(?:-desktop|-electron-app)?/(\d+\.[\d.]+).*Electron/'
  name: '$1'
  version: '$2'

fix for bot.yaml

# Generic detections
- regex: '[a-z0-9_-]*(?:(?<!cu|power[ _]|m[ _])bot(?![ _]TAB|[ _]?5[0-9]|[ _]Senior|[ _]Junior)|analyzer|appengine|archiver?|checker|collector|crawl|crawler|(?<!node-|uclient-|Mikrotik/\d\.[x\d] |electron-)fetch(?:er)?|indexer|inspector|monitor|(?<!Microsoft |banshee-)project(?!or)|(?<!Google Wap|Blue )proxy|research|resolver|robots|(?<!Cam)scanner|scraper|script|searcher|(?<!-)security|spider|study|transcoder|uptime|user[ _]?agent|validator)(?:[^a-z]|$)'
  name: 'Generic Bot'

[liviuconcioiu]

Now, I see why I had a hard time getting the Electron regex working :(.

@sanchezzzhak idea with the updated regexes is great.

[sanchezzzhak]

@sgiehl can I make the changes suggested above in this branch?

[sgiehl]

Feel free to make changes to that branch. I currently haven't time to.
Changing the regex should work. But we could also see if changing the priority here helps:

device-detector/DeviceDetector.php

Lines 198 to 203 in c2713ba

	$this->addClientParser(new FeedReader());
	$this->addClientParser(new MobileApp());
	$this->addClientParser(new MediaPlayer());
	$this->addClientParser(new PIM());
	$this->addClientParser(new Browser());
	$this->addClientParser(new Library());