Fix dedupe finder performance issue on looking up table size

[eileenmcnaughton]

Overview

Fix dedupe finder performance issue on looking up table size

Before

Our dedupe queries seem to be particularly slow, rule dependent, and I found that the the issue was slow SELECT COUNT(*) FROM civicrm_address etc queries. We recently fixed some of these in other place (SnapShot, an upgrade script). At the time I was under the impression the technique used here also had performance issues - ie

      SELECT TABLE_ROWS as row_count, TABLE_NAME as table_name FROM information_schema.TABLES WHERE
      AND TABLE_SCHEMA = DATABASE();

However, I tested that query and it is not slow

414 rows in set, 1 warning (0.103 sec)

And is quicker for less tables.

I also traced back the origin of the code comment to that effect & nothing in there seems to back it up

#8636
https://issues.civicrm.org/jira/browse/CRM-19027

After

Use of a faster function, which we can also use from those other places

Technical Details

This is very hard to access under r-run as it deals with the ordering of queries for a perceived performance improvement - however I could step through it with CRM_Contact_Import_Parser_ContactTest.testIgnoreLocationTypeId and per the screenshots you can see that it re-ordered the array to put the civicrm_address query first based on the table size being smaller

Original sort order

See in the variable $tableQueries the query involving the contact table is first

It decides the the table lengths

See in the $cachedResults there are more rows in the contact table

**And it re-orders based on that **

See in the variable $tableQueries the query involving the (shorter) address table is now first

Comments

[civibot]

(Standard links)

• If you are reviewing this pull-request, you may wish to consult the test sites and the Review Standards (long template, short template).

[eileenmcnaughton]

Hmm - well if that is the test that hits it it is not where I expected ....

CRM_Contact_Import_Parser_ContactTest.testIgnoreLocationTypeId

[eileenmcnaughton]

we've been running this in production for several weeks now

[colemanw]

I'm happy with this between the r-run in production and the test coverage.

[eileenmcnaughton]

test this please