Filter out keys that cannot be statistically modeled

[lajohn4747]

CU-860pbbbng
resolves #286

Filter out columns from detection method that are not statistically modeled:
PII fields
Primary keys
Any other IDs
Text Data

[sdv-team]

Task linked: CU-860pbbbng SDMetrics - Detection metrics should only use statistically modeled columns (filter out the rest) #286

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (87790f2) 77.12% compared to head (3dd9750) 77.30%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #525      +/-   ##
==========================================
+ Coverage   77.12%   77.30%   +0.18%     
==========================================
  Files          93       93              
  Lines        3488     3507      +19     
==========================================
+ Hits         2690     2711      +21     
+ Misses        798      796       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[R-Palazzo]

Maybe you can do something like:

if sdtype not in [numerical, datetime, categorical]:
    drop_columns.append(column)

And:

if column in column_keys:
      drop_columns.append(column)

With column_keys a list of the primary, foreign and alternate keys

[lajohn4747]

if column in column_keys:
      drop_columns.append(column)

With column_keys a list of the primary, foreign and alternate keys

@R-Palazzo would the foreign keys and alternate keys be labeled in metadata?

[R-Palazzo]

if column in column_keys:
      drop_columns.append(column)

With column_keys a list of the primary, foreign and alternate keys

@R-Palazzo would the foreign keys and alternate keys be labeled in metadata?

Yes if they exist you can access the alternate keys through metadata['alternate_keys'] and find the foreign key with the relationships

[R-Palazzo]

Looks good! Just let one suggestion

[amontanez24]

LGTM!