Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add logPersonalInformation to recognizers #3296

Merged
merged 85 commits into from
Mar 10, 2021

Conversation

Zerryth
Copy link
Contributor

@Zerryth Zerryth commented Feb 10, 2021

Fixes #3295

Description

  • Currently adaptive recognizers can log telemetry, however they always log personally identifiable information (PII)
  • PR will add enable ability to toggle logging PII on/off

Specific Changes

test

  • Add AdaptiveRecognizer class that has:
    • logPersonalInformation flag
    • fillRecognizerResultTelemetryProperties override that takes into account the logPii flag when creating telemetry properties
  • Per pattern in JS recognizers, add AdaptiveRecognizerConfiguration interface
    • Will only define optional BoolExpression logPersonalInformation member

Context

See microsoft/botbuilder-dotnet#5066 (comment)

Testing

This PR is 90% tests. Generally each recognizer tested 3 scenarios:

  • Log PII = true
  • Log PII = false
  • Log PII is false by default

@coveralls
Copy link

coveralls commented Feb 10, 2021

Pull Request Test Coverage Report for Build 638064761

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 38 unchanged lines in 7 files lost coverage.
  • Overall coverage decreased (-0.03%) to 85.095%

Files with Coverage Reduction New Missed Lines %
libraries/botbuilder-ai/src/qnaMakerRecognizer.ts 1 66.2%
libraries/botbuilder-dialogs-adaptive/src/recognizers/crossTrainedRecognizerSet.ts 1 96.81%
libraries/botbuilder-dialogs-adaptive/src/recognizers/entityRecognizers/entityRecognizer.ts 1 92.0%
libraries/botbuilder-dialogs-adaptive/src/recognizers/recognizerSet.ts 2 91.25%
libraries/botbuilder-dialogs-adaptive/src/recognizers/multiLanguageRecognizer.ts 4 78.85%
libraries/botbuilder-dialogs/src/recognizer.ts 9 60.34%
libraries/botbuilder-ai-orchestrator/src/orchestratorAdaptiveRecognizer.ts 20 61.65%
Totals Coverage Status
Change from base Build 637952007: -0.03%
Covered Lines: 18749
Relevant Lines: 20989

💛 - Coveralls

@@ -63,13 +64,15 @@ export class RecognizerSet extends Recognizer implements RecognizerSetConfigurat

// merge intents
for (const intentName in result.intents) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you're here - can we rewrite this to use for...of instead of for...in? It's generally a more sane option.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An example might be:

for (const [intentName, intent] of Object.entries(result.intents)) {
  ...
}

Copy link
Contributor Author

@Zerryth Zerryth Feb 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactored to use for-of loops! Also renaming the variables to what you specified really helps clarify what parts are going where, as it gets confusing with the class name of an intent is IntentScore, which includes more than just the score @_@...


Although separately, going through this recognizer for the refactor, it merges the RecognizerResult's of multiple recognizers in a really bizarre manner:
RecognizerResult

  • Text: whatever is the last RecognizerResult in the array of results from multiple recognizers registered, will "win" the spot of Text in the merged RecognizerResult
  • Intent: In C# the method doc includes this "Intents will be merged by picking the intent with the MaxScore.". However they both actually just push all intents recognized from each RecognizerResult in the array of results into the merged RecognizerResult
  • Entities: Contrast with the doc in C# regarding entities "Entities are merged as a simple union of all of the Entities." -- this is the actual behavior what's happening (all entities recognized by all recognizers are included in the merged RecognizerResult)

The behavior is the same in both C# and JS, however it seems buggy in both, if we're going off by what the C# doc says vs. what is actually happening...iunno. I don't know enough about the design decision around RecognizerSet to make any behavioral changes.

So refactored, keeping behavior the same--just what's happening in this recognizer seems odd to begin with

@Zerryth
Copy link
Contributor Author

Zerryth commented Mar 10, 2021

Phew. Ok, updated with round of requested changes!

Ready for a another review @joshgummersall whenever you're available

Copy link
Contributor

@joshgummersall joshgummersall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

@joshgummersall joshgummersall merged commit e0d9198 into main Mar 10, 2021
@joshgummersall joshgummersall deleted the zerryth/logpii-in-recognizers branch March 10, 2021 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

port: Add LogPersonalInformation to Recognizers & Correct TopScoreIntent (#5066)
3 participants