Add support for custom prediction function in colab and jupyter notebook modes. #1842
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@jameswex please review. |
jameswex
reviewed
Feb 13, 2019
| @@ -57,6 +57,11 @@ You can use the What-If Tool to analyze a classification or regression | |||
| that takes TensorFlow Example or SequenceExample protos | |||
| (data points) as inputs directly in a jupyter or colab notebook. | |||
|
|
|||
| You can also use What-If-Tool with your own custom prediction function that takes | |||
| Tensorflow Example and produces predictions. In this mode, you can load any model | |||
There was a problem hiding this comment.
I would say "takes TensorFlow examples and produces predictions" (that way it includes Examples and SequenceExamples)
jameswex
reviewed
Feb 13, 2019
| @@ -181,14 +181,16 @@ class ServingBundle(object): | |||
| Predict API. | |||
| estimator: An estimator to use instead of calling an external model. | |||
| feature_spec: A feature spec for use with the estimator. | |||
| custom_predict_fn: A custom prediction function | |||
jameswex
reviewed
Feb 13, 2019
| @@ -336,14 +340,18 @@ def set_estimator_and_feature_spec(self, estimator, feature_spec): | |||
| Returns: | |||
| self, in order to enabled method chaining. | |||
| """ | |||
| # If custom function is set, remove it before setting estimator | |||
| self.delete('custom_predict_fn') | |||
| self.delete('compare_custom_predict_fn') | |||
There was a problem hiding this comment.
i think this should jsut delete the custom_predict_fn, not the compare one
There was a problem hiding this comment.
done, makes sense if you want to compare an estimator to a predict function.
jameswex
reviewed
Feb 13, 2019
| @@ -363,10 +371,77 @@ def set_compare_estimator_and_feature_spec(self, estimator, feature_spec): | |||
| Returns: | |||
| self, in order to enabled method chaining. | |||
| """ | |||
| # If custom function is set, remove it before setting estimator | |||
| self.delete('custom_predict_fn') | |||
There was a problem hiding this comment.
i think this should only delete the custom compare predict fn
jameswex
reviewed
Feb 13, 2019
| """ | ||
| # If estimator is set, remove it before setting predict_fn | ||
| self.delete('estimator_and_spec') | ||
| self.delete('compare_estimator_and_spec') |
jameswex
reviewed
Feb 13, 2019
| self, in order to enabled method chaining. | ||
| """ | ||
| # If estimator is set, remove it before setting predict_fn | ||
| self.delete('estimator_and_spec') |
jameswex
reviewed
Feb 13, 2019
|
Thanks James, I added all your comments. |
jameswex
approved these changes
Feb 13, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add custom prediction function support in colab and jupyter mode so it is easier to load non-TensorFlow models into WIT.
WITConfigBuilder now has a "set_custom_predict_fn" method to pass a python function for prediction purposes. Parts of notebook backend and inference_utils are updated to use the custom function when provided.
No UI changes.
Ran WIT Toxicity Text Model Comparison and WIT Model Comparison colabs with custom_predict_fn to verify the tool is getting inferences correctly.