Skip to content

Conversation

@kanpuriyanawab
Copy link
Collaborator

@kanpuriyanawab kanpuriyanawab commented Jan 16, 2023

This PR adds AlbertClassifier class to keras_nlp, letting the user instantiate the classifier with AlbertaBackbone, out-of-the-box.

Closes #660 !

@kanpuriyanawab
Copy link
Collaborator Author

kanpuriyanawab commented Jan 17, 2023

@jbischof This PR is ready for review.

Please consider following points while reviewing

  1. Albert Presets are not implemented yet, so in classproperty from_preset() of AlbertClassifier, I've raised a NotImplementedError(), please let me know your views on it!
  2. The Tests pass locally.

@kanpuriyanawab kanpuriyanawab marked this pull request as ready for review January 17, 2023 05:11
@kanpuriyanawab
Copy link
Collaborator Author

kanpuriyanawab commented Jan 17, 2023

@jbischof @mattdangerw This PR is ready for review as well !

@kanpuriyanawab
Copy link
Collaborator Author

kanpuriyanawab commented Jan 17, 2023

CI is green @jbischof

@mattdangerw mattdangerw self-requested a review January 18, 2023 03:07
Copy link
Member

@mattdangerw mattdangerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Left a few comments, mostly because we have made some changes to base classes here.

@kanpuriyanawab
Copy link
Collaborator Author

kanpuriyanawab commented Jan 22, 2023

@mattdangerw @jbischof
Thanks for the comments ! Task has simplified the usage of kerasNLP even more !

Couple of notes about following commit.

  1. Albert doesn't have classifier presets yet, thus I had to remove usage of classifier_presets from albert_classifier.py
  2. when I am running tests for albert_presets_test.py , all are getting skipped on local

@kanpuriyanawab
Copy link
Collaborator Author

@jbischof This PR is ready for review.

Copy link
Member

@mattdangerw mattdangerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Just some small issues. There are some docs changes we will need, but I think we can leave that to a follow up.

You still have some formatting issues, make sure to run our formatting script.

Examples:
```python
# Call classifier on the inputs.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will need to add some more documentation for overall usages here, see BERT as an example -> https://github.com/keras-team/keras-nlp/blob/master/keras_nlp/models/bert/bert_classifier.py#L55-L161

But I'm also OK with this being a follow up, we have some work to do on our class level docstrings anyway.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's just do this one as a follow up, I've opened #688. Other comments still apply!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made current docstrings alone with Bert classifier, once the pr merges and we have clear cut idea on how the docstrings for all models are to be restructured, I can put a follow up PR.

@kanpuriyanawab
Copy link
Collaborator Author

Still WIP

@kanpuriyanawab
Copy link
Collaborator Author

kanpuriyanawab commented Jan 28, 2023

merging with master messed up albert_presets.py changes. fixed it. Ready for review @mattdangerw @jbischof

I'm not sure why nightly is fialing in CI

@kanpuriyanawab
Copy link
Collaborator Author

CI is almost green.
Not sure why nightly is failing @mattdangerw @jbischof

@mattdangerw
Copy link
Member

Thanks so much @shivance ! Not sure what is going on with nightly, but I suspect it is unrelated to your PR.

Nightly testing is important for us to have, but it will definitely surface errors in the upstream tf release as often as our own. I will take a closer look today. Thanks!

@kanpuriyanawab
Copy link
Collaborator Author

@mattdangerw How does nightly differ from dev and stable release?

@mattdangerw
Copy link
Member

@shivance only difference is that nightly will test against the tf-nightly and tensorflow-text-nightly packages. Which are nightly "unstable" releases of our upstream dependencies. Often what will happen is the tf-nightly package will break our testing because of a bug in a recent tf change.

@kanpuriyanawab
Copy link
Collaborator Author

Got it ! Thanks ! @mattdangerw
By the way, I'm really interested in GSoC this year at KerasNLP ( I love working here and luckily kerasnlp participate in gsoc last year under tensorflow). Will it participate this year too?

What are potential projects ? (My guess is, ideally it should align with the current roadmap of KerasNLP)

@mattdangerw
Copy link
Member

By the way, I'm really interested in GSoC this year at KerasNLP

I'm pretty sure yes! But we are just figuring out all the details right now actually, will follow up when we have more.

Also, confirmed that this nightly test is unrelated to your change. Will pull this in tomorrow, and thank you!

@mattdangerw mattdangerw merged commit 85514c5 into keras-team:master Jan 31, 2023
@kanpuriyanawab kanpuriyanawab deleted the alberta_classifier branch February 13, 2023 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add AlbertClassifier

2 participants