Skip to content

Conversation

@Cyber-Machine
Copy link
Contributor

@Cyber-Machine Cyber-Machine commented Jan 24, 2023

PR for #657.

Added metadata for generating preset table on keras-io.

Copy link
Member

@mattdangerw mattdangerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Left some initial comments.

@mattdangerw
Copy link
Member

Thanks @Cyber-Machine ! I timed out this week, but will look at this next Monday!

One thing you could do if you want to keep moving on this is start looking at the keras.io side. We probably want to ready the PRs in tandem, as we might discover changes we need here when rendering the table for our documentation.

There's a few things you'll need to do there:

  • Before making any changes to keras.io, you will want to be able to render the docs site locally. There are instructions on the README.
  • When you are ready to make changes, you will want to have keras.io point to this branch of keras-nlp. This is a bit tricky! Once you have a working setup for keras.io, you can try running pip install git+https://github.com/Cyber-Machine/keras-nlp.git@preset_table to install this branch (or you could make sure to locally install keras-nlp from your branch in the same python env as the keras.io setup).
  • Lastly, you will need to write the actual table rendering code, and make sure the table renders properly. See this commit as a starting point for code changes to keras.io.

This will all be somewhat tricky as we are coordinating across repos, so if you have any question I can try to help out next Monday!

@Cyber-Machine
Copy link
Contributor Author

Sure @mattdangerw, over this weekend I will look into ways how we can generate table on keras-io. Will keep you updated!

Copy link
Member

@mattdangerw mattdangerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Just a few small things.

"Extra Large size of ALBERT where all input is lowercased. "
"Trained on English Wikipedia + BooksCorpus."
),
"params": 222595584,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can actually add "official_name" and "path" for all models, even ones that are not yet documented on keras.io.

The path should always match the directory the model is in in KerasNLP. The official_name should match the original paper.

mask_positions,
mask_ids,
) = tf_text.mask_language_model(
(token_ids, mask_positions, mask_ids,) = tf_text.mask_language_model(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you update your install of black (pip install -U black), you should get the latest formatting here.

Copy link
Member

@mattdangerw mattdangerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! This looks good.

I think we should move this dict to the top of each preset, and I see some spots where our descriptions could still use some updating, but I can fix those up as I merge.

@mattdangerw mattdangerw merged commit d658112 into keras-team:master Feb 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants