[code_search] HP Tuning with Katib

[jlewi]

It would be great to get HP tuning integrated into the code search example.

A simple grid search for some parameter would be a great place to start.

I think there are a couple blocking issues

• I think we might need to make some code changes to easily expose certain parameters like learning rate for HP tuning

  • Its not clear to me how T2T is meant to interact with HP tuning; I think T2T is intended to make everything repeatable and so defines sets of hyperparameters in code and then you just select the set you want.
  • I think we override that but we will need to modify the code.

• We are still waiting on Katib to support TFJob

  • Tf event mc katib#235 support for event files.

[jlewi]

HP parameters are listed in the code
https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/models/transformer.py#L1307

Looks like these should be settable via command line argument just by prefixing them with "--hp"
https://github.com/tensorflow/tensor2tensor/blob/8bcbdccf85c0fc60f07945c469ff3213d2e0810d/tensor2tensor/bin/t2t_trainer.py#L139

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.