Add Lightning-like Profiler to Keras

[innat]

Reopening from here.

Describe the feature and the current behavior/state

In PyTorch-Lightning, you can identify bottlenecks in your code using Profiler API. And I've found it super useful and easy to inspect the pros and cons of the code. This API offers 3 type of inferface, namely simple, advance and pytorch. I mostly use simple that summarize the execution time of medium to high level training + validation + inference API. The saved log files look something as follows.

# by passing a string
trainer = Trainer(..., profiler="simple")

# or by passing an instance
from pytorch_lightning.profiler import SimpleProfiler

profiler = SimpleProfiler()
trainer = Trainer(..., profiler=profiler)

I don't know, in keras, if we have anything like this already. By having such high-level functionality in keras would really great. It will help to inspect the bottleneck of dataloader, model, callbacks etc, by reporting the time and memory consumption in the system.

I looked for it before, discussion.

Will this change the current api? How?

This can be considered as a Callback, I think.

keras.callbacks.Profiler
or
keras.utils.Profiler

Who will benefit from this feature?

The keras community. Engineers and researchers, both. Mostly engineers in production phase for code optimization.

Contributing

• Do you want to contribute a PR? (yes/no):
• If yes, please read this page for instructions
• Briefly describe your candidate solution(if contributing):

[nkovela1]

Hi @innat, this would be a good addition as a Callback! Would you be willing to contribute a PR to add this Profiler Callback?

[innat]

Yes, I'm willing to contribute. Before that, could you please give some thought on this. Thanks.

[fchollet]

Before that, could you please give some thought on keras-team/tf-keras#350 (comment). Thanks.

There are no profiling plans from our side at this time -- if you want to create this feature, you're welcome to do it, it would be a very valuable addition.

[ghsanti]

I've tried something hereI don't know if it's a meaningful plot yet, using Trace. Seems to be ok for CPU only though.

GIST