Lightening like engineering

[duvi86]

Hello,
I would like to know if there is a way to avoid the loop over episode and store info in tensorboard like in PyTorch lightning ?

Thanks

[iffiX]

Hi,
Currently machin has no such feature, since it was designed for readability and customizability, but we can provide it in the following ways:

1. Add an auto launcher, you only need to provide launching params, however, your environment needs to be OpenAI gym compatible.

   m.train(agent="DQN", env="LunarLander-v2", model_creator=lambda : YourModel(hidden_dim=256), learning_rate=1e-3....)
   

2. Add an auto launcher, you need to provide launching params, then you will define your environment input and output types, such as "image" for image and videos, "sequence" for sound and text input, "feature vector" for vector, or custom types. For custom types you will need to specify how to save/pickle it, and how to log it to tensorboard, these will be callback functions. For each input/output you can also specify whether to save them, etc., using a bool indicator.

   import machin as m
   class SomeEnvironment(m.Environment):
       def __init__():
           self.env = your_environment_class
           self.env_def = m.EnvironmentDef(
                m.InArg(0), m.VectorType(1, t.int32),                         # size 1 action
                m.InKwArg("some_arg"), m.VectorType(1, t.int32),              # size 1 action
                m.Out(0), m.ImageType(240, 320, 3)                            # 240x320 RGB
                m.Out(0), m.VectorType(20, t.float32),                        # size 20 feature vector
                m.DataCallBack, m.ImageType(240, 320, 3), self.env.render,    # For automatic rendered result collection
           )
   
   # then
   m.train(agent="DQN", env=SomeEnvironment(), model_creator=lambda : YourModel(hidden_dim=256), learning_rate=1e-3....)
   

Which solution would you prefer? Or both? For both solutions, we can let the framework automatically handle the loop, logging data, auto-save/load your model, etc. we can also support adding transformation steps like in pytorch lightning, if your model cannot directly handle your environment output/input.

If you just want a way to support pytorch lightning, then we are afraid that it is not feasible, since pytorch lightning is trying to be device agnostic, this is not compatible with some of our distributed algorithms, and would require complete rewriting of the whole framework.

[iffiX]

Recently I have reconsidered about this and find that our framework is applicable to pytorch lightning, we will also introduce nni hyper parameter tuning in the next update.

[duvi86]

Great Thanks!

…

On Sat, 13 Mar 2021, 22:46 Iffi, ***@***.***> wrote: Recently I have reconsidered about this and find that our framework is applicable to pytorch lightning, we will also introduce nni hyper parameter tuning in the next update. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#7 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACYQPKR3I24RGGRF4THUEJLTDPMLRANCNFSM4X7ARKJA> .