How to declare a network ?

In DLL, there are 3 ways of constructing a network type. Each have their advantages and disadvantages. This page shows the 3 ways of declaring the same network.

Method 1 - Static (Recommended)

using dbn_t = dll::dbn_desc<
    dll::dbn_layers<
        dll::rbm_desc<28 * 28, 100, dll::momentum, dll::batch_size<25>>::layer_t,
        dll::rbm_desc<100, 200, dll::momentum, dll::batch_size<25>>::layer_t,
        dll::rbm_desc<200, 10, dll::momentum, dll::batch_size<25>, dll::hidden<dll::unit_type::SOFTMAX>>::layer_t>,
    dll::batch_size<50>>::dbn_t;

 auto dbn = std::make_unique<dbn_t>();

 // Ready to use

As seen, all the information about the network is in the type. It is clear what the network is composed of and what are the parameters. This will be the fastest network at runtime, although the difference is not huge (maximum of 10% in release mode). Unfortunately, this means that every value is fixed at compile time. Moreover, this is the slowest to compile, especially if you have several networks in the same source file. Keep in mind that the data of the network will be directly inside the network meaning that the type can be pretty big.

Method 2 - Hybrid

using dbn_t = dll::dyn_dbn_desc<
    dll::dbn_layers<
        dll::rbm_desc<28 * 28, 100, dll::momentum, dll::batch_size<25>>::layer_t,
        dll::rbm_desc<100, 200, dll::momentum, dll::batch_size<25>>::layer_t,
        dll::rbm_desc<200, 10, dll::momentum, dll::batch_size<25>, dll::hidden<dll::unit_type::SOFTMAX>>::layer_t>,
    dll::batch_size<50>>::dbn_t;

 auto dbn = std::make_unique<dbn_t>();

 // Ready to use

(dbn_desc becomes dyn_dbn_desc)

This will be a bit slower to execute, but much faster to compile, especially if you have several networks in the same source files. It is also very expressive and easy to understand. Moreover, you can still change some values as runtime as you would with method 3.

Method 3 - Dynamic

using dbn_t =
    dll::dbn_desc<
        dll::dbn_layers<
            dll::dyn_rbm_desc<dll::momentum>::layer_t,
            dll::dyn_rbm_desc<dll::momentum>::layer_t,
            dll::dyn_rbm_desc<dll::momentum, dll::hidden<dll::unit_type::SOFTMAX>>::layer_t>>::dbn_t;

auto dbn = std::make_unique<dbn_t>();

dbn->template layer_get<0>().init_layer(28 * 28, 100);
dbn->template layer_get<1>().init_layer(100, 200);
dbn->template layer_get<2>().init_layer(200, 10);

dbn->template layer_get<0>().batch_size = 25;
dbn->template layer_get<1>().batch_size = 25;
dbn->template layer_get<2>().batch_size = 25;

In that case, the dimensions of the network are fully dynamic. You can change them at runtime (using a command line argument for instance). This is realy faster to compile than the method 1 and a bit faster to compile than the method 2. The runtime performance is in line with the method 2.

Assertions

When assertions are enabled (NDEBUG not set), the dynamic versions are really slower since the compiler cannot remove the assertions at compile time. There will be a ton of tests for dimensions access.