Support saving and loading AnyMoE gating layer

[EricLBuehler]

No description provided.

[github-actions]

Code Metrics Report

  ===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Dockerfile              1           34           25            0            9
 Happy                   1          442          369            0           73
 JSON                    9           21           21            0            0
 Python                 35         1363         1174           37          152
 TOML                   18          487          438            2           47
-------------------------------------------------------------------------------
 Jupyter Notebooks       1            0            0            0            0
 |- Markdown             1           60           30           22            8
 |- Python               1           96           87            1            8
 (Total)                            156          117           23           16
-------------------------------------------------------------------------------
 Markdown               21         1554            0         1174          380
 |- BASH                 5          101           98            0            3
 |- Python               5           98           88            0           10
 |- Rust                 4          221          198           10           13
 |- TOML                 2           75           63            0           12
 (Total)                           2049          447         1184          418
-------------------------------------------------------------------------------
 Rust                  129        41798        37892          741         3165
 |- Markdown            76          710           13          659           38
 (Total)                          42508        37905         1400         3203
===============================================================================
 Total                 216        45699        39919         1954         3826
===============================================================================