[Numpy] Dense does not support float32 + float64.

[sxjscience]

The following code snippet will raise an error with the current MXNet. The reason is that the dtype of a will be float64 while the dense layer only supports float32.

import mxnet as mx
from mxnet.gluon import nn
import numpy as np
mx.npx.set_np()

net = nn.Dense(16)
net.initialize()

a = mx.np.array(np.ones((10, 4)))
out = net(a)
print(out)

MXNetError: Traceback (most recent call last):
  File "../src/io/../operator/elemwise_op_common.h", line 135
MXNetError: Check failed: assign(&dattr, vec.at(i)): Incompatible attr in node FullyConnected at 1-th input: expected float64, got float32

[yzhliu]

As discussed with @sxjscience before, one solution is to always convert to fp32 when reading from numpy array. what do you think? @reminisce @haojin2 @szha

[yzhliu]

We'll have an global default dtype switch, and it will be set to float32 if not changed by user.