synaptic bit-slicing

[saba-er]

Hi,
I was wondering if and how synaptic bit-slicing is supported by the simulator? For example, the needed weight precision is 4 bits but the device only supports 2 bits and hence you need to use 2 devices per synaptic weight. Then, the analog computation (e.g. multiplication) will take into account the fact that the two devices combined represent a single weight (might be partially handled in the digital periphery?).

[maljoras]

Hi @SaBaKa2020 ,
thanks again! Currently, the simulator does not do bit-slicing in the way you are describing it by default, however, this is definitely on our roadmap to add tools to support that in a convenient way. With the VectorUnitCell multiple devices per crosspoint can be used, as well as a significance factor, however, the currents would always be added up "in analog" i.e. before the ADC.

Having said this, it is actually very easy to build your own bit slicing layer that sums the outputs of the weight slices in digital: You could just define an new module (e.g. AnalogLinearBitSlicing) that internally has two analog tiles, e.g. linear_1 = AnalogLinear() and linear_2 = AnalogLinear() and combine the output in the forward method, e.g. y = linear_1(x) + factor * linear_2(x). In case you would implement it in this way, we would highly appreciate if you would contribute it to the toolkit by making pull request!

[HeatPhoenix]

Hi @SaBaKa2020 , thanks again! Currently, the simulator does not do bit-slicing in the way you are describing it by default, however, this is definitely on our roadmap to add tools to support that in a convenient way. With the VectorUnitCell multiple devices per crosspoint can be used, as well as a significance factor, however, the currents would always be added up "in analog" i.e. before the ADC.

@maljoras, can a VectorUnitCell setup be combined with, say, InferenceRPUConfig in the sense of rpu_config contains a vector of devices -> with each device is (inference only w/ a given noise model) contributing a part of the full weight. In other words, can we achieve this bit-slicing within an inference only setup?

I could not find an example describing such a usage, only this example.

[HeatPhoenix]

Opened a pull request in #631 to hopefully help implement this and solve this issue.

[maljoras-sony]

VectorUnitCell is more meant to explore multiple devices during in-situ training, that are also combine currents before the ADC conversion. The example I gave above using AnalogLinaer would do the gradient through the bit-sliced layers. On the other hand, if the ConductanceConverter of InferenceRPUConfig is used (where one could also use multiple weight slices), then the slicing is done after convergence (that is training the DNN without the bit-sliced weights). So these are all valid approaches, but slightly different concepts.