Multiple AdaptersFusion on same adapters

[luccitan]

Environment info

• adapter-transformers version: 2.2.0
• Platform: macOS-10.16-x86_64-i386-64bit
• Python version: 3.8.12
• PyTorch version (GPU?): 1.10.2 (False)

Details

Hello, I am ramping up on the usage of adapters and I would like to add Fusion Layers (with Fuse) for several adapters.
The issue is that I would like to have multiple Fuse for the same bag of adapters (trained on different datasets).

The Fuse layer is referenced through the name of the adapters, so to my knowledge having multiple fuse for same adapters would have the same reference, leading to a collision.
One workaround is to simply switch the order of the adapters name:

fuse1 = tf.adapters.composition.Fuse(['adapter1', 'adapter2'])
fuse2 = tf.adapters.composition.Fuse(['adapter2', 'adapter1'])

It is a bit a quick&dirty workaround, and one additional disadvantage is that it would not be possible to have more than 2 different fuses with the same 2 adapters.
Is there a more proper way to have this kind of feature?

Thanks!

[calpt]

Hey @luccitan, I believe we don't have any better way of doing what you describe at the moment. However, having a possibility to name AdapterFusion layers makes sense to me. Will keep this open as a feature request.