[WIP] Add Moonshine

[eustlb]

No description provided.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[xenova]

(just some notes in the meantime).

[xenova]

Suggested change

	from transformers.models.moonshine.modeling_moonshine import MoonshineConfig
	from transformers.models.moonshine.modeling_moonshine import MoonshineDecoder
	from transformers.models.moonshine.modeling_moonshine import (
	MoonshineConfig,
	MoonshineForConditionalGeneration,
	)

[xenova]

Suggested change

	final_weights = {}
	for k, v in encoder_state_dict.items():
	final_weights[f"model.encoder.{k}"] = v
	for k, v in converted_decoder_weights.items():
	final_weights[f"model.decoder.{k}"] = v
	if model_name == 'tiny':
	config = MoonshineConfig()
	elif model_name == 'base':
	config = MoonshineConfig(
	hidden_size=416,
	num_hidden_layers=8,
	num_attention_heads=8,
	)
	else:
	raise ValueError(f"Unknown model name {model_name}")
	final_weights['proj_out.weight'] = converted_decoder_weights['embed_tokens.weight']
	model = MoonshineForConditionalGeneration(config)
	model.load_state_dict(final_weights)
	model.save_pretrained(pytorch_dump_folder_path)

[xenova]

To align with other parts of the library, it might be better to use "input_values" instead of "input _features", since "input_features" are typically for models that perform feature extraction, whereas for moonshine, we use the raw audio values instead.