ERROR:model.generate(input_tokens.ids, max_length=200)-------AttributeError: 'list' object has no attribute 'ids'

[geng-lee]

How to fix it?

generated_token_ids = model.generate(input_tokens.ids, max_length=200)

AttributeError Traceback (most recent call last)
in <cell line: 12>()
10 input_tokens = tokenizer(input_midi)
11
---> 12 generated_token_ids = model.generate(input_tokens.ids, max_length=200)
13 # generated_midi = tokenizer(generated_token_ids)
14 # generated_midi.dump("/content/out/continued.mid")

AttributeError: 'list' object has no attribute 'ids'

[elloza]

First of all, congratulations on the work! It is all very well structured and all resources are available.

I was trying to replicate the results and also do some small test like this and I have some doubts that maybe @Natooz can help me here.

Using the model card code available here:
https://huggingface.co/Natooz/Maestro-REMI-bpe20k

And trying with this example midi:

!wget -O song.mid https://thesession.org/tunes/750/midi/1

import torch
from transformers import AutoModelForCausalLM
from miditok import REMI
from miditoolkit import MidiFile

torch.set_default_device("cuda")
model = AutoModelForCausalLM.from_pretrained("Natooz/Maestro-REMI-bpe20k", trust_remote_code=True, torch_dtype="auto")
tokenizer = REMI.from_pretrained("Natooz/Maestro-REMI-bpe20k")
input_midi = MidiFile("song.mid")
input_tokens = tokenizer(input_midi)

"""
Here we obtain a list of <class 'miditok.classes.TokSequence'>

TokSequence(tokens=['Bar_None', 'Position_0', 'Program_0', 'Pitch_67', 'Velocity_111', 'Duration_0.5.8', 'Position_8', 'Program_0', 'Pitch_71', 'Velocity_79', 'Duration_0.3.8', 'Position_12', ....., ], 
ids=[4, 173, 220, 51, ..... 111],
bytes=None, events=[Event(type=Bar, value=None, time=0, desc=0), Event(type=Position, value=0, time=0, desc=0), ....], ids_bpe_encoded=False, 
_ids_no_bpe=None)

I guess these ids are the ones we need to pass to the model.

"""

# id list to pytorch tensor and add batch dimension (I dont know if this is rigth)
input_tensor = torch.tensor(input_tokens[0].ids, dtype=torch.long).unsqueeze(0)

# device
device = "cuda" if torch.cuda.is_available() else "cpu"
input_tensor = input_tensor.to(device)

# generate tokens
generated_token_ids = model.generate(input_tensor, max_length=200)
print(generated_token_ids)

# This step fails
generated_midi = tokenizer(generated_token_ids)

# back to midi
generated_midi.dump("continued.mid")

When I try to get the midi back from the tokenizer I get an error like this:

The input sequence has one dimension less than expected (1 instead of 2).It is being unsqueezed to conform with the tokenizer's i/o format (('I', 'T'))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[<ipython-input-9-c1a9266f5d9b>](https://localhost:8080/#) in <cell line: 11>()
      9 generated_token_ids = model.generate(input_tensor, max_length=200)
     10 print(generated_token_ids)
---> 11 generated_midi = tokenizer(generated_token_ids.unsqueeze(0))
     12 # back to midi
     13 generated_midi.dump("continued.mid")

3 frames
[/usr/local/lib/python3.10/dist-packages/miditok/midi_tokenizer.py](https://localhost:8080/#) in _are_ids_bpe_encoded(self, ids)
   1803         :return: boolean, True if ids are encoded with BPE, False otherwise.
   1804         """
-> 1805         return np.any(np.array(ids) >= len(self._vocab_base))
   1806 
   1807     def decode_bpe(self, seq: Union[TokSequence, List[TokSequence]]):

TypeError: '>=' not supported between instances of 'NoneType' and 'int'

• Which input exactly should I give to the model?
• What does the tokenizer expect as input to get the midi from the generated ids?

I hope you could give me a hand with this.

Thanks a lot!

[Natooz]

Hi, thank you for your kind comment! :)

In most cases, models take input ids (i.e. integers which can be seen as the idx of the tokens in the vocabulary), which allows it to use embeddings (I won't go into that here).
I didn't look closely at the code as I would likely require to run things to fix what's wrong, but I'll give you my advice on how to debug in this situation. :)
Generally, a model takes an input tensor with two dimensions (N,T). The first one is for the N input sequences as we usually generate multiple ones in parallel for speed purposes (but if you only have one sequence then N=1) and the second dimension for the sequence length in numbers of token ids (T). I believe you did it right here.

For the output, the tokenizer can expect different input format depending on its configuration. Here if I remember well the tokenizer do not support Program tokens, so the input/output should have the format (I,T) for I tracks and T tokens. You can find more details in the docs (this docs is a bit too "up-to-date" as it concerns the incoming MidiTok V3 but the concerned paragraph is still 100% relevant).
Hope it helped!

[elloza]

Thank you so much @Natooz. I think that I understand better the code now.
I think I got at least the "hello world" of the model you posted with the following code:

import torch
from transformers import AutoModelForCausalLM
from miditok import REMI
from miditoolkit import MidiFile

torch.set_default_device("cuda")
model = AutoModelForCausalLM.from_pretrained("Natooz/Maestro-REMI-bpe20k", trust_remote_code=True, torch_dtype="auto")
tokenizer = REMI.from_pretrained("Natooz/Maestro-REMI-bpe20k")
input_midi = MidiFile("song.mid")
input_tokens = tokenizer(input_midi)

print("ORIGINAL TOKENS:")
print(input_tokens[0].ids)
"""
[175, 52, 1336, 138, 1130, 142, 49, 1336, 150, 1130, 154, 56, 995, 158, 4681, 1083, 134, 51, 1336, 142, 1077, 146, 47, 1336, 154, 1077, 158, 54, 995, 175, 4856, 1281, 138, 52, 1336, 146, 1130, 150, 49, 1336, 158, 1130, 175, 56, 995, 134, 4507, 1083, 142, 15579, 144, 9000, 146, 3659, 1130, 154, 54, 995, 158, 4525, 1281, 134, 47, 995, 138, 7024, 1281, 146, 52, 1336, 154, 1130, 158, 49, 1336, 246, 1130, 138, 56, 995, 142, 4279, 1083, 150, 51, 1336, 158, 1077, 175, 47, 1336, 138, 1077, 142, 54, 995, 146, 4497, 1281, 154, 52, 1336, 175, 1130, 134, 49, 1336, 142, 1130, 146, 56, 995, 150, 4475, 1083, 158, 15579, 160, 9000, 175, 3690, 1130, 138, 54, 995, 142, 4809, 1281, 150, 14462, 175, 64, 1336, 138, 1058, 142, 64, 1336, 150, 1058, 154, 61, 995, 158, 3514, 1130, 134, 63, 1336, 142, 926, 146, 59, 995, 150, 5339, 926, 158, 63, 995, 175, 4161, 926, 138, 64, 1336, 146, 1058, 150, 64, 1336, 158, 1058, 175, 61, 995, 134, 3597, 1130, 142, 59, 995, 146, 5113, 1083, 154, 52, 995, 158, 5134, 926, 134, 61, 1336, 142, 1167, 146, 64, 1336, 154, 1058, 158, 64, 1336, 246, 1058, 138, 61, 995, 142, 3731, 1130, 150, 63, 1336, 158, 926, 175, 59, 995, 134, 5597, 926, 142, 63, 995, 146, 4076, 926, 154, 19260, 158, 6499, 1058, 134, 63, 995, 138, 4385, 926, 146, 61, 995, 150, 5339, 1077, 158, 15579, 160, 9000, 175, 3690, 1130, 138, 54, 995, 142, 4809, 1281, 150, 49, 1336, 158, 1281]
"""

# id list to pytorch tensor and add batch dimension 
input_tensor = torch.tensor(input_tokens[0].ids, dtype=torch.long).unsqueeze(0)

# device
device = "cuda" if torch.cuda.is_available() else "cpu"
input_tensor = input_tensor.to(device)

# generate tokens
generated_token_ids = model.generate(input_tensor, max_length=200, max_new_tokens=200)

from miditok.utils import convert_ids_tensors_to_list
from miditok import TokSequence

token_ids_list = convert_ids_tensors_to_list(generated_token_ids[0].cpu())
print(token_ids_list)

"""
[175, 52, 1336, 138, 1130, 142, 49, 1336, 150, 1130, 154, 56, 995, 158, 4681, 1083, 134, 51, 1336, 142, 1077, 146, 47, 1336, 154, 1077, 158, 54, 995, 175, 4856, 1281, 138, 52, 1336, 146, 1130, 150, 49, 1336, 158, 1130, 175, 56, 995, 134, 4507, 1083, 142, 15579, 144, 9000, 146, 3659, 1130, 154, 54, 995, 158, 4525, 1281, 134, 47, 995, 138, 7024, 1281, 146, 52, 1336, 154, 1130, 158, 49, 1336, 246, 1130, 138, 56, 995, 142, 4279, 1083, 150, 51, 1336, 158, 1077, 175, 47, 1336, 138, 1077, 142, 54, 995, 146, 4497, 1281, 154, 52, 1336, 175, 1130, 134, 49, 1336, 142, 1130, 146, 56, 995, 150, 4475, 1083, 158, 15579, 160, 9000, 175, 3690, 1130, 138, 54, 995, 142, 4809, 1281, 150, 14462, 175, 64, 1336, 138, 1058, 142, 64, 1336, 150, 1058, 154, 61, 995, 158, 3514, 1130, 134, 63, 1336, 142, 926, 146, 59, 995, 150, 5339, 926, 158, 63, 995, 175, 4161, 926, 138, 64, 1336, 146, 1058, 150, 64, 1336, 158, 1058, 175, 61, 995, 134, 3597, 1130, 142, 59, 995, 146, 5113, 1083, 154, 52, 995, 158, 5134, 926, 134, 61, 1336, 142, 1167, 146, 64, 1336, 154, 1058, 158, 64, 1336, 246, 1058, 138, 61, 995, 142, 3731, 1130, 150, 63, 1336, 158, 926, 175, 59, 995, 134, 5597, 926, 142, 63, 995, 146, 4076, 926, 154, 19260, 158, 6499, 1058, 134, 63, 995, 138, 4385, 926, 146, 61, 995, 150, 5339, 1077, 158, 15579, 160, 9000, 175, 3690, 1130, 138, 54, 995, 142, 4809, 1281, 150, 49, 1336, 158, 1281, 194, 1083, 134, 897, 136, 8854, 678, 139, 10040, 630, 7730, 5379, 146, 3960, 4334, 152, 308, 10017, 4544, 7206, 4479, 14899, 7487, 15121, 501, 16610, 618, 132, 16528, 574, 135, 430, 4992, 698, 137, 822, 140, 2316, 3713, 1189, 141, 11512, 19849, 713, 145, 574, 147, 713, 11286, 281, 14706, 667, 151, 584, 7109, 472, 2443, 154, 1548, 156, 1323, 11892, 285, 19631, 1323, 160, 337, 858, 18350, 430, 822, 15010, 132, 822, 133, 11532, 1189, 7612, 240, 8748, 1118, 139, 1080, 5249, 629, 1122, 10015, 3162, 713, 1323, 143, 348, 667, 9856, 472, 147, 3162, 4603, 1402, 150, 13304, 624, 4548, 7165, 1402, 155, 399, 5651, 472, 13976, 16326, 429, 160, 453, 175, 3076, 3081, 4619, 132, 399, 6123, 15841, 136, 414, 9258, 139, 14216, 6285, 263, 429, 142, 425, 355, 6310, 5772, 374, 7681, 627, 145, 1224, 146, 1227, 9934, 263, 12840, 3509, 149, 727, 8210, 298, 5772, 1243, 151, 657, 5250, 627, 9184, 242, 472, 5446, 472, 4561, 242, 429, 10095, 7866, 242, 241, 161, 713, 154, 727, 159, 2849, 1731, 12503, 288, 310, 516, 1122, 156, 3162, 8407, 290, 140, 368, 8555, 264, 309, 63, 1724, 7684, 374, 66, 109, 111, 132, 4519, 4611, 51]

"""
converted_back_midi = tokenizer.tokens_to_midi([token_ids_list])
converted_back_midi.dump('continued.mid')

I'm not entirely sure it worked well, but at least it does follow the sequence provided.

I am still studying the documentation and code a bit as this domain is a bit new to me.

Thank you very much for the reply and help.

Regards,