[P1] TypeError: Object of type type is not JSON serializable

[srn-source]

I try to run train.py of alpaca examples with " TinyLlama/TinyLlama-1.1B-Chat-v1.0" but i got this error before training.

what should i do? please guide me.
I am searching for it, i think it is about model.config cannot convert to json? i am not sure

[PinetreePantry]

Could you paste here the command or the script that you are running? I think the issue is related to Pyvene unable to serialize the config of the model that you are using

[srn-source]

@PinetreePantry i use the same in readme demo.

python train.py --model_name_or_path TinyLlama/TinyLlama-1.1B-Chat-v1.0
--data_path ./alpaca_data.json
--output_dir ./test/
--layers "8;19"
--rank 4
--position "f1+l1"
--num_train_epochs 1
--per_device_train_batch_size 4
--per_device_eval_batch_size 4
--gradient_accumulation_steps 8
--evaluation_strategy "no"
--save_strategy "no"
--learning_rate 2e-5
--weight_decay 0.
--warmup_ratio 0.03
--lr_scheduler_type "cosine"
--logging_steps 1

[frankaging]

hey @srn-source, please try to add --report_to none and rerun. Thanks!

[srn-source]

@frankaging Its work, what happen? but thank you so much.

[PinetreePantry]

I think the reason might be because you accidentally enabled (probably by default) tensorboard logging, which triggered this line in the transformers library code. This would require a serialization of the Pyvene model's configs. Our model's configs contain types, not serializable by json.

[frankaging]

Marking this as close, and will track the progress in #70

[Shigerello]

please try to add --report_to none and rerun. Thanks!

Below is another workaround in a Python code snippet. This approach prevents TensorBoard logging from attempting to serialize the Pyvene model's configs, which cannot be serialized due to the use of type.

from transformers.integrations.integration_utils import TensorBoardCallback
trainer.remove_callback(TensorBoardCallback)