Killed when using long list of training data, How to solve it ?

[housebaby]

System Info

what does split mean hear,as no difference between train or others?

    self.data_list = []
    if split == "train":
        with open(dataset_config.train_data_path, encoding='utf-8') as fin:
            for line in fin:
                data_dict = json.loads(line.strip())
                self.data_list.append(data_dict)
    else:
        with open(dataset_config.val_data_path, encoding='utf-8') as fin:
            for line in fin:
                data_dict = json.loads(line.strip())
                self.data_list.append(data_dict)

Information

• The official example scripts
• My own modified scripts

🐛 Describe the bug

Error logs

Expected behavior

If I change the long training list to small list, it works
How to adapt to large training dataset?

[ddlBoJack]

split means your train, val, and test sets here