-
Notifications
You must be signed in to change notification settings - Fork 742
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous #3135
Comments
Hello, Im Also getting the same error, can someone please check **>
ValueError Traceback (most recent call last) File ~\anaconda3\envs\Thesis_2\lib\site-packages\gluonts\dataset\multivariate_grouper.py:87, in MultivariateGrouper.call(self, dataset) File ~\anaconda3\envs\Thesis_2\lib\site-packages\gluonts\dataset\multivariate_grouper.py:125, in MultivariateGrouper._group_all(self, dataset) File ~\anaconda3\envs\Thesis_2\lib\site-packages\gluonts\dataset\multivariate_grouper.py:152, in MultivariateGrouper._prepare_test_data(self, dataset) File ~\anaconda3\envs\Thesis_2\lib\site-packages\gluonts\dataset\multivariate_grouper.py:205, in MultivariateGrouper._transform_target(funcs, dataset) ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (959,) + inhomogeneous part. |
Hello, Load CLang-8 datasetdf = pd.read_csv('Data/clang8.csv') Create Hugging Face Datasetdataset = Dataset.from_pandas(df) Split datasettrain_test_split = dataset.train_test_split(test_size=0.1) Load pretrained model from the specific checkpointmodel = T5ForConditionalGeneration.from_pretrained('./results/checkpoint-3000') Load tokenizertokenizer = AutoTokenizer.from_pretrained('t5-small') Define training arguments (no need to set training-specific args for evaluation)training_args = TrainingArguments( Initialize Trainertrainer = Trainer( Function to evaluate in chunksdef evaluate_in_chunks(trainer, dataset, chunk_size=100):total_size = len(dataset)all_predictions = []for i in range(0, total_size, chunk_size):chunk = dataset.select(range(i, min(i + chunk_size, total_size)))predictions = trainer.predict(chunk)all_predictions.append(predictions.predictions)return all_predictionsEvaluate the model in chunks#predictions = evaluate_in_chunks(trainer, eval_dataset, chunk_size=100) Generate predictionspredictions = trainer.predict(eval_dataset) -- Check if predictions are real or notprint("Checking predictions: "+str(len(predictions))) predictions = np.asarray(predictions, dtype='object') --pred_texts = tokenizer.batch_decode(np.squeeze(predictions), skip_special_tokens=True) pred_texts = []for prediction in predictions:pred_texts.append(tokenizer.decode(prediction[prediction], skip_special_token=True))--#print(predictions.shape) --Save predictions and referenceswith open('predictions.txt', 'w') as pred_file, open('references.txt', 'w') as ref_file: Initialize ERRANTannotator = errant.load('en') Align predictions and referenceswith open('predictions.txt', 'r') as pred_file, open('references.txt', 'r') as ref_file: aligned = [] Evaluate using ERRANTP, R, F = errant.scorer(aligned) |
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (32,) + inhomogeneous part.
`from typing import Any, Dict, Iterable, Optional
from gluonts.dataset.loader import TrainDataLoader
from gluonts.itertools import Cached
from gluonts.torch.batchify import batchify
import pytorch_lightning as pl
import torch
from gluonts.core.component import validated
from gluonts.dataset.common import Dataset
from gluonts.dataset.field_names import FieldName
from gluonts.dataset.loader import as_stacked_batches
from gluonts.dataset.stat import calculate_dataset_statistics
from gluonts.itertools import Cyclic
from gluonts.time_feature import (
get_lags_for_frequency,
time_features_from_frequency_str,
)
from gluonts.torch.model.estimator import PyTorchLightningEstimator
from gluonts.torch.model.predictor import PyTorchPredictor
from gluonts.torch.modules.loss import DistributionLoss, NegativeLogLikelihood
from gluonts.transform import (
AddObservedValuesIndicator,
AddTimeFeatures,
Chain,
DummyValueImputation,
ExpectedNumInstanceSampler,
InstanceSampler,
InstanceSplitter,
TestSplitSampler,
Transformation,
ValidationSplitSampler,
VstackFeatures
)
from peft import LoraConfig, get_peft_model
from gluonts.torch.model.deepar import DeepAREstimator
from gluonts.torch.distributions import StudentTOutput, NormalOutput
from gluon_utils.gluon_ts_distributions.implicit_quantile_network import (
ImplicitQuantileNetworkOutput,
)
from lag_llama.gluon.lightning_module import LagLlamaLightningModule
PREDICTION_INPUT_NAMES = [
"past_target",
"past_observed_values",
]
TRAINING_INPUT_NAMES = PREDICTION_INPUT_NAMES + [
"future_target",
"future_observed_values",
]
class LagLlamaEstimator(PyTorchLightningEstimator):
"""
An estimator training a ConvTSMixer model for forecasting.
`
The text was updated successfully, but these errors were encountered: