Feature extraction from multimodal data

[soujanyaporia]

For those enquiring about how to extract visual and audio features, please check this out: https://github.com/soujanyaporia/MUStARD.

[v-anh]

Hi @soujanyaporia, thank you for your hard work on it.
I have a question regarding the Audio Feature extraction.
I've read the IEMOCAP audio data from your pickle file: https://github.com/declare-lab/conv-emotion/blob/master/DialogueRNN/DialogueRNN_features.zip
And I saw that the audio data for each wav file is extracted into an array of 100 values. But checked the MUStARD and they setting it as an array of 283 values. Could you help to share some sort of config or the way you set it as 100 values for each wav file?

Thank you

[siddhantpathakk]

Same question as @v-anh

I am using the MELD dataset and the MELD features from the pickle file under https://github.com/declare-lab/conv-emotion/blob/master/DialogueRNN/DialogueRNN_features.zip

In the original paper for DialogueRNN, it mentions about using openSMILE toolkit to retrieve the audio features, as (300,0) vectors for each utterance.
Could you explain the pre-processing done on the raw mp4 files to get these vectors using the toolkit? I wish to utilize and make an inference system, I have the mp4 files and I wish to use the openSMILE toolkit to get the [300,] vectors.