How to read a nc file ?

[szukiyu]

I tried executing the program.
But I couldn’t read nc file of CMEMS Data Access Portal.

In SSTseq class, data is read as a .pkl file.
https://github.com/emited/flow/blob/master/flow/datasets/nc.py#L31

Could you tell me how to convert the .nc file to .pkl file ?

[andrewzm]

Agree -- it would be good to have the pkl files for reproducibility, or else a set of instructions/source files to obtain the pkl files from the original source.

[chenxiaodanhit]

Hello, may i ask you if you successfully transform the nc file into the pkl file, I also have some troubles and hope to get your help. Many thanks!

[andrewzm]

No, sorry, I ended up doing everything from scratch in TensorFlow, which worked great.

…

On Mon, Apr 26, 2021 at 1:18 PM chenxiaodanhit ***@***.***> wrote: Hello, may i ask you if you successfully transform the nc file into the pkl file, I also have some troubles and hope to get your help. Many thanks! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEU3KLC5EJ6MXAJTUAGNTTKTLQ7ANCNFSM4FEGYPAQ> .

[toddwyl]

Is there any tips for preprocessing the nc file? @andrewzm @szukiyu @chenxiaodanhit

[andrewzm]

I ended up extending this work to the more realistic case of when data are gappy and noisy, using a statistical modelling approach. I then illustrated our method on the same data set, so I preprocessed it. You can see all my code here: https://github.com/andrewzm/deepIDE The pre-processing is done in the folder 1_Preproc_data <https://github.com/andrewzm/deepIDE/tree/master/1_Preproc_data> ... in R I'm afraid! But the resulting data is in the arrays required for training your network, which can be easily exported and converted to some Python format if need be. If you are interested in the actual paper, see here: https://www.sciencedirect.com/science/article/pii/S2211675320300026

…

On Sun, 2 May 2021, 16:18 Wen YongLiang, ***@***.***> wrote: Is there any tips for preprocessing the nc file? @andrewzm <https://github.com/andrewzm> @szukiyu <https://github.com/szukiyu> @chenxiaodanhit <https://github.com/chenxiaodanhit> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEU3LYY52B2ZGEUMSA6E3TLTVBPANCNFSM4FEGYPAQ> .

[toddwyl]

I ended up extending this work to the more realistic case of when data are gappy and noisy, using a statistical modelling approach. I then illustrated our method on the same data set, so I preprocessed it. You can see all my code here: https://github.com/andrewzm/deepIDE The pre-processing is done in the folder 1_Preproc_data https://github.com/andrewzm/deepIDE/tree/master/1_Preproc_data ... in R I'm afraid! But the resulting data is in the arrays required for training your network, which can be easily exported and converted to some Python format if need be. If you are interested in the actual paper, see here: https://www.sciencedirect.com/science/article/pii/S2211675320300026
…
On Sun, 2 May 2021, 16:18 Wen YongLiang, @.***> wrote: Is there any tips for preprocessing the nc file? @andrewzm https://github.com/andrewzm @szukiyu https://github.com/szukiyu @chenxiaodanhit https://github.com/chenxiaodanhit — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABEU3LYY52B2ZGEUMSA6E3TLTVBPANCNFSM4FEGYPAQ .

Thanks a lot!