Skip to content

Add zstandard compression#706

Merged
jkbk2004 merged 8 commits into
NOAA-EMC:developfrom
DusanJovic-NOAA:add_zstandard
Oct 31, 2023
Merged

Add zstandard compression#706
jkbk2004 merged 8 commits into
NOAA-EMC:developfrom
DusanJovic-NOAA:add_zstandard

Conversation

@DusanJovic-NOAA
Copy link
Copy Markdown
Collaborator

Description

(Instructions: this, and all subsequent sections of text should be removed and filled in as appropriate.)
Provide a detailed description of what this PR does.

New model_configure configuration parameter has been added (zstandard_level). Valid values are 0-19 (default is 0). Value 0 turns off compression, values 1-19 specify the level of compression (1=fastest compression, 19=best compression ratio),

What bug does it fix, or what feature does it add? Add option to enable zstandard compression.

Is a change of answers expected from this PR? No.

Issue(s) addressed

Link the issues to be closed with this PR, whether in this repository, or in another repository.
(Remember, issues should always be created before starting work on a PR branch!)

Testing

How were these changes tested? Regression test on Hera and Acorn
What compilers / HPCs was it tested with? Inetl, GNU
Are the changes covered by regression tests? No. Currently nccmpcan not compare netcdf files with zstandard compression enabled. The nccmp program needs to be updated.

Have the ufs-weather-model regression test been run? Yes. On what platform? Hera

  • Will the code updates change regression test baseline? No.
  • Please show the baseline directory below.
  • Please commit the regression test log files in your ufs-weather-model branch

Copy link
Copy Markdown
Collaborator

@junwang-noaa junwang-noaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes are good, the code looks cleaner, thanks.

@zach1221 zach1221 requested a review from jkbk2004 October 31, 2023 20:08
@jkbk2004 jkbk2004 merged commit 1250b41 into NOAA-EMC:develop Oct 31, 2023
@DusanJovic-NOAA DusanJovic-NOAA deleted the add_zstandard branch November 1, 2023 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

add option to use zstd compression

4 participants