Skip to content

add log_diag_field_info to dmUpdate #1090

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
thomas-robinson merged 15 commits into from
Apr 21, 2023
Merged

add log_diag_field_info to dmUpdate #1090

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
845b714
add routine to modern diag and move first write to routine
Dec 12, 2022
0e87d64
add routine for setting separator and correct logic for nml flag
Dec 12, 2022
4fb2e91
add nml flag to run log routine for modern diag test
Dec 13, 2022
30023eb
Merge remote-tracking branch 'upstream/dmUpdate' into log_field
Dec 21, 2022
b53c1a9
remove added routine and rework in order to call from main module
Dec 21, 2022
1d04b51
whitespace
Dec 28, 2022
83b8706
check read freq amd value set in obj
Jan 25, 2023
01b2bc9
Revert "whitespace"
Jan 25, 2023
d27674e
revert unneeded code changes with current dmUpdate
Apr 7, 2023
f4271c2
Merge remote-tracking branch 'upstream/dmUpdate' into log_field
Apr 7, 2023
2382a5f
fix issues with merge; add to static register routine
Apr 7, 2023
79d548f
missed some prints
Apr 7, 2023
6f0b5ad
fix errors with scalars
Apr 18, 2023
8a55506
change scalar axis_name to better reflect code changes
Apr 21, 2023
6f5f556
whitespace
Apr 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 30 additions & 8 deletions diag_manager/diag_manager.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -436,13 +436,20 @@ INTEGER FUNCTION register_diag_field_array(module_name, field_name, axes, init_t
INTEGER, OPTIONAL, INTENT(in) :: area !< Id of the area field
INTEGER, OPTIONAL, INTENT(in) :: volume !< Id of the volume field
CHARACTER(len=*), OPTIONAL, INTENT(in) :: realm !< String to set as the modeling_realm attribute
logical :: allow_log

if (use_modern_diag) then
! check if logging registered fields, uses arg and nml flag
IF ( PRESENT(do_not_log) ) THEN
allow_log = .NOT.do_not_log
ELSE
allow_log = .TRUE.
END IF
register_diag_field_array = fms_diag_object%fms_register_diag_field_array( &
& module_name, field_name, axes, init_time, long_name=long_name, &
& units=units, missing_value=missing_value, var_range=range, mask_variant=mask_variant, &
& standard_name=standard_name, verbose=verbose, do_not_log=do_not_log, err_msg=err_msg, &
& interp_method=interp_method, tile_count=tile_count, area=area, volume=volume, realm=realm)
& standard_name=standard_name, verbose=verbose, do_not_log=.not.(do_diag_field_log.AND.allow_log), &

@thomas-robinson thomas-robinson Dec 16, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is adding unneeded complexity. do_not_log is optional, so if it's not present, then it won't be passed on because it's optional in the next routine (register_diag_field_array) as well.

You should move this logic to where it's actually used instead of now actually passing something to do_not_log.

@rem1776 rem1776 Dec 16, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic basically prints the log if either the nml flag or do_not_log is set to output , I added it here cause I needed the nml flag and this was the last place where it would be accessible.

I'm gonna change it around anyway so I might take out the do_not_log from the object's register routine if it ends up being useless (which seems likely)

& err_msg=err_msg, interp_method=interp_method, tile_count=tile_count, area=area, volume=volume, realm=realm)
else
register_diag_field_array = register_diag_field_array_old(module_name, field_name, axes, init_time, &
& long_name=long_name, units=units, missing_value=missing_value, range=range, mask_variant=mask_variant, &
Expand Down Expand Up @@ -479,6 +486,7 @@ INTEGER FUNCTION register_static_field(module_name, field_name, axes, long_name,
!! with this field
CHARACTER(len=*), OPTIONAL, INTENT(in) :: realm !< String to set as the value to the
!! modeling_realm attribute
logical :: allow_log

! Fatal error if the module has not been initialized.
IF ( .NOT.module_is_initialized ) THEN
Expand All @@ -487,10 +495,16 @@ INTEGER FUNCTION register_static_field(module_name, field_name, axes, long_name,
END IF

if (use_modern_diag) then
! check if logging registered fields, uses arg and nml flag
IF ( PRESENT(do_not_log) ) THEN
allow_log = .NOT.do_not_log
ELSE
allow_log = .TRUE.
END IF
register_static_field = fms_diag_object%fms_register_static_field(module_name, field_name, axes, &
& long_name=long_name, units=units, missing_value=missing_value, range=range, mask_variant=mask_variant, &
& standard_name=standard_name, dynamic=DYNAMIC, do_not_log=do_not_log, interp_method=interp_method,&
& tile_count=tile_count, area=area, volume=volume, realm=realm)
& standard_name=standard_name, dynamic=DYNAMIC, do_not_log=.not.(do_diag_field_log.AND.allow_log), &

@thomas-robinson thomas-robinson Dec 16, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same comment as above.

& interp_method=interp_method, tile_count=tile_count, area=area, volume=volume, realm=realm)
else
register_static_field = register_static_field_old(module_name, field_name, axes, &
& long_name=long_name, units=units, missing_value=missing_value, range=range, mask_variant=mask_variant, &
Expand Down Expand Up @@ -3780,10 +3794,12 @@ END SUBROUTINE closing_file

!> @brief Initialize Diagnostics Manager.
!! @details Open and read diag_table. Select fields and files for diagnostic output.
SUBROUTINE diag_manager_init(diag_model_subset, time_init, err_msg)
SUBROUTINE diag_manager_init(diag_model_subset, time_init, err_msg, logfile_separator)
INTEGER, OPTIONAL, INTENT(IN) :: diag_model_subset
INTEGER, DIMENSION(6), OPTIONAL, INTENT(IN) :: time_init !< Model time diag_manager initialized
CHARACTER(len=*), INTENT(out), OPTIONAL :: err_msg
CHARACTER(len=1), INTENT(in), OPTIONAL :: logfile_separator !< character to use as a csv-style data field separator

@thomas-robinson thomas-robinson Dec 16, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could just as easily be a module or namelist variable. Then you don't need to change the interface at all

!! (defaults to '|')

CHARACTER(len=*), PARAMETER :: SEP = '|'

Expand Down Expand Up @@ -3914,7 +3930,7 @@ SUBROUTINE diag_manager_init(diag_model_subset, time_init, err_msg)
END IF

if (use_modern_diag) then
CALL fms_diag_object%init(diag_subset_output)
CALL fms_diag_object%init(diag_subset_output)
endif
if (.not. use_modern_diag) then
CALL parse_diag_table(DIAG_SUBSET=diag_subset_output, ISTAT=mystat, ERR_MSG=err_msg_local)
Expand All @@ -3928,12 +3944,18 @@ SUBROUTINE diag_manager_init(diag_model_subset, time_init, err_msg)

! open diag field log file
IF ( do_diag_field_log.AND.mpp_pe().EQ.mpp_root_pe() ) THEN
open(newunit=diag_log_unit, file='diag_field_log.out', action='WRITE')
WRITE (diag_log_unit,'(777a)') &
open(newunit=diag_log_unit, file='diag_field_log.out', action='WRITE')
if( .not. use_modern_diag) then
WRITE (diag_log_unit,'(777a)') &
& 'Module', SEP, 'Field', SEP, 'Long Name', SEP,&
& 'Units', SEP, 'Number of Axis', SEP, 'Time Axis', SEP,&
& 'Missing Value', SEP, 'Min Value', SEP, 'Max Value', SEP,&
& 'AXES LIST'
else
if( present(logfile_separator)) then
call fms_diag_object%fms_set_field_log_separator(logfile_separator)
endif
endif
END IF

module_is_initialized = .TRUE.
Expand Down
152 changes: 151 additions & 1 deletion diag_manager/fms_diag_object.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@
module fms_diag_object_mod
use mpp_mod, only: fatal, note, warning, mpp_error, mpp_pe, mpp_root_pe, stdout
use diag_data_mod, only: diag_null, diag_not_found, diag_not_registered, diag_registered_id, &
&DIAG_FIELD_NOT_FOUND, diag_not_registered, max_axes, TWO_D_DOMAIN
&DIAG_FIELD_NOT_FOUND, diag_not_registered, max_axes, TWO_D_DOMAIN, &
& diag_log_unit, CMOR_MISSING_VALUE, use_cmor
USE time_manager_mod, ONLY: set_time, set_date, get_time, time_type, OPERATOR(>=), OPERATOR(>),&
& OPERATOR(<), OPERATOR(==), OPERATOR(/=), OPERATOR(/), OPERATOR(+), ASSIGNMENT(=), get_date, &
& get_ticks_per_second
Expand All @@ -34,6 +35,7 @@ module fms_diag_object_mod
use fms_diag_buffer_mod
#endif
use mpp_domains_mod, only: domain1d, domain2d, domainUG, null_domain2d
use platform_mod
implicit none
private

Expand Down Expand Up @@ -70,13 +72,16 @@ module fms_diag_object_mod
procedure :: fms_get_diag_field_id_from_name
procedure :: fms_get_axis_name_from_id
procedure :: fms_diag_send_complete
procedure :: fms_set_field_log_separator
#ifdef use_yaml
procedure :: get_diag_buffer
#endif
end type fmsDiagObject_type

type (fmsDiagObject_type), target :: fms_diag_object

character(len=1) :: logfile_sep = '|' !< separator used in csv-style log file of registered fields

public :: fms_register_diag_field_obj
public :: fms_register_diag_field_scalar
public :: fms_register_diag_field_array
Expand Down Expand Up @@ -318,6 +323,10 @@ INTEGER FUNCTION fms_register_diag_field_array(this, module_name, field_name, ax
& axes=axes, longname=long_name, units=units, missing_value=missing_value, varrange=var_range, &
& mask_variant=mask_variant, standname=standard_name, do_not_log=do_not_log, err_msg=err_msg, &
& interp_method=interp_method, tile_count=tile_count, area=area, volume=volume, realm=realm)
if( present(do_not_log)) then
if( .not. do_not_log ) call fms_log_field_info(module_name, field_name, axes, long_name, units,&
& missing_value, range=var_range, dynamic=.true.)
end if
#endif
end function fms_register_diag_field_array

Expand Down Expand Up @@ -361,6 +370,10 @@ INTEGER FUNCTION fms_register_static_field(this, module_name, field_name, axes,
& longname=long_name, units=units, missing_value=missing_value, varrange=range, &
& standname=standard_name, do_not_log=do_not_log, area=area, volume=volume, realm=realm, &
& static=.true.)
if( present(do_not_log)) then
if( .not. do_not_log ) call fms_log_field_info(module_name, field_name, axes, long_name, units,&
& missing_value, range, dynamic=.false.)
end if
#endif
end function fms_register_static_field

Expand Down Expand Up @@ -673,4 +686,141 @@ subroutine dump_diag_obj( filename )
call mpp_error( FATAL, "You can not use the modern diag manager without compiling with -Duse_yaml")
#endif
end subroutine

!> @brief Writes brief diagnostic field info to the log file.
!! @details If the <TT>do_diag_field_log</TT> namelist parameter is .TRUE.,
!! then a line briefly describing diagnostic field is added to
!! the log file. Normally users should not call this subroutine
!! directly, since it is called by register_static_field and
!! register_diag_field if do_not_log is not set to .TRUE.. It is
!! used, however, in LM3 to avoid excessive logs due to the
!! number of fields registered for each of the tile types. LM3
!! code uses a do_not_log parameter in the registration calls,
!! and subsequently calls this subroutine to log field information
!! under a generic name.
SUBROUTINE fms_log_field_info(module_name, field_name, axes, long_name, units,&

@thomas-robinson thomas-robinson Dec 16, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this routine used by both old and new? Do we need two?

@rem1776 rem1776 Dec 16, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This routine is only used by the new. Its pretty similar to the old but some info needed to come from the diag object, also it has the changes for the separator and writes the header instead of doing that in the init.

My thinking was that if theres a new register routine in a separate file it would make sense to add a new log routine too. I think I'll change it though, it will simplifiy the do_not_log logic as well if I just make some changes to the original and use that for both.

& missing_value, range, dynamic )
CHARACTER(len=*), INTENT(in) :: module_name !< Module name
CHARACTER(len=*), INTENT(in) :: field_name !< Field name
INTEGER, DIMENSION(:), INTENT(in) :: axes !< Axis IDs
CHARACTER(len=*), OPTIONAL, INTENT(in) :: long_name !< Long name for field.
CHARACTER(len=*), OPTIONAL, INTENT(in) :: units !< Unit of field.
CLASS(*), OPTIONAL, INTENT(in) :: missing_value !< Missing value value.
CLASS(*), DIMENSION(:), OPTIONAL, INTENT(IN) :: range !< Valid range of values for field.
LOGICAL, OPTIONAL, INTENT(in) :: dynamic !< <TT>.TRUE.</TT> if field is not static.

logical, save :: wrote_header = .false. !< set if header was already written

! ---- local vars
CHARACTER(len=256) :: lmodule, lfield, lname, lunits
CHARACTER(len=64) :: lmissval, lmin, lmax
CHARACTER(len=8) :: numaxis, timeaxis
CHARACTER(len=256) :: axis_name, axes_list
INTEGER :: i
REAL :: missing_value_use !< Local copy of missing_value
REAL, DIMENSION(2) :: range_use !< Local copy of range
! return if disabled via nml, we'll just get the nml val from the original call
!IF ( .NOT.do_diag_field_log ) RETURN
IF ( mpp_pe().NE.mpp_root_pe() ) RETURN

! Fatal error if range is present and its extent is not 2.
IF ( PRESENT(range) ) THEN
IF ( SIZE(range) .NE. 2 ) THEN
CALL mpp_error (FATAL, 'fms_diag_object_mod::fms_log_field_info: extent of range should be 2')
END IF
END IF

lmodule = TRIM(module_name)
lfield = TRIM(field_name)

IF ( PRESENT(long_name) ) THEN
lname = TRIM(long_name)
ELSE
lname = ''
END IF

IF ( PRESENT(units) ) THEN
lunits = TRIM(units)
ELSE
lunits = ''
END IF

WRITE (numaxis,'(i1)') SIZE(axes)

IF (PRESENT(missing_value)) THEN
IF ( use_cmor ) THEN
WRITE (lmissval,*) CMOR_MISSING_VALUE
ELSE
SELECT TYPE (missing_value)
TYPE IS (real(kind=r4_kind))
missing_value_use = missing_value
TYPE IS (real(kind=r8_kind))
missing_value_use = real(missing_value)
CLASS DEFAULT
CALL mpp_error( FATAL, 'diag_util_mod::log_diag_field_info: The missing_value is not one of the supported'// &
& ' types of real(kind=4) or real(kind=8)')
END SELECT
WRITE (lmissval,*) missing_value_use
END IF
ELSE
lmissval = ''
ENDIF

IF ( PRESENT(range) ) THEN
SELECT TYPE (range)
TYPE IS (real(kind=r4_kind))
range_use = range
TYPE IS (real(kind=r8_kind))
range_use = real(range)
CLASS DEFAULT
CALL mpp_error('diag_util_mod::log_diag_field_info',&
& 'The range is not one of the supported types of real(kind=4) or real(kind=8)', FATAL)
END SELECT
WRITE (lmin,*) range_use(1)
WRITE (lmax,*) range_use(2)
ELSE
lmin = ''
lmax = ''
END IF

IF ( PRESENT(dynamic) ) THEN
IF (dynamic) THEN
timeaxis = 'T'
ELSE
timeaxis = 'F'
END IF
ELSE
timeaxis = ''
END IF

axes_list=''
DO i = 1, SIZE(axes)
axis_name = fms_diag_object%fms_get_axis_name_from_id(axes(i))
IF ( TRIM(axes_list) /= '' ) axes_list = TRIM(axes_list)//','
axes_list = TRIM(axes_list)//TRIM(axis_name)
END DO

if ( .not. wrote_header ) then
WRITE (diag_log_unit,'(777a)') &
& 'Module', logfile_sep, 'Field', logfile_sep, 'Long Name', logfile_sep,&
& 'Units', logfile_sep, 'Number of Axis', logfile_sep, 'Time Axis', logfile_sep,&
& 'Missing Value', logfile_sep, 'Min Value', logfile_sep, 'Max Value', logfile_sep,&
& 'AXES LIST'
wrote_header = .true.
endif

WRITE (diag_log_unit,'(777a)') &
& TRIM(lmodule), logfile_sep, TRIM(lfield), logfile_sep, TRIM(lname), logfile_sep,&
& TRIM(lunits), logfile_sep, TRIM(numaxis), logfile_sep, TRIM(timeaxis), logfile_sep,&
& TRIM(lmissval), logfile_sep, TRIM(lmin), logfile_sep, TRIM(lmax), logfile_sep,&
& TRIM(axes_list)
END SUBROUTINE fms_log_field_info

!> Staticly sets the separator for the log file if one is passed during initialization
subroutine fms_set_field_log_separator(this, separator)
class(fmsDiagObject_type) :: this
character(len=1), intent(in) :: separator
logfile_sep = separator
end subroutine fms_set_field_log_separator

end module fms_diag_object_mod
2 changes: 1 addition & 1 deletion test_fms/diag_manager/test_diag_manager2.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -646,7 +646,7 @@ _EOF
'


printf "&diag_manager_nml \n use_modern_diag = .true. \n/" | cat > input.nml
printf "&diag_manager_nml \n use_modern_diag = .true. \n do_diag_field_log = .true. \n/" | cat > input.nml
cat <<_EOF > diag_table.yaml
title: test_diag_manager
base_date: 2 1 1 0 0 0
Expand Down