Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

debug GPU conservation #614

Closed
wants to merge 31 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
d8ee900
save states each day
juliasloan25 Feb 12, 2024
7505231
static map save all output
juliasloan25 Feb 12, 2024
5cd6041
save state every dt
juliasloan25 Feb 12, 2024
12ca519
t_end 200secs
juliasloan25 Feb 12, 2024
d30e856
debug script
juliasloan25 Feb 13, 2024
162c2cb
add debug plots; step one dt
juliasloan25 Feb 20, 2024
1eac65c
convert fields GPU to CPU for plots
juliasloan25 Mar 19, 2024
a9578c9
use gpu field for extrema
juliasloan25 Mar 19, 2024
426df4c
use unqualified plot()
juliasloan25 Mar 19, 2024
37c9e6b
create cpu space for debug plots
juliasloan25 Mar 19, 2024
a3cdf37
debug plots work cpu
juliasloan25 Mar 19, 2024
5896f4f
use cpu device
juliasloan25 Mar 19, 2024
a28c5be
output debug plots to artifacts dir
juliasloan25 Mar 19, 2024
4b48092
use Array(parent())
juliasloan25 Mar 20, 2024
bafd389
print more info
juliasloan25 Mar 20, 2024
4a1a780
runs on clima interactively
juliasloan25 Mar 20, 2024
0081813
rm cpu checks
juliasloan25 Mar 20, 2024
19ca907
don't access ocean integ for amip
juliasloan25 Mar 20, 2024
974bbcb
add cpu AMIP runs for comparison
juliasloan25 Mar 20, 2024
f7dc272
compare states at end of sim
juliasloan25 Mar 20, 2024
262df46
save states based on space
juliasloan25 Mar 21, 2024
ed8a3d1
comment out state saving
juliasloan25 Mar 21, 2024
60841a9
run for 1 day
juliasloan25 Mar 21, 2024
8f9aba1
1 day in driver
LenkaNovak Mar 21, 2024
e97569f
don't run compare jobs
juliasloan25 Mar 21, 2024
6146f45
set dt_rad: 2days
juliasloan25 Mar 21, 2024
122a128
t_end 400s, debug in coupling loop
juliasloan25 Mar 21, 2024
e3e2088
fix state saving
juliasloan25 Mar 21, 2024
58a1833
get device suffix correctly
juliasloan25 Mar 22, 2024
37ebde4
assert at end of comparison
juliasloan25 Mar 22, 2024
491499c
save on root only
juliasloan25 Mar 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
603 changes: 341 additions & 262 deletions .buildkite/pipeline.yml

Large diffs are not rendered by default.

22 changes: 22 additions & 0 deletions config/model_configs/amip_albedo_function.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
alpha_rayleigh_uh: 0
alpha_rayleigh_w: 10
apply_limiter: false
dt: "150secs"
dt_cpl: 150
dt_rad: "2days"
dt_save_to_sol: "1days"
dz_bottom: 30
dz_top: 3000
h_elem: 4
job_id: "amip_albedo_function"
land_albedo_type: "function"
mode_name: "amip"
moist: "equil"
precip_model: "0M"
rad: "gray"
rayleigh_sponge: true
run_name: "amip_albedo_function"
t_end: "400secs"
vert_diff: "true"
z_elem: 50
z_stretch: false
22 changes: 22 additions & 0 deletions config/model_configs/amip_albedo_static_map.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
alpha_rayleigh_uh: 0
alpha_rayleigh_w: 10
apply_limiter: false
dt: "150secs"
dt_cpl: 150
dt_rad: "2days"
dt_save_to_sol: "1days"
dz_bottom: 30
dz_top: 3000
h_elem: 4
job_id: "amip_albedo_static_map"
land_albedo_type: "map_static"
mode_name: "amip"
moist: "equil"
precip_model: "0M"
rad: "gray"
rayleigh_sponge: true
run_name: "amip_albedo_static_map"
t_end: "400secs"
vert_diff: "true"
z_elem: 50
z_stretch: false
22 changes: 22 additions & 0 deletions config/model_configs/amip_albedo_temporal_map.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
alpha_rayleigh_uh: 0
alpha_rayleigh_w: 10
apply_limiter: false
dt: "150secs"
dt_cpl: 150
dt_rad: "2days"
dt_save_to_sol: "1days"
dz_bottom: 30
dz_top: 3000
h_elem: 4
job_id: "amip_albedo_temporal_map"
land_albedo_type: "map_temporal"
mode_name: "amip"
moist: "equil"
precip_model: "0M"
rad: "gray"
rayleigh_sponge: true
run_name: "amip_albedo_temporal_map"
t_end: "400secs"
vert_diff: "true"
z_elem: 50
z_stretch: false
26 changes: 26 additions & 0 deletions config/model_configs/amip_target_topo_diagedmf_shortrun.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
FLOAT_TYPE: "Float32"
anim: false
apply_limiter: false
atmos_config_file: "config/longrun_configs/longrun_aquaplanet_rhoe_equil_55km_nz63_allsky_diagedmf_0M.yml"
dt: "100secs"
dt_cloud_fraction: "1hours"
dt_cpl: 100
dt_rad: "2days"
dt_save_state_to_disk: "1days"
dt_save_to_sol: "1days"
energy_check: false
hourly_checkpoint: true
idealized_insolation: false
job_id: "amip_target_topo_diagedmf_shortrun"
land_albedo_type: "map_temporal"
mode_name: "amip"
mono_surface: false
netcdf_output_at_levels: true
run_name: "amip_target_topo_diagedmf_shortrun"
start_date: "19790301"
surface_setup: "PrescribedSurface"
t_end: "400secs"
topo_smoothing: true
topography: "Earth"
turb_flux_partition: "CombinedStateFluxes"
use_reference_state: false
4 changes: 2 additions & 2 deletions config/model_configs/gpu_amip_albedo_function.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ alpha_rayleigh_w: 10
apply_limiter: false
dt: "150secs"
dt_cpl: 150
dt_rad: "1hours"
dt_rad: "2days"
dt_save_to_sol: "1days"
dz_bottom: 30
dz_top: 3000
Expand All @@ -16,7 +16,7 @@ precip_model: "0M"
rad: "gray"
rayleigh_sponge: true
run_name: "gpu_amip_albedo_function"
t_end: "300secs"
t_end: "400secs"
vert_diff: "true"
z_elem: 50
z_stretch: false
4 changes: 2 additions & 2 deletions config/model_configs/gpu_amip_albedo_static_map.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ alpha_rayleigh_w: 10
apply_limiter: false
dt: "150secs"
dt_cpl: 150
dt_rad: "1hours"
dt_rad: "2days"
dt_save_to_sol: "1days"
dz_bottom: 30
dz_top: 3000
Expand All @@ -16,7 +16,7 @@ precip_model: "0M"
rad: "gray"
rayleigh_sponge: true
run_name: "gpu_amip_albedo_static_map"
t_end: "300secs"
t_end: "400secs"
vert_diff: "true"
z_elem: 50
z_stretch: false
4 changes: 2 additions & 2 deletions config/model_configs/gpu_amip_albedo_temporal_map.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ alpha_rayleigh_w: 10
apply_limiter: false
dt: "150secs"
dt_cpl: 150
dt_rad: "1hours"
dt_rad: "2days"
dt_save_to_sol: "1days"
dz_bottom: 30
dz_top: 3000
Expand All @@ -16,7 +16,7 @@ precip_model: "0M"
rad: "gray"
rayleigh_sponge: true
run_name: "gpu_amip_albedo_temporal_map"
t_end: "300secs"
t_end: "400secs"
vert_diff: "true"
z_elem: 50
z_stretch: false
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ atmos_config_file: "config/longrun_configs/longrun_aquaplanet_rhoe_equil_55km_nz
dt: "100secs"
dt_cloud_fraction: "1hours"
dt_cpl: 100
dt_rad: "1hours"
dt_rad: "2days"
dt_save_state_to_disk: "1days"
dt_save_to_sol: "1days"
energy_check: false
Expand All @@ -19,7 +19,7 @@ netcdf_output_at_levels: true
run_name: "gpu_amip_target_topo_diagedmf_shortrun"
start_date: "19790301"
surface_setup: "PrescribedSurface"
t_end: "200secs"
t_end: "400secs"
topo_smoothing: true
topography: "Earth"
turb_flux_partition: "CombinedStateFluxes"
Expand Down
3 changes: 2 additions & 1 deletion config/model_configs/gpu_slabplanet_albedo_function.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ apply_limiter: false
conservation_softfail: true
dt: "200secs"
dt_cpl: 200
dt_rad: "2days"
dt_save_to_sol: "3600secs"
energy_check: true
h_elem: 4
Expand All @@ -13,5 +14,5 @@ mono_surface: true
precip_model: "0M"
rad: "gray"
run_name: "gpu_slabplanet_albedo_function"
t_end: "10days"
t_end: "400secs"
vert_diff: "true"
3 changes: 2 additions & 1 deletion config/model_configs/gpu_slabplanet_albedo_static_map.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ apply_limiter: false
conservation_softfail: true
dt: "200secs"
dt_cpl: 200
dt_rad: "2days"
dt_save_to_sol: "3600secs"
energy_check: true
h_elem: 4
Expand All @@ -13,5 +14,5 @@ mono_surface: true
precip_model: "0M"
rad: "gray"
run_name: "gpu_slabplanet_albedo_static_map"
t_end: "10days"
t_end: "400secs"
vert_diff: "true"
3 changes: 2 additions & 1 deletion config/model_configs/gpu_slabplanet_albedo_temporal_map.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ apply_limiter: false
conservation_softfail: true
dt: "200secs"
dt_cpl: 200
dt_rad: "2days"
dt_save_to_sol: "3600secs"
energy_check: true
h_elem: 4
Expand All @@ -13,5 +14,5 @@ mono_surface: true
precip_model: "0M"
rad: "gray"
run_name: "gpu_slabplanet_albedo_temporal_map"
t_end: "10days"
t_end: "400secs"
vert_diff: "true"
3 changes: 2 additions & 1 deletion config/model_configs/slabplanet_albedo_function.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
apply_limiter: false
dt: "200secs"
dt_cpl: 200
dt_rad: "2days"
dt_save_to_sol: "3600secs"
energy_check: true
h_elem: 4
Expand All @@ -12,5 +13,5 @@ mono_surface: true
precip_model: "0M"
rad: "gray"
run_name: "slabplanet_albedo_function"
t_end: "10days"
t_end: "400secs"
vert_diff: "true"
3 changes: 2 additions & 1 deletion config/model_configs/slabplanet_albedo_static_map.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
apply_limiter: false
dt: "200secs"
dt_cpl: 200
dt_rad: "2days"
dt_save_to_sol: "3600secs"
energy_check: true
h_elem: 4
Expand All @@ -12,5 +13,5 @@ mono_surface: true
precip_model: "0M"
rad: "gray"
run_name: "slabplanet_albedo_static_map"
t_end: "10days"
t_end: "400secs"
vert_diff: "true"
3 changes: 2 additions & 1 deletion config/model_configs/slabplanet_albedo_temporal_map.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
apply_limiter: false
dt: "200secs"
dt_cpl: 200
dt_rad: "2days"
dt_save_to_sol: "3600secs"
energy_check: true
h_elem: 4
Expand All @@ -12,5 +13,5 @@ mono_surface: true
precip_model: "0M"
rad: "gray"
run_name: "slabplanet_albedo_temporal_map"
t_end: "10days"
t_end: "400secs"
vert_diff: "true"
Loading