Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix restart simulation time #1302

Merged
merged 8 commits into from
Oct 3, 2024
Merged

Fix restart simulation time #1302

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
5f53e6d
Remove bug
blaisb Oct 2, 2024
216d261
Add a test for the previous bug
blaisb Oct 3, 2024
5914040
Add changelog entry
blaisb Oct 3, 2024
77c7c12
Fix indentation
blaisb Oct 3, 2024
e90c0e4
Update applications_tests/lethe-fluid-matrix-free/CMakeLists.txt
blaisb Oct 3, 2024
d033a4c
Apply prm identation
blaisb Oct 3, 2024
d615e0e
Fix test to be very sensitive
blaisb Oct 3, 2024
6d79adf
Indent prm file
lpsaavedra Oct 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,13 @@
All notable changes to the Lethe project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/).


## [Master] - 2024-10-03

### Fixed

- MINOR Simulations with time-dependent boundary conditions would not restart corresctly in the matrix-free solver because the initial guess of the solution was reset to zero. This is fixed now and the initial guess of the solution is identical. [#1302](https://github.com/chaos-polymtl/lethe/pull/1302)

## [Master] - 2024-09-30

### Fixed
Expand Down
10 changes: 10 additions & 0 deletions applications_tests/lethe-fluid-matrix-free/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,16 @@ file(COPY tgv_restart_bdf1/enstrophy.checkpoint DESTINATION "${CMAKE_CURRENT_BIN
file(COPY tgv_restart_bdf1/kinetic_energy.checkpoint DESTINATION "${CMAKE_CURRENT_BINARY_DIR}/tgv_restart_bdf1.${_build_type}")
file(COPY tgv_restart_bdf1/L2Error_FD.checkpoint DESTINATION "${CMAKE_CURRENT_BINARY_DIR}/tgv_restart_bdf1.${_build_type}")


blaisb marked this conversation as resolved.
Show resolved Hide resolved

file(COPY turbulent_taylor_couette_restart/restart.pvdhandler DESTINATION "${CMAKE_CURRENT_BINARY_DIR}/turbulent_taylor_couette_restart.${_build_type}")
file(COPY turbulent_taylor_couette_restart/restart.simulationcontrol DESTINATION "${CMAKE_CURRENT_BINARY_DIR}/turbulent_taylor_couette_restart.${_build_type}")
file(COPY turbulent_taylor_couette_restart/restart.triangulation DESTINATION "${CMAKE_CURRENT_BINARY_DIR}/turbulent_taylor_couette_restart.${_build_type}")
file(COPY turbulent_taylor_couette_restart/restart.triangulation_fixed.data DESTINATION "${CMAKE_CURRENT_BINARY_DIR}/turbulent_taylor_couette_restart.${_build_type}")
file(COPY turbulent_taylor_couette_restart/restart.triangulation.info DESTINATION "${CMAKE_CURRENT_BINARY_DIR}/turbulent_taylor_couette_restart.${_build_type}")
file(COPY turbulent_taylor_couette_restart/enstrophy.checkpoint DESTINATION "${CMAKE_CURRENT_BINARY_DIR}/turbulent_taylor_couette_restart.${_build_type}")
file(COPY turbulent_taylor_couette_restart/kinetic_energy.checkpoint DESTINATION "${CMAKE_CURRENT_BINARY_DIR}/turbulent_taylor_couette_restart.${_build_type}")

deal_ii_pickup_tests()

if(CMAKE_BUILD_TYPE STREQUAL "Debug")
Expand Down
161 changes: 161 additions & 0 deletions applications_tests/lethe-fluid-matrix-free/generators/turbulent_taylor_couette_generator.prm
View file
Open in desktop
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this generator exactly? If we are adding the restart files anyways... Is it always needed? The tgv restart test does not have this, do we need to add it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because if we change the version of p4est we need to regenerate all of the restart files. With Audrey we decided to keep the generators because of that. Ultimately I would love to migrate to a stage where we generate the restart files within the tests themselves instead of keeping them on the repository.

Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
# Listing of Parameters
#----------------------

set dimension = 3

#---------------------------------------------------
# Simulation Control
#---------------------------------------------------

subsection simulation control
set method = bdf2
set output name = taylor-couette
set output path = ./output/
set time end = 10.5
set time step = 0.01
set output frequency = 0
end

#------------------------ ---------------------------
# Physical Properties
#---------------------------------------------------

subsection physical properties
subsection fluid 0
set kinematic viscosity = 6.25e-05
end
end

subsection restart
set checkpoint = true
set frequency = 10
set filename = restart
set restart = false
blaisb marked this conversation as resolved.
Show resolved Hide resolved
end

#---------------------------------------------------
# Initial conditions
#---------------------------------------------------

subsection initial conditions
set type = nodal
subsection uvwp
set Function constants = epsilon=0.1, ri=0.5, omega=1.0, d=0.5 , A= -0.3333333333333333, B= 0.3333333333333333
set Function expression = cos(atan2(y,x))*(epsilon*omega*ri*cos(atan2(y,x))*sin(((sqrt(x*x+y*y)-ri)*pi)/ri)*sin(z/d)) - sin(atan2(y,x))*(A*(sqrt(x*x+y*y)) + B/(sqrt(x*x+y*y)) + epsilon*omega*ri*sin(atan2(y,x))*sin(((sqrt(x*x+y*y)-ri)*pi)/ri)*sin(z/d)); sin(atan2(y,x))*(epsilon*omega*ri*cos(atan2(y,x))*sin(((sqrt(x*x+y*y)-ri)*pi)/ri)*sin(z/d)) + cos(atan2(y,x))*(A*(sqrt(x*x+y*y)) + B/(sqrt(x*x+y*y)) + epsilon*omega*ri*sin(atan2(y,x))*sin(((sqrt(x*x+y*y)-ri)*pi)/ri)*sin(z/d)); 0.0; ((0.5*A*A*(x*x+y*y)) + (2*A*B*ln(sqrt(x*x+y*y)))) - (0.5*B*B/(x*x+y*y)) + (0.5*(epsilon*omega*ri)*(epsilon*omega*ri)*cos(2*atan2(y,x))*sin((2*(sqrt(x*x+y*y)-ri)*pi)/ri)*sin(2*z/d))
end
end

#---------------------------------------------------
# Mesh
#---------------------------------------------------

subsection mesh
set type = dealii
set grid type = cylinder_shell
set grid arguments = 3.14159265359 : 0.5 : 1.0 : 5 : 4 : true
set initial refinement = 1
end

#---------------------------------------------------
# FEM
#---------------------------------------------------

subsection FEM
set velocity order = 2
set pressure order = 2
end

#---------------------------------------------------
# Post-Processing
#---------------------------------------------------

subsection post-processing
set verbosity = verbose
set calculate kinetic energy = true
set calculate enstrophy = true
set calculate average velocities = false
set initial time = 0.
end

#---------------------------------------------------
# Boundary Conditions
#---------------------------------------------------

subsection boundary conditions
set number = 3
set fix pressure constant = true
set time dependent = true
subsection bc 0
set type = function
set beta = 1
subsection u
set Function expression = -y
end
subsection v
set Function expression = x
end
subsection w
set Function expression = 0
end
end
subsection bc 1
set type = noslip
end
subsection bc 2
set type = periodic
set id = 2
set periodic_id = 3
set periodic_direction = 2
end
end

#---------------------------------------------------
# Non-Linear Solver Control
#---------------------------------------------------

subsection non-linear solver
subsection fluid dynamics
set tolerance = 1e-3
set verbosity = quiet
end
end

#---------------------------------------------------
# Linear Solver Control
#---------------------------------------------------

subsection linear solver
subsection fluid dynamics
set method = gmres
set max iters = 100
set relative residual = 1e-4
set minimum residual = 1e-7
set preconditioner = gcmg
set verbosity = quiet

# MG parameters
set mg verbosity = quiet
set mg min level = -1
set mg level min cells = 16

# Smoother
set mg smoother iterations = 10
set mg smoother eig estimation = true

# Eigenvalue estimation parameters
set eig estimation smoothing range = 5
set eig estimation cg n iterations = 20
set eig estimation verbosity = quiet

# Coarse-grid solver
set mg coarse grid solver = gmres
set mg gmres max iterations = 2000
set mg gmres tolerance = 1e-7
set mg gmres reduce = 1e-4
set mg gmres max krylov vectors = 30
set mg gmres preconditioner = ilu
set ilu preconditioner fill = 1
set ilu preconditioner absolute tolerance = 1e-10
set ilu preconditioner relative tolerance = 1.00
Comment on lines +150 to +159
Copy link
Collaborator

@lpsaavedra lpsaavedra Oct 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we replace this by a direct solver? So that it is the same as the other one?
set mg coarse grid solver = direct

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

end
end
10 changes: 10 additions & 0 deletions applications_tests/lethe-fluid-matrix-free/turbulent_taylor_couette_restart.output
View file
Open in desktop
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand how we can check from this output that the initial guess is correct, since we are not printing anything there. Shouldn't we maybe print the L2 norm of the vector or something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On my computer the iteration did not converge at all when the initial guess was not correct. So the simulation would just crash instead (because the initial guess of 0 is so far away from the state of the simulation).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You were right, the test was not sensitive enough. I have fixed it to add the monitoring of the tolerance of the Newton step.
The new test breaks on master and work on this branch. So to me this is a very good measure of the quality of the test.
Master:

Fluid Dynamics

Newton iteration: 0 - Residual: 6.45982
alpha = 1 res = 0.008684
||du||_L2 = 7.134 ||du||_Linfty = 0.3355
||dp||_L2 = 0.3082 ||dp||_Linfty = 0.01942
1:53
is given by master
1:53

On the branch:

Fluid Dynamics

Newton iteration: 0 - Residual: 0.00353967
alpha = 1 res = 7.206e-09
||du||_L2 = 0.004831 ||du||_Linfty = 0.0003414
||dp||_L2 = 0.0008221 ||dp||_Linfty = 6.221e-05
Enstrophy : 0.3507
Kinetic energy : 0.02601

Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
Running on 1 MPI rank(s)...
Number of active cells: 20
Number of degrees of freedom: 1080
Volume of triangulation: 7.36551
************************
---> Simulation Restart
************************
Number of active cells: 160
Number of degrees of freedom: 6800
Volume of triangulation: 7.39983
165 changes: 165 additions & 0 deletions applications_tests/lethe-fluid-matrix-free/turbulent_taylor_couette_restart.prm
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
# Listing of Parameters
#----------------------

set dimension = 3

#---------------------------------------------------
# Simulation Control
#---------------------------------------------------

subsection simulation control
set method = bdf2
set output name = taylor-couette
set output path = ../
set time end = 10.6
set time step = 0.01
set output frequency = 0
end

#------------------------ ---------------------------
# Physical Properties
#---------------------------------------------------

subsection physical properties
subsection fluid 0
set kinematic viscosity = 6.25e-05
end
end

#---------------------------------------------------
# Restart
#---------------------------------------------------

subsection restart
set checkpoint = false
set frequency = 10
set filename = restart
set restart = true
blaisb marked this conversation as resolved.
Show resolved Hide resolved
end

#---------------------------------------------------
# Initial conditions
#---------------------------------------------------

subsection initial conditions
set type = nodal
subsection uvwp
set Function constants = epsilon=0.1, ri=0.5, omega=1.0, d=0.5 , A= -0.3333333333333333, B= 0.3333333333333333
set Function expression = cos(atan2(y,x))*(epsilon*omega*ri*cos(atan2(y,x))*sin(((sqrt(x*x+y*y)-ri)*pi)/ri)*sin(z/d)) - sin(atan2(y,x))*(A*(sqrt(x*x+y*y)) + B/(sqrt(x*x+y*y)) + epsilon*omega*ri*sin(atan2(y,x))*sin(((sqrt(x*x+y*y)-ri)*pi)/ri)*sin(z/d)); sin(atan2(y,x))*(epsilon*omega*ri*cos(atan2(y,x))*sin(((sqrt(x*x+y*y)-ri)*pi)/ri)*sin(z/d)) + cos(atan2(y,x))*(A*(sqrt(x*x+y*y)) + B/(sqrt(x*x+y*y)) + epsilon*omega*ri*sin(atan2(y,x))*sin(((sqrt(x*x+y*y)-ri)*pi)/ri)*sin(z/d)); 0.0; ((0.5*A*A*(x*x+y*y)) + (2*A*B*ln(sqrt(x*x+y*y)))) - (0.5*B*B/(x*x+y*y)) + (0.5*(epsilon*omega*ri)*(epsilon*omega*ri)*cos(2*atan2(y,x))*sin((2*(sqrt(x*x+y*y)-ri)*pi)/ri)*sin(2*z/d))
end
end

#---------------------------------------------------
# Mesh
#---------------------------------------------------

subsection mesh
set type = dealii
set grid type = cylinder_shell
set grid arguments = 3.14159265359 : 0.5 : 1.0 : 5 : 4 : true
set initial refinement = 1
end

#---------------------------------------------------
# FEM
#---------------------------------------------------

subsection FEM
set velocity order = 2 #3 for Q3
set pressure order = 2 #3 for Q3
end

#---------------------------------------------------
# Post-Processing
#---------------------------------------------------

subsection post-processing
set verbosity = verbose
set calculate kinetic energy = true
set calculate enstrophy = true
set calculate average velocities = false
set initial time = 0.
end

#---------------------------------------------------
# Boundary Conditions
#---------------------------------------------------

subsection boundary conditions
set number = 3
set fix pressure constant = true
set time dependent = true
subsection bc 0
set type = function
set beta = 1
subsection u
set Function expression = -y
end
subsection v
set Function expression = x
end
subsection w
set Function expression = 0
end
end
subsection bc 1
set type = noslip
end
subsection bc 2
set type = periodic
set id = 2
set periodic_id = 3
set periodic_direction = 2
end
end

#---------------------------------------------------
# Non-Linear Solver Control
#---------------------------------------------------

subsection non-linear solver
subsection fluid dynamics
set tolerance = 1e-3
set verbosity = quiet
end
end

#---------------------------------------------------
# Linear Solver Control
#---------------------------------------------------

subsection linear solver
subsection fluid dynamics
set method = gmres
set max iters = 100
set relative residual = 1e-4
set minimum residual = 1e-7
set preconditioner = gcmg
set verbosity = quiet

# MG parameters
set mg verbosity = quiet
set mg min level = -1
set mg level min cells = 16

# Smoother
set mg smoother iterations = 10
set mg smoother eig estimation = true

# Eigenvalue estimation parameters
set eig estimation smoothing range = 5
set eig estimation cg n iterations = 20
set eig estimation verbosity = quiet

# Coarse-grid solver
set mg coarse grid solver = gmres
set mg gmres max iterations = 2000
set mg gmres tolerance = 1e-7
set mg gmres reduce = 1e-4
set mg gmres max krylov vectors = 30
set mg gmres preconditioner = ilu
set ilu preconditioner fill = 1
set ilu preconditioner absolute tolerance = 1e-10
set ilu preconditioner relative tolerance = 1.00
end
end
1 change: 1 addition & 0 deletions ...tions_tests/lethe-fluid-matrix-free/turbulent_taylor_couette_restart/enstrophy.checkpoint
View file
Open in desktop

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions ..._tests/lethe-fluid-matrix-free/turbulent_taylor_couette_restart/kinetic_energy.checkpoint
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions ...cations_tests/lethe-fluid-matrix-free/turbulent_taylor_couette_restart/restart.pvdhandler
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
0
Time File
8 changes: 8 additions & 0 deletions ..._tests/lethe-fluid-matrix-free/turbulent_taylor_couette_restart/restart.simulationcontrol
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Simulation control
dt_0 0.01
dt_1 0.01
dt_2 0.01
dt_3 0.01
CFL 0.0219711
Time 10.6
Iter 1061
Binary file added ...ions_tests/lethe-fluid-matrix-free/turbulent_taylor_couette_restart/restart.triangulation
View file
Open in desktop
Binary file not shown.
2 changes: 2 additions & 0 deletions ...tests/lethe-fluid-matrix-free/turbulent_taylor_couette_restart/restart.triangulation.info
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
version nproc n_attached_fixed_size_objs n_attached_variable_size_objs n_coarse_cells
5 1 1 0 20
Binary file added ...lethe-fluid-matrix-free/turbulent_taylor_couette_restart/restart.triangulation_fixed.data
View file
Open in desktop
Binary file not shown.
Loading
Loading