Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve timer output for the fluid applications #1187

Merged
merged 8 commits into from
Jul 8, 2024
Merged

Improve timer output for the fluid applications #1187

merged 8 commits into from
Jul 8, 2024

Conversation

lpsaavedra
Copy link
Collaborator

@lpsaavedra lpsaavedra commented Jul 4, 2024

Description

Trying to analyze times for different benchmarks using the matrix-free application, I realized that there were time sections missing, i.e., there was 10-15% of the total time that was not reported in the final time table. In addition, the functions to update multiphysics vectors were always being executed even if only the fluid dynamics was enabled (in these calls we convert from deal.II to Trilinos vectors), which lead to unnecessary time spent there.

Solution

Added additional time sections so that the time summary table reflects the overall time reported at the top. The changes were also applied to the matrix-based application so that they can be easily compared. The matrix-free application now checks whether other physics are enabled or not to only perform the update of the multiphysics vectors when needed.

As examples of the new timer output, this is how it would look like for the TGV case:

  • For lethe-fluid:
+---------------------------------------------------------+------------+------------+
| Total wallclock time elapsed since start                |        16s |            |
|                                                         |            |            |
| Section                                     | no. calls |  wall time | % of total |
+---------------------------------------------+-----------+------------+------------+
| Assemble RHS                                |       335 |      4.43s |        28% |
| Assemble matrix                             |       224 |      5.31s |        33% |
| Calculate CFL and percolate time vectors    |       112 |    0.0411s |      0.26% |
| Calculate and output norms after Newton its |       224 |    0.0336s |      0.21% |
| Calculate enstrophy                         |       112 |    0.0717s |      0.45% |
| Calculate kinetic energy                    |       112 |    0.0514s |      0.32% |
| Distribute constraints after linear solve   |       224 |   0.00389s |         0% |
| Read mesh and manifolds                     |         1 |   0.00292s |         0% |
| Set initial conditions                      |         1 |  7.72e-07s |         0% |
| Setup DOFs                                  |         1 |   0.00489s |         0% |
| Setup ILU                                   |       111 |      5.78s |        36% |
| Solve linear system                         |       224 |     0.254s |       1.6% |
| Write checkpoint                            |         1 |   0.00411s |         0% |
+---------------------------------------------+-----------+------------+------------+
  • For lethe-fluid-matrix-free:
+----------------------------------------------------------+------------+------------+
| Total wallclock time elapsed since start                 |      8.17s |            |
|                                                          |            |            |
| Section                                      | no. calls |  wall time | % of total |
+----------------------------------------------+-----------+------------+------------+
| Assemble RHS                                 |       338 |    0.0937s |       1.1% |
| Assemble matrix                              |       227 |  0.000153s |         0% |
| Calculate CFL and percolate time vectors     |       112 |    0.0349s |      0.43% |
| Calculate and output norms after Newton its  |       227 |    0.0309s |      0.38% |
| Calculate enstrophy                          |       112 |    0.0642s |      0.79% |
| Calculate kinetic energy                     |       112 |    0.0447s |      0.55% |
| Calculate time derivative previous solutions |       111 |   0.00199s |         0% |
| Distribute constraints after linear solve    |       227 |   0.00216s |         0% |
| Evaluate non linear term and tau             |       111 |    0.0124s |      0.15% |
| Read mesh and manifolds                      |         1 |    0.0029s |         0% |
| Set initial conditions                       |         1 |  6.51e-07s |         0% |
| Setup DoFs                                   |         1 |   0.00221s |         0% |
| Setup GMG                                    |       111 |       2.4s |        29% |
| Solve linear system                          |       227 |      5.46s |        67% |
+----------------------------------------------+-----------+------------+------------+

Testing

No time outputs are present in any test. However, I tested this with two of the matrix-free benchmarks (steady and transient) to verify that everything works as it should when outputting the timer per iteration or at the end of the simulation.

Documentation

The Timer section in the General, CFD and Multiphysics parameters section of the documentation was updated.

Miscellaneous (will be removed when merged)

Checklist (will be removed when merged)

See this page for more information about the pull request process.

Code related list:

  • All in-code documentation related to this PR is up to date (Doxygen format)
  • Lethe documentation is up to date
  • The branch is rebased onto master
  • Changelog (CHANGELOG.md) is up to date
  • Code is indented with indent-all and .prm files (examples and tests) with prm-indent

Pull request related list:

  • Labels are applied
  • There are at least 2 reviewers (or 1 if small feature) excluding the responsible for the merge
  • If this PR closes an issue or is related to a project, it is linked in the "Projects" or "Development" section
  • The PR description is cleaned and ready for merge

@lpsaavedra lpsaavedra added the WIP When a PR is open but not ready for review label Jul 4, 2024
@lpsaavedra lpsaavedra changed the title [WIP] Improve timer output for the lethe-fluid and lethe-fluid-matrix-free applications [WIP] Improve timer output for the fluid applications Jul 5, 2024
@lpsaavedra lpsaavedra added the Enhancement New feature or request label Jul 5, 2024
@lpsaavedra lpsaavedra changed the title [WIP] Improve timer output for the fluid applications Improve timer output for the fluid applications Jul 8, 2024
@lpsaavedra lpsaavedra requested a review from blaisb July 8, 2024 13:19
@lpsaavedra lpsaavedra added Ready for review and removed WIP When a PR is open but not ready for review labels Jul 8, 2024
@blaisb blaisb merged commit ef14246 into master Jul 8, 2024
8 checks passed
@blaisb blaisb deleted the fix_timings branch July 8, 2024 19:21
M-Badri pushed a commit to M-Badri/lethe that referenced this pull request Sep 29, 2024
escription
Trying to analyze times for different benchmarks using the matrix-free application, I realized that there were time sections missing, i.e., there was 10-15% of the total time that was not reported in the final time table. In addition, the functions to update multiphysics vectors were always being executed even if only the fluid dynamics was enabled (in these calls we convert from deal.II to Trilinos vectors), which lead to unnecessary time spent there.

Solution
Added additional time sections so that the time summary table reflects the overall time reported at the top. The changes were also applied to the matrix-based application so that they can be easily compared. The matrix-free application now checks whether other physics are enabled or not to only perform the update of the multiphysics vectors when needed.

As examples of the new timer output, this is how it would look like for the TGV case:

For lethe-fluid:
+---------------------------------------------------------+------------+------------+
| Total wallclock time elapsed since start                |        16s |            |
|                                                         |            |            |
| Section                                     | no. calls |  wall time | % of total |
+---------------------------------------------+-----------+------------+------------+
| Assemble RHS                                |       335 |      4.43s |        28% |
| Assemble matrix                             |       224 |      5.31s |        33% |
| Calculate CFL and percolate time vectors    |       112 |    0.0411s |      0.26% |
| Calculate and output norms after Newton its |       224 |    0.0336s |      0.21% |
| Calculate enstrophy                         |       112 |    0.0717s |      0.45% |
| Calculate kinetic energy                    |       112 |    0.0514s |      0.32% |
| Distribute constraints after linear solve   |       224 |   0.00389s |         0% |
| Read mesh and manifolds                     |         1 |   0.00292s |         0% |
| Set initial conditions                      |         1 |  7.72e-07s |         0% |
| Setup DOFs                                  |         1 |   0.00489s |         0% |
| Setup ILU                                   |       111 |      5.78s |        36% |
| Solve linear system                         |       224 |     0.254s |       1.6% |
| Write checkpoint                            |         1 |   0.00411s |         0% |
+---------------------------------------------+-----------+------------+------------+

For lethe-fluid-matrix-free:
+----------------------------------------------------------+------------+------------+
| Total wallclock time elapsed since start                 |      8.17s |            |
|                                                          |            |            |
| Section                                      | no. calls |  wall time | % of total |
+----------------------------------------------+-----------+------------+------------+
| Assemble RHS                                 |       338 |    0.0937s |       1.1% |
| Assemble matrix                              |       227 |  0.000153s |         0% |
| Calculate CFL and percolate time vectors     |       112 |    0.0349s |      0.43% |
| Calculate and output norms after Newton its  |       227 |    0.0309s |      0.38% |
| Calculate enstrophy                          |       112 |    0.0642s |      0.79% |
| Calculate kinetic energy                     |       112 |    0.0447s |      0.55% |
| Calculate time derivative previous solutions |       111 |   0.00199s |         0% |
| Distribute constraints after linear solve    |       227 |   0.00216s |         0% |
| Evaluate non linear term and tau             |       111 |    0.0124s |      0.15% |
| Read mesh and manifolds                      |         1 |    0.0029s |         0% |
| Set initial conditions                       |         1 |  6.51e-07s |         0% |
| Setup DoFs                                   |         1 |   0.00221s |         0% |
| Setup GMG                                    |       111 |       2.4s |        29% |
| Solve linear system                          |       227 |      5.46s |        67% |
+----------------------------------------------+-----------+------------+------------+
Testing
No time outputs are present in any test. However, I tested this with two of the matrix-free benchmarks (steady and transient) to verify that everything works as it should when outputting the timer per iteration or at the end of the simulation.

Documentation
The Timer section in the General, CFD and Multiphysics  parameters section of the documentation was updated.

Former-commit-id: ef14246
blaisb pushed a commit that referenced this pull request Sep 30, 2024
escription
Trying to analyze times for different benchmarks using the matrix-free application, I realized that there were time sections missing, i.e., there was 10-15% of the total time that was not reported in the final time table. In addition, the functions to update multiphysics vectors were always being executed even if only the fluid dynamics was enabled (in these calls we convert from deal.II to Trilinos vectors), which lead to unnecessary time spent there.

Solution
Added additional time sections so that the time summary table reflects the overall time reported at the top. The changes were also applied to the matrix-based application so that they can be easily compared. The matrix-free application now checks whether other physics are enabled or not to only perform the update of the multiphysics vectors when needed.

As examples of the new timer output, this is how it would look like for the TGV case:

For lethe-fluid:
+---------------------------------------------------------+------------+------------+
| Total wallclock time elapsed since start                |        16s |            |
|                                                         |            |            |
| Section                                     | no. calls |  wall time | % of total |
+---------------------------------------------+-----------+------------+------------+
| Assemble RHS                                |       335 |      4.43s |        28% |
| Assemble matrix                             |       224 |      5.31s |        33% |
| Calculate CFL and percolate time vectors    |       112 |    0.0411s |      0.26% |
| Calculate and output norms after Newton its |       224 |    0.0336s |      0.21% |
| Calculate enstrophy                         |       112 |    0.0717s |      0.45% |
| Calculate kinetic energy                    |       112 |    0.0514s |      0.32% |
| Distribute constraints after linear solve   |       224 |   0.00389s |         0% |
| Read mesh and manifolds                     |         1 |   0.00292s |         0% |
| Set initial conditions                      |         1 |  7.72e-07s |         0% |
| Setup DOFs                                  |         1 |   0.00489s |         0% |
| Setup ILU                                   |       111 |      5.78s |        36% |
| Solve linear system                         |       224 |     0.254s |       1.6% |
| Write checkpoint                            |         1 |   0.00411s |         0% |
+---------------------------------------------+-----------+------------+------------+

For lethe-fluid-matrix-free:
+----------------------------------------------------------+------------+------------+
| Total wallclock time elapsed since start                 |      8.17s |            |
|                                                          |            |            |
| Section                                      | no. calls |  wall time | % of total |
+----------------------------------------------+-----------+------------+------------+
| Assemble RHS                                 |       338 |    0.0937s |       1.1% |
| Assemble matrix                              |       227 |  0.000153s |         0% |
| Calculate CFL and percolate time vectors     |       112 |    0.0349s |      0.43% |
| Calculate and output norms after Newton its  |       227 |    0.0309s |      0.38% |
| Calculate enstrophy                          |       112 |    0.0642s |      0.79% |
| Calculate kinetic energy                     |       112 |    0.0447s |      0.55% |
| Calculate time derivative previous solutions |       111 |   0.00199s |         0% |
| Distribute constraints after linear solve    |       227 |   0.00216s |         0% |
| Evaluate non linear term and tau             |       111 |    0.0124s |      0.15% |
| Read mesh and manifolds                      |         1 |    0.0029s |         0% |
| Set initial conditions                       |         1 |  6.51e-07s |         0% |
| Setup DoFs                                   |         1 |   0.00221s |         0% |
| Setup GMG                                    |       111 |       2.4s |        29% |
| Solve linear system                          |       227 |      5.46s |        67% |
+----------------------------------------------+-----------+------------+------------+
Testing
No time outputs are present in any test. However, I tested this with two of the matrix-free benchmarks (steady and transient) to verify that everything works as it should when outputting the timer per iteration or at the end of the simulation.

Documentation
The Timer section in the General, CFD and Multiphysics  parameters section of the documentation was updated.

Former-commit-id: ef14246
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request Ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants