Kokkos Function System #31653

NamjaeChoi · 2025-10-01T19:44:52Z

Functors are not dispatched but plugged inside the user's parallel code, so I can't think of a way to achieve static polymorphism here. I think I will have to introduce virtual functions minimally in the wrapper class, but any idea to avoid it would be welcome, if you have any. Of course, if the user specifies the exact object type instead of using the abstract type, they will be able to evaluate functors statically.

Concerns are performance and SYCL, but regarding SYCL, Intel is adding an extension to allow virtual functions on GPU. We may be able to leverage it when the time comes to support SYCL. I don't think we are interested in supporting SYCL for backends other than Intel GPU.

NamjaeChoi · 2025-10-21T19:46:55Z

@lindsayad My next thing to look into would be this. I will start from functions. Do you have any thoughts on this?

lindsayad · 2025-10-23T18:20:49Z

If we use virtuals, should we consider just adapting the current functor system?

NamjaeChoi · 2025-10-23T18:25:11Z

The problem with that is we can't construct MOOSE objects directly on device. If an object that has virtual methods is constructed on host, the vtable will be populated with host function pointers. What I'm trying to do here is to define a thin wrapper that has virtual methods and is light enough to be built on device, and let the wrapper call the wrapped functor methods statically.

lindsayad

I think this looks ok to me

framework/include/kokkos/base/KokkosFunctorWrapper.h

NamjaeChoi · 2025-10-23T20:34:45Z

Still working... just want to run test suite once

moosebuild · 2025-10-24T02:20:14Z

Job Documentation, step Docs: sync website on 2d163bd wanted to post the following:

View the site here

This comment will be updated on new commits.

moosebuild · 2025-10-24T04:34:01Z

Job Coverage, step Generate coverage on 2d163bd wanted to post the following:

Framework coverage

	0cc44f	#31653 2d163b
	Total	Total	+/-	New
Rate	86.02%	85.98%	-0.04%	100.00%
Hits	124557	124799	+242	493
Misses	20246	20354	+108	0

Diff coverage report

Full coverage report

Modules coverage

Coverage did not change

Full coverage reports

Reports

This comment will be updated on new commits.

lindsayad · 2025-10-24T20:25:07Z

going to unsubscribe for now. Mention me when you want more input or it's ready for review

NamjaeChoi · 2025-10-24T21:28:00Z

@lindsayad I might have to hold this. Virtual functions on device seem to require RDC to be fully functional.

NamjaeChoi · 2025-10-25T01:56:29Z

The code is currently in a state where CUDA will work when we have RDC.

NamjaeChoi · 2025-11-01T02:29:12Z

I don't want to indefinitely hold this PR for RDC. I will temporarily disable using virtual dispatch for GPU.

NamjaeChoi · 2025-11-03T20:32:59Z

@lindsayad You can review this. I temporarily blocked the code path that requires RDC with a runtime error as the ETA of it is unknown.

framework/doc/content/syntax/KokkosFunctions/index.md

framework/doc/content/source/kokkos/auxkernels/KokkosFunctionAux.md

framework/doc/content/source/kokkos/functions/KokkosConstantFunction.md

lindsayad · 2025-11-03T23:48:41Z

framework/doc/content/source/kokkos/functions/KokkosPiecewiseConstant.md

+!listing test/tests/kokkos/functions/piecewise_constant/kokkos_piecewise_constant.i start=[func_x_y] end=[] include-end=true
+
+!listing test/tests/kokkos/functions/piecewise_constant/kokkos_piecewise_constant.i start=[func_xy_data] end=[] include-end=true
+
+!listing test/tests/kokkos/functions/piecewise_constant/kokkos_piecewise_constant.i start=[func_csv] end=[] include-end=true
+
+!listing test/tests/kokkos/functions/piecewise_constant/kokkos_piecewise_constant.i start=[func_json] end=[] include-end=true


Suggested change

!listing test/tests/kokkos/functions/piecewise_constant/kokkos_piecewise_constant.i start=[func_x_y] end=[] include-end=true

!listing test/tests/kokkos/functions/piecewise_constant/kokkos_piecewise_constant.i start=[func_xy_data] end=[] include-end=true

!listing test/tests/kokkos/functions/piecewise_constant/kokkos_piecewise_constant.i start=[func_csv] end=[] include-end=true

!listing test/tests/kokkos/functions/piecewise_constant/kokkos_piecewise_constant.i start=[func_json] end=[] include-end=true

!listing test/tests/kokkos/functions/piecewise_constant/kokkos_piecewise_constant.i block=KokkosFunctions

I've grown suspicious that we're missing some doc hooks?

What do you mean?

Did you just not want to show the top-level block before? I was worried that there was a documentation issue if you used the syntax I suggested. But the fact that you made the changes I suggested makes me not worry

I just wanted to I show all the four possible data formats, but your change also serves the same purpose.

…lab#30655

framework/include/functions/FunctionInterface.h

framework/include/functions/PiecewiseTabularBase.h

framework/include/kokkos/actions/AddKokkosFunctionAction.h

framework/include/kokkos/base/KokkosFunctor.h

framework/include/problems/FEProblemBase.h

framework/src/functions/PiecewiseTabularInterface.C

lindsayad · 2025-11-04T20:02:56Z

framework/src/kokkos/actions/AddKokkosFunctionAction.K

+  if (fp.Parse(_name, vars) == -1) // -1 for success
+    mooseWarning("Function name '" + _name + "' could evaluate as a ParsedFunction");


I don't understand this warning. The actual name of the function in the input file is parsable? This warning should add more context if it remains. Like "name parses to a function... we don't use the name itself as a function; hopefully that was not your intent"

So the issue linked to this warning is #7659. Do you want to improve the message?

I would change it to an error in your new action. And then yea I think we could improve the message string in both the old action (which remains a warning in case someone somewhere is relying on the behavior) and your new action to describe the possible conflict outlined in #7659

framework/src/kokkos/base/KokkosFunctor.K

lindsayad · 2025-11-04T20:07:33Z

test/tests/kokkos/functions/constant_function/tests

+    type = 'Exodiff'
+    input = 'kokkos_constant_function.i'
+    exodiff = 'kokkos_constant_function_out.e'
+    requirement = 'The Kokkos function system shall include a constant function.'


Mention ability to retrieve abstract type

Hmm the context is a bit weird.

I would just like it mentioned somewhere given that you have a distinct requirement regarding the concrete type

moosebuild · 2025-11-04T21:07:57Z

Job Precheck, step Clang format on a3b2b2f wanted to post the following:

Your code requires style changes.

A patch was auto generated and copied here
You can directly apply the patch by running, in the top level of your repository:

curl -s https://mooseframework.inl.gov/docs/PRs/31653/clang_format/style.patch | git apply -v

Alternatively, with your repository up to date and in the top level of your repository:

git clang-format 0cc44f880140671a55e87b96a292d17d48977833

…b#30655

moosebuild · 2025-11-05T01:57:12Z

Job Test, step Results summary on 2d163bd wanted to post the following:

Framework test summary

Compared against 0cc44f8 in job civet.inl.gov/job/3354977.

Removed tests

Added tests

Test	Time (s)
`kokkos/functions/piecewise_constant.piecewise_constant/json`	1.46
`kokkos/functions/piecewise_constant.piecewise_constant/xy_data`	1.34
`kokkos/functions/piecewise_constant.piecewise_constant/csv`	1.31
`kokkos/functions/piecewise_constant.piecewise_constant/x_y`	1.27
`kokkos/functions/concrete_type.invalid_type`	1.26
`kokkos/functions/constant_function.test`	1.16
`kokkos/functions/concrete_type.concrete_type`	1.16
`kokkos/functions/default_function.test`	1.10

Run time changes

Modules test summary

Compared against 0cc44f8 in job civet.inl.gov/job/3354977.

Removed tests

Added tests

Run time changes

Test	Base (s)	Head (s)	+/-
`stochastic_tools/test:samplers/mcmc.mcmc_sampling/pmcmc_base_bounds`	2.93	4.76	+62.16%
`stochastic_tools/test:web_server_control.stochastic_control/batch_reset`	3.32	5.20	+56.61%
`stochastic_tools/test:samplers/mcmc.mcmc_sampling/pmcmc_base`	2.89	4.43	+53.63%

NamjaeChoi · 2025-11-05T17:39:09Z

Any other comments?

GiudGiud assigned NamjaeChoi Oct 2, 2025

NamjaeChoi force-pushed the kokkos_functor branch from c76eb6e to 2f65c74 Compare October 2, 2025 14:56

NamjaeChoi moved this to In progress in NEAMS MP CONNECT/GPU 26 Oct 15, 2025

NamjaeChoi added this to NEAMS MP CONNECT/GPU 26 Oct 15, 2025

lindsayad reviewed Oct 23, 2025

View reviewed changes

framework/include/kokkos/base/KokkosFunctorWrapper.h Show resolved Hide resolved

NamjaeChoi force-pushed the kokkos_functor branch from 2f65c74 to 3a865ff Compare October 23, 2025 20:29

NamjaeChoi changed the title ~~WIP: Kokkos Functor System~~ Kokkos Function System and Functor Scratch Oct 23, 2025

NamjaeChoi changed the title ~~Kokkos Function System and Functor Scratch~~ Kokkos Function System Oct 23, 2025

NamjaeChoi force-pushed the kokkos_functor branch 3 times, most recently from c723295 to 68df514 Compare October 24, 2025 00:13

NamjaeChoi force-pushed the kokkos_functor branch 3 times, most recently from 2d9d24b to 977be17 Compare October 24, 2025 19:19

NamjaeChoi force-pushed the kokkos_functor branch from 977be17 to 6e2a5af Compare October 24, 2025 23:31

Make FunctionBase class idaholab#30655

7302a2b

NamjaeChoi force-pushed the kokkos_functor branch from 6e2a5af to ca47cdc Compare November 1, 2025 02:27

NamjaeChoi force-pushed the kokkos_functor branch 3 times, most recently from f9d7ff6 to 1b25b33 Compare November 2, 2025 03:11

NamjaeChoi marked this pull request as ready for review November 3, 2025 20:32

NamjaeChoi requested review from loganharbour and roystgnr as code owners November 3, 2025 20:32

NamjaeChoi added 4 commits November 3, 2025 13:42

Kokkos functor and function base classes idaholab#30655

b1c9f5a

Refactor PiecewiseTabularBase idaholab#30655

052abc0

Add Kokkos functions to the problem idaholab#30655

20807f6

Kokkos function examples idaholab#30655

645e5c9

NamjaeChoi force-pushed the kokkos_functor branch 2 times, most recently from 679c2de to 1b668cb Compare November 3, 2025 20:49

lindsayad reviewed Nov 3, 2025

View reviewed changes

NamjaeChoi added 3 commits November 4, 2025 10:11

Update documentation idaholab#30655

31ae89b

Add tests idaholab#30655

b6f88ea

Temporarily disable retrieving abstract Kokkos function for GPU idaho…

33fca55

…lab#30655

NamjaeChoi force-pushed the kokkos_functor branch from 1b668cb to 33fca55 Compare November 4, 2025 17:11

lindsayad reviewed Nov 4, 2025

View reviewed changes

NamjaeChoi force-pushed the kokkos_functor branch 2 times, most recently from d03ea8b to 81360be Compare November 4, 2025 21:29

Add a device linking step to be prepared for RDC idaholab#30655

e961bd0

NamjaeChoi force-pushed the kokkos_functor branch from 81360be to fcb84b3 Compare November 4, 2025 21:51

NamjaeChoi added 2 commits November 4, 2025 15:11

Add a unit test that shows virtual function call works on GPU idahola…

44ef5bf

…b#30655

Address comments idaholab#30655

51802dc

NamjaeChoi force-pushed the kokkos_functor branch 2 times, most recently from 37a7a98 to 0e59f51 Compare November 4, 2025 23:33

Address comments 2 idaholab#30655

2d163bd

NamjaeChoi force-pushed the kokkos_functor branch from 0e59f51 to 2d163bd Compare November 5, 2025 00:49

moosebuild added the PR: Failed but allowed label Nov 5, 2025

		if (fp.Parse(_name, vars) == -1) // -1 for success
		mooseWarning("Function name '" + _name + "' could evaluate as a ParsedFunction");

Kokkos Function System #31653

Are you sure you want to change the base?

Kokkos Function System #31653

Uh oh!

Conversation

NamjaeChoi commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NamjaeChoi commented Oct 21, 2025

Uh oh!

lindsayad commented Oct 23, 2025

Uh oh!

NamjaeChoi commented Oct 23, 2025

Uh oh!

lindsayad left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

NamjaeChoi commented Oct 23, 2025

Uh oh!

moosebuild commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

moosebuild commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Framework coverage

Modules coverage

Full coverage reports

Uh oh!

lindsayad commented Oct 24, 2025

Uh oh!

NamjaeChoi commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NamjaeChoi commented Oct 25, 2025

Uh oh!

NamjaeChoi commented Nov 1, 2025

Uh oh!

NamjaeChoi commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

moosebuild commented Nov 4, 2025

Uh oh!

moosebuild commented Nov 5, 2025

Framework test summary

Removed tests

Added tests

NamjaeChoi commented Oct 1, 2025 •

edited

Loading

moosebuild commented Oct 24, 2025 •

edited

Loading

moosebuild commented Oct 24, 2025 •

edited

Loading

NamjaeChoi commented Oct 24, 2025 •

edited

Loading

NamjaeChoi commented Nov 3, 2025 •

edited

Loading