2022.07.28 Meeting Notes

Agenda

• Individual/group updates
• Face/edge centered fields

Individual/group updates

JM

• still figuring out performance issue for prolongation/restriction
• different approaches are good/bad for small/large pack sizes and small/large block sizes
• new abstraction for prolong/restriction should also help in designing face/edge fields

LR

• sparse bound done (also removed old machinery) at same or better performance
• also allows to tune order of communication
• new way for individual alloc/decalloc and init of sparse vars

AJ

• calculating chi^2 for various parameters like numbers of block per rank
• collecting traces for messages, too (in addition to the task based ones)

FG

• working on face and edge centered fields, see below

PG

• got scaling data on Frontier, results should go in the paper
  • packing issue above also relevant for performance here
• updated CI environment to use GitHub Actions rather than Gitlab

Face/edge centered fields

• mock-up downstream app for fake-loop advection for illustration
• agreement that new "sparse" pack should be used (and old packing machinery potentially go away entirely at some point)
• for indexing we should stick to current convention (block first,then variables, then kji)
• big question: just CellVariables with new distinguishing MetaData (and nx#+1) or different variables types (or using substructures inside CellVariables)
  • CellVariables approach seems like least complicated
• will schedule online hackathon (likely in Sep)

Long term ideas

• autotuning (of buffer filling routines, comm paths, pack_size, ...)
• set pack size by number of packs per rank
• make sure GetOrAdd is only called with default number of packs
• in-situ/inline analysis and post-processing
• separate output "thread"
• remove "non-pack" machinery (enforce index b) to reduce interface

Next meeting in tbd.