Re-developed makedep in python #158
Merged
Merged
Conversation
- Works with system python (2.7) and later (tested with 3.9)
- Processes #include without invoking cpp
- Currently naive regex without honoring/interpretting CPP macros
- Calculates link dependencies directly rather than invoking "make"
to infer the link list
Changes relative to the bash version of makedep:
- No longer adding/using a macro "SRC_DIRS" since it was only used to
record the arguments for regenerating the makefile. Now we record the
command line to achieve the same re-run ability.
- Adds new option "-e" to link all externals.
- This was the default before but I think in time we can make this
approach more intelligent and figure out which functions/subroutines
are used elsewhere.
- Processes F90 `include` to build dependencies
- Currently not tested with nested `include`s
Comments:
- As usual, the bizarre self-referencing in drifters.F90 needed some
special handling. The self-referencing is associated with code in CPP
block to generate local test programs. Now, we filter out circular
self-references rather than complain like make does. A large-separation
circular dependence will not be caught and lead to unpreditable
behavior. Such circular dependence is not something that makes sense to
support or allow.
- Using only native python featues, no packages, to avoid the "package
dependence" we are trying to avoid with makedep.
- This python version appears to be of orders of magnitude faster than
the bash version. Should have started here ...
Todo:
[ ] Improve some list/dictionary comprehensions. We use list
comprehensions quite a lot but a few "clunky" functions remain when
the first attempt at a comprehension failed. I never figrued out a
working dictionary comprehension.
[ ] Add a solution for the need to always link "externals".
- @marshallward suggested this fix to recursively follow F90 includes when building the list of dependencies. - Renamed the function from nested_h() to nested_inc() to be better describe function Co-authored-by: Marshall Ward <marshall.ward@gmail.com>
- The function includes_in_path() had many unused arguments and was left over from development. It turned out to be easy and clean to replace with a list comprehension.
- @marshallward reported that FMS2 has multiple versions of the same file int he search path. To avoid fatal errors, we are now allowing an ambiguous outcome and simple throwing out a warning that two files of the same name were encountered. Co-authored-by: Marshall Ward <marshall.ward@gmail.com>
Codecov Report
@@ Coverage Diff @@
## dev/gfdl #158 +/- ##
=========================================
Coverage 34.02% 34.02%
=========================================
Files 259 259
Lines 70204 70204
Branches 13013 13012 -1
=========================================
Hits 23884 23884
Misses 41822 41822
Partials 4498 4498 Continue to review full report at Codecov.
|
marshallward
approved these changes
Jul 7, 2022
Member
marshallward
left a comment
marshallward
left a comment
There was a problem hiding this comment.
This worked for my MacOS patch (with FMS1) and with FMS2 (on Linux), and solves the "bash 3.x" problem on default MacOS systems.
I didn't test MacOS with FMS2 but hopefully we can infer that it will work as well. (The issues with each respective target appear independent.)
Member
|
Gaea regression: https://gitlab.gfdl.noaa.gov/ogrp/MOM6/-/pipelines/16068 ✔️ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
to infer the link list
Changes relative to the bash version of makedep:
includeto build dependenciesincludesComments:
Todo:
[ ] Improve some list/dictionary comprehensions. We use list comprehensions quite a lot but a few "clunky" functions remain when the first attempt at a comprehension failed. I never figured out a working dictionary comprehension.
[ ] Add a solution for the need to always link "externals".