Using ford with an existing project

[runburg]

I have inherited a very large fortran project and for my own sanity I would like to have some good documentation. I think ford is the right answer.

I am trying to slowly migrate this project over to but I'm to get some advice for how to do this. If I try and run ford on the whole codebase, I get a bunch of parsing errors 'NoneType' object is not subscriptable.

I also get Unexpected SUBROUTINE messages which end up throwing a NotImplementedError() which results in

  File "C:\Users\John.Runburg\AppData\Local\Programs\Python\Python311\Lib\site-packages\ford\fortran_project.py", line 121, in __init__
    print(f"Warning: Error parsing {relative_path}.\n\t{e.args[0]}")
                                                        ~~~~~~^^^
IndexError: tuple index out of range

and Unexpected variable messages which result in the same as above.

If I throw out only focus on the source files that do not result in these parsing issues, ford makes it to Correlating information from different parts of your project... and then throws the error

 File "C:\Users\John.Runburg\AppData\Local\Programs\Python\Python311\Lib\site-packages\ford\fortran_project.py", line 234, in <dictcomp>
    module: set(module.deplist)
                ^^^^^^^^^^^^^^
AttributeError: 'FortranSubmodule' object has no attribute 'deplist'

The codebase is written almost entirely in modern fortran. But are there certain features that ford will get hung up on or that commonly cause these kind of problems.

Thanks!

[runburg]

Okay so I managed to get ford running on a decent chunk of the codebase. It seems like it has trouble parsing '$' especially when it is in the name of a function, subroutine, derived type, etc. Is this a common problem and is there a way around it?

[ZedThree]

Hi @runburg , thanks for the bug report. $ is not in the standard Fortran character set, so I guess either the project relies on a compiler extension, or it uses a custom preprocessor. I'm afraid you get lots of non-meaningful errors because our parser doesn't catch it early enough.

If it's the latter, that's easily dealt with -- see the preprocessing options.

If it's a compiler extension, on the other hand, we'd probably have to look at handling it in the FORD parser. Maybe through an option like "extra_identifier_characters" or something.

A third option is to replace $ in identifiers with another character -- this would make the source more portable, but of course, you may not wish to modify a large legacy project.

[runburg]

Hmm...I'm on Windows using Intel® Fortran Compiler Classic 2021.8.0 with Fortran Preprocessor 21.0-1007 right out of the box (from the intel oneAPI toolset). So I don't think I'm using any extensions or custom preprocessor but maybe there is something going on behind the scenes that I don't know about (I am still new to Fortran and this project).

The use of $ is so prevalent that I'm wary of trying to modify things at this point.

FWIW, it seems that intel at least supports using $ in names..

Is there any other information I can provide to try and get a clearer answer on what's happening?

Thanks for your help!

[ZedThree]

The bit in green means it's a language extension, but it looks like it's enabled by default for Intel.

A quick survey of other compiler manuals shows that gfortran and PGI accept it with a flag, IBM doesn't accept it at all, NAG does, Cray does but not at the start of names.

I'll have a look at adding it as an option to FORD, but I would suggest it's a good idea to try and replace it with another character.

[runburg]

Oh whoops I see now. Thanks for the clarification.

Long-term, I agree that I should change things. But it is used thousands of times in this project and it's not on the top of my to do list right now. In the meantime, FORD is working great otherwise!

[jmbcastro]

Hello @runburg, I am in a similar situation to yours. I also get numerous Unexpected SUBROUTINE and Unexpected variable errors when parsing the source code. Did you find a solution to this problem?

[ZedThree]

@jmbcastro Are you able to post a short snippet that reproduces the issue? Or, if your source code is public, I can have a look myself.

[jmbcastro]

Hi @ZedThree, thank you for your response. Unfortunately the code is not public. I was digging around and found out that the errors were due to situations like these in several points of the source code:

First

  INTERFACE
    SUBROUTINE OpenConnection_C            &
               [ALIAS:'OpenConnection_C']  &
             (sServerName,                    &
              iPort,                          &
              sFamily,                        &
              iDomainId,                      &
              iEpoch,                       &
              sValue)
             
             ...

    END SUBROUTINE

    SUBROUTINE PrintInfo_C (ivar1)  &
          BIND(C, NAME = 'PrintInfo_C')

        IMPORT :: C_INT
        INTEGER(KIND = C_INT), VALUE :: ivar

    END SUBROUTINE

  END INTERFACE

Those subroutines are elsewhere coded in C. FORD does not seem to like Interfaces with subroutines inside.

Second

CHARACTER(LEN=250)FUNCTION FirstWord( sString ) RESULT( sRes )
   ...
END FUNCTION FirstWord

INTEGER FUNCTION GetLen( sFormat ) RESULT( iRes )
   ...
END FUNCTION GetLen

FORD cannot understand that CHARACTER(LEN=250) and INTEGER refer to the type of the output value.

Third

      SUBROUTINE sub(a,  !.. Input: bla bla bla 
     >                      b,  !.. Input: bla bla bla 
     >               c,d,  !.. Input: bla bla bla 
     >                 e,f,g,  !.. Input: bla bla bla 
     >          h,i,j,k,  !.. Input: bla bla bla 
     >                    o)  !.. OUTPUT: bla bla bla 
    
       ...
    
      RETURN
      END

This is in F77. FORD does not like the inline comments explaining what the arguments are.

Fourth

	real function f1(a,b,c)
        ...
      	if(c.lt.0.0) c=0.0
	    f1=a+b+c
	    return
	    end

        subroutine f2 (a,b,c)
        ...
	    return
	    end

This is some poorly formatted F77 code, where we have tabs before the return and end statements. It seems FORD cannot handle that.

I hope this helps you to make FORD more robust.

Cheers

[ZedThree]

Thanks for the examples @jmbcastro

1. This is non-standard Fortran, specifically the [ALIAS:'OpenConnection_C']. Everything else should work though. I'm afraid that we really only intend to support standard Fortran.
2. Please could you clarify? These examples seem to work fine for me, although I needed some whitespace before the function keyword. What are you expecting to see, and what are you seeing instead?
3. Ford expects the argument docstrings to be on the variable declarations, rather than in the argument list. The F77 files are rewritten internally to parse them as free-form Fortran, and I suspect this is just incompatible with Ford documentation. Unless you need these comments for some other tool, I suggest it's easiest to rewrite these as standard Ford docstrings.
4. Strictly speaking tabs are not valid Fortran, but we should probably accept this! I'm surprised this fails, so I'll try and look into this.

[jmbcastro]

Hi @ZedThree,

1. I can confirm it, if I remove the [ALIAS:'OpenConnection_C'] it does work.
2. Indeed the problem is just the lack of the white space in CHARACTER(LEN=250)FUNCTION. It causes a chain reaction of seemingly unrelated errors.
3. OK, but if standard Fortran allows these inline comments, I guess FORD should too, perhaps by just ignoring them.
4. Sounds good.

Cheers

[ZedThree]

I'll have a look at fixing 2 and 3