specify valid placement for doc comments and runnableExamples

[timotheecour]

• the manual https://nim-lang.github.io/Nim/manual.html is terse about valid placement for doc comments:

Documentation comments are comments that start with two ##. Documentation comments are tokens; they are only allowed at certain places in the input file as they belong to the syntax tree!

and https://nim-lang.github.io/Nim/docgen.html is also rather terse.

• docgen specifies we need ## \:

var numValues: int ## \
  ## `numValues` stores the number of values

which seems to suggest that it won't work without ## \

and indeed that's what stdlib uses:

type
  ForLoopStmt2* {.compilerproc.} = object ## \
    ## A special type that marks a macro as a `for-loop macro`:idx:.
    ## See `"For Loop Macro" <manual.html#macros-for-loop-macro>`_.

but not everywhere:

type
  NimNodeObj* = object
  NimNode* {.magic: "PNimrodNode".} = ref NimNodeObj
    ## Represents a Nim AST node. Macros operate on this type.

however it does seem to work without it, in the sense that nim doc generates correctly without ## \

Furthermore, ## \ is actually incorrectly handled in other places, eg:

type Person* = object
  ## This type contains a description of a person
  name*: string ## some name
  age*: int
  age2*: int
    ## some age2
    ## rest
  age3*: int
    ## some age3
  age4*: int ## \
    ## some age4
  age5*: int ## \
    ## some age5
  ageHidden: int ## \
    ## this is not exported

renders as

=> the ## \ should not be shown

• the manual doesn't mention that non-exported fields of objects still get docgen'd (IMO that's fine that they are docgen'd, it just needs to be specified)

• for routines, stdlib uses this:

proc unsafeAddr*[T](x: T): ptr T {.magic: "Addr", noSideEffect.} =
  ## Builtin `addr` operator for taking the address of a memory
  discard

template a6*: untyped =
  ## foo
  1

instead of this:

proc unsafeAddr*[T](x: T): ptr T {.magic: "Addr", noSideEffect.} = discard
  ## Builtin `addr` operator for taking the address of a memory
template a6*: untyped = 1
  ## ok (docgen correctly)

even though the 2nd form works (docgen's correctly) and is shorter and also is consistent with the way it works for other symbols, eg:

const a1* = 1
  ## ok
var
  b1* = 1 ## ok
  b2* = "foo" ## ok
  b3*: float ## ok
  b4*: float = 1
    ## Some long comment.
    ## On multiple lines.
  b5*: float = 1
    ## Another long comment.
    ## On multiple lines.

type A4* = int
  ## ok

(this came up here nim-lang/Nim#14474 (comment))

(controversial) 1-liner routines followed by doc comment + runnableExamples

this never worked:

proc square*(x: int): int = x*x
  ## squares `x`
  runnableExamples: assert square(3) == 9

giving a Error: invalid indentation parser error;
and this:

proc square*(x: int): int = x*x
  ## squares `x`
runnableExamples: assert square(3) == 9

works but cuts the runnableExamples from that symbol, and docgen's it at the top of the module.
Instead this works:

proc square*(x: int): int =
  ## squares `x`
  runnableExamples: assert square(3) == 9
  x*x

however, it makes a common case (1-liners) just a little bit less convenient, eg when adding a runnableExamples to an existing 1-liner, this forces you to move the 1-liner body below the runnableExamples.
It's also somewhat inconsistent with the fact that this works:

proc square*(x: int): int = x*x
  ## squares `x`

• what should be behavior of doc comments + runnableExamples that occur in the wrong place eg:

proc square4*(x: int): int =
  ## comment1
  runnableExamples: discard 1
  ## comment2
  runnableExamples: discard 2

  var y = x
  ## comment3
  y = y * y ## comment4
  runnableExamples: discard 3
  return y ## comment5

as of nim-lang/Nim#14439, this will docgen comment1, the 1st runnable, coment2, the 2nd runnable, and everything else will not be docgen'd as it's considered part of implementation, not of declaration; in particular discard 3 will neither be docgen'd nor run (whereas as of 1.2, this would docgen comment1, and all runnableExamples; skipping comment2).

And if we agree with semantics as of #14439, IMO we should give an error for a runnableExample that's misplaced like discard 3 since user may think it's run and docgen'd even though it's not. Whereas for a misplaced doc comment, IMO a warning is more appropriate (which can always be turned into error via warningAsError:xxx)

proposal

to summarise:

• remove all mentions in docs of ## \
• explictly allow doc comments after 1 liner (which already work but arent' mentioned in docs nor used in stdlib):

template a6*: untyped = 1
  ## ok (docgen correctly)

• given errors for misplaced runnableExamples and warning for misplaced doc comments
• I'm happy to drop (or defer) the controversial 1-liner routines followed by doc comment + runnableExamples

proc square*(x: int): int = x*x
  ## squares `x`
  runnableExamples: assert square(3) == 9

• give warning (not error) when nim doc can't recognize what's inside a doc comment (refs nim doc .. compiler crash (regression 0.19.6 => 1.0) Nim#14474 (comment))

links

see also this regression timotheecour/Nim#784