@-tags include everything after them

[lpw25]

I'm pretty sure I implemented this behaviour once upon a time and I did so to match ocamldoc's existing behaviour. But I wonder if it is a mistake to have @tags extend onwards until either the end of the comment or another @tag. I just found something like:

(** Some docs about things

     @param foo Stuff about a parameter
     
     More stuff about the whole function, not just the parameter. *)

and it seems a very reasonable thing to write, but it gets rendered very badly because everything after @param is considered part of the tag. (Also #329 is right, the markup for tags is bad).

Perhaps we should take a blank line as ending the tag? I think the actual tags we support would be fine only taking a single paragraph as input.

[jonludlam]

I think it's useful to see the existing bits of code that might be affected by this change. I did a very naive run of the docs ci on the latest version of every package with some very basic instrumentation to detect more than one nestable-block-elements in these tags. It's a bit tricky to get things exact, but it's a good representative set. The output is here: https://gist.github.com/jonludlam/e099db11eb65404e502948056450d54c

[jonludlam]

I went through those examples of the affected code, though not exhaustively -- there are a lot! I did find instances where the change would not be good, but I also found many more where the change would make the docs better. So I'm in favour of the suggestion of having a blank line end the tag. We might also want to consider other ways to end it, like a heading? I wouldn't suggest all block markup as there are some instances in the above of e.g. lists being used sensibly in @param and @return tags.

[dbuenzli]

I wouldn't suggest all block markup as there are some instances in the above of e.g. lists being used sensibly in @param and @return tags.

I think it should be all blocks but introduce a way for making multiple blocks if you saw it needed.

I can see two possibilities here:

1. Having a way to escape blank lines, I suggested @ here.
2. Having proper delimiters as a suggested by @EmileTrotignon here. But I think it would be better to have it as @ directives rather than overloading or inventing new brackets. E.g. @tag @begin-blocks … @end-blocks.

Solution 2. looks cleaner to me.

[panglesd]

I've tried to use a bit @jonludlam's experiment to understand more what would be fixed/broken depending on our choice of syntax.

I tried to open at least one link per package in Jon's list, and to classify them¹. The classification is two-dimensional: what is the second item in the @-block, and whether the author expected it to have split the block.

In case that is not clear, open this for examples on how I categorized the entries.

  @param foo The number of things

  The function runs in linear time

is in "New line", "Should break" ; while

  @param foo When foo is:
    - [true] then bli
    - [false] then blu

is in "List", "Should not break".

In case it needs disambiguation:

  @param foo When foo is:

    - [true] then bli
    - [false] then blu

is in "New line", "Should not break".

Newline

• Should break : 59 entries

  • uwt https://github.com/fdopen/uwt/blob/master/src/uwt_preemptive.mli#L35
  • ocaml-trace https://github.com/c-cube/ocaml-trace/blob/v0.8/src/core/trace_core.mli#L50
  • timere https://github.com/daypack-dev/timere/blob/timere-0.8.0/timere/timere.mli#L428
  • tezos https://gitlab.com/tezos/tezos/-/blob/v17.3/src/lib_stdlib/lwt_pipe.mli#L240
  • Syndic https://github.com/Cumulus/Syndic/blob/v1.6.1/lib/syndic_opml1.mli#L97
  • stdlib-random https://github.com/ocaml/stdlib-random/blob/1.2.0/random5o/random5o.mli#L239
  • sqlite3_utils https://github.com/c-cube/sqlite3_utils/blob/v0.5/src/sqlite3_utils.mli#L233
  • sqlite3-ocaml https://github.com/mmottl/sqlite3-ocaml/blob/5.1.0/src/sqlite3.mli#L77
  • SPOC https://github.com/mathiasbourgoin/SPOC/blob/20210823/Spoc/Mem.mli#L66
  • shell https://github.com/janestreet/shell/blob/v0.17.0/unix_extended/src/unix_extended.mli#L21
  • server-reason-react https://github.com/ml-in-barcelona/server-reason-react/blob/0.3.1/packages/Belt/src/Belt_SortArray.mli#L50
  • root1d https://github.com/Chris00/root1d/blob/0.5.1/src/Root1D.mli#L57
  • re2 https://github.com/janestreet/re2/blob/v0.17.0/src/regex_intf.ml#L139
  • qcheck https://github.com/c-cube/qcheck/blob/v0.22/src/ounit/QCheck_ounit.mli#L52
  • pcre2 https://github.com/camlp5/pcre2-ocaml/blob/c6e88b9aa567ad444ef999b9b101ae5041f4be0b/src/pcre2.mli#L264
  • optimization1d https://github.com/Chris00/optimization1d/blob/0.6.1/src/min1D.mli#L25
  • ocaml-odepack https://github.com/Chris00/ocaml-odepack/blob/0.7.1/src/odepack.mli#L102
  • tezos https://gitlab.com/tezos/tezos/-/blob/octez-v20.1/src/lib_stdlib_unix/key_value_store.mli#L78
  • ocamlgraph https://github.com/backtracking/ocamlgraph/blob/2.1.0/src/coloring.mli#L60
  • ocamlformat https://github.com/ocaml-ppx/ocamlformat/blob/support.0.4/format_.mli#L759
  • stdslib https://github.com/ocaml/ocaml/blob/5.2.0/otherlibs/unix/unixLabels.mli#L160
  • moonpool https://github.com/c-cube/moonpool/blob/v0.7/src/forkjoin/moonpool_forkjoin.mli#L31
  • mirage-crypto https://github.com/mirage/mirage-crypto/blob/v1.1.0/src/mirage_crypto.mli#L200
  • mesh https://github.com/Chris00/mesh/blob/0.9.5/triangle/mesh_triangle.mli#L221
  • lwt_log https://github.com/ocsigen/lwt_log/blob/1.1.2/src/core/lwt_log_core.mli#L95
  • lwt_preemptive https://github.com/ocsigen/lwt/blob/5.7.0/src/unix/lwt_preemptive.mli#L53
  • rems-project https://github.com/rems-project/sail/blob/0.18/src/lib/target.mli#L116
  • ocaml-junit https://github.com/Khady/ocaml-junit/blob/2.0.2/junit/junit_xml.mli#L208
  • irmin https://github.com/mirage/irmin/blob/3.7.2/src/irmin/conf.mli#L78
  • ocaml-interval https://github.com/Chris00/ocaml-interval/blob/1.6/src-intel/fpu_rename_all.mli#L24
  • bap https://github.com/BinaryAnalysisPlatform/bap/blob/v2.5.0/lib/graphlib/graphlib.mli#L860
  • ocaml-gammu https://github.com/Chris00/ocaml-gammu/blob/0.9.4/src/gammu.mli#L333
  • ocaml-freetds https://github.com/kennknowles/ocaml-freetds/blob/0.7/src/dblib.mli#L101
  • fftw-ocaml https://github.com/Chris00/fftw-ocaml/blob/0.8.5/src/fftw3.mli#L439
  • eqaf https://github.com/mirage/eqaf/blob/v0.9/lib/eqaf.mli#L96
  • eio https://github.com/ocaml-multicore/eio/blob/v1.1/lib_eio/net.mli#L280
  • ocaml-dropbox https://github.com/Chris00/ocaml-dropbox/blob/0.2/src/dropbox.mli#L812
  • dose3 https://gitlab.com/irill/dose3/-/blob/7.0.0/src/algo/depsolver_int.mli#L178
  • ocaml-dockerfile https://github.com/ocurrent/ocaml-dockerfile/blob/v8.2.2/src/dockerfile.mli#L250
  • ocaml-docker https://github.com/Chris00/ocaml-docker/blob/0.2.2/src/docker.mli#L127
  • cudf https://gitlab.com/irill/cudf/-/blob/v0.10/lib/cudf.mli#L258
  • ocaml-ctypes https://github.com/yallop/ocaml-ctypes/blob/0.23.0/src/ctypes-foreign/foreign.mli#L77
  • ocaml-cow https://github.com/mirage/ocaml-cow/blob/v2.5.0/src/html.mli#L195
  • core_unix https://github.com/janestreet/core_unix/blob/v0.17.0/filename_unix/src/filename_unix.mli#L43
  • core_kernel https://github.com/janestreet/core_kernel/blob/v0.9.1/src/union_find.mli#L9
  • core https://github.com/janestreet/core/blob/v0.17.1/filename_base/src/filename_base.mli#L14
  • ocaml-containers https://github.com/c-cube/ocaml-containers/blob/v3.6/src/unix/CCUnix.mli#L194
  • ocaml-cohttp https://github.com/mirage/ocaml-cohttp/blob/v6.0.0_beta2/cohttp-lwt/src/s.ml#L85
  • calculon https://github.com/c-cube/calculon/blob/v0.8/src/core/Run_main.mli#L10
  • ocaml-cairo https://github.com/Chris00/ocaml-cairo/blob/0.6.4/src/cairo.mli#L810
  • cactus https://github.com/mirage/cactus/blob/1.0.0/src/cache_intf.ml#L39
  • ocaml-bwrap https://github.com/Chris00/ocaml-bwrap/blob/0.1/src/bwrap.mli#L56
  • bogue https://github.com/sanette/bogue/blob/20240928/lib/bogue.mli#L435
  • cryptography https://gitlab.com/nomadic-labs/cryptography/ocaml-bls12-381/-/blob/6.1.0/src/bls12_381.mli#L268
  • batteries-included https://github.com/ocaml-batteries-team/batteries-included/blob/v3.8.0/src/batSubstring.mli#L51
  • bap https://github.com/BinaryAnalysisPlatform/bap/blob/v2.5.0/lib/bap_primus/bap_primus.mli#L2325
  • bap https://github.com/BinaryAnalysisPlatform/bap/blob/v2.3.0/lib/bap_main/bap_main_event.mli#L125
  • atd https://github.com/ahrefs/atd/blob/2.9.1/atd/src/expand.mli#L27
  • amqp-client https://github.com/andersfugmann/amqp-client/blob/2.3.0/async/src/channel.mli#L69

• Should not break: 9 entries

  • tezos https://gitlab.com/tezos/tezos/-/blob/v17.3/src/proto_alpha/lib_protocol/validate.mli#L258
  • pcre2-ocaml https://github.com/camlp5/pcre2-ocaml/blob/7.5.2/src/pcre2.mli#L425
  • ocaml-opentelemetry https://github.com/imandra-ai/ocaml-opentelemetry/blob/v0.10/src/client-ocurl/config.mli#L55
  • miou https://github.com/robur-coop/miou/blob/v0.3.0/lib/miou.mli#L701
  • ocaml-lmdb https://github.com/Drup/ocaml-lmdb/blob/1.0/src/lmdb.mli#L201
  • index https://github.com/mirage/index/blob/1.6.2/src/index_intf.ml#L58
  • data-encoding https://gitlab.com/nomadic-labs/data-encoding/-/blob/v1.0.1/src/v1.mli#L2445
  • ocaml-caqti https://github.com/paurkedal/ocaml-caqti/blob/v2.1.2/caqti/lib/caqti_request.mli#L50
  • asak https://github.com/nobrakal/asak/blob/0.5/src/partition.mli#L23

Lists

• Should break: Nothing!

• Should not break: 9 entries

  • ocaml-trace https://github.com/c-cube/ocaml-trace/blob/v0.8/src/fuchsia/trace_fuchsia.mli#L30 (anyway markup is wrong)
  • tezt https://gitlab.com/nomadic-labs/tezt/-/blob/4.1.0/lib_json/JSON.mli#L223
  • slap https://github.com/akabe/slap/blob/v4.1.1/lib/slap_mat.cppo.mli#L320
  • slap https://github.com/akabe/slap/blob/v4.1.1/lib/slap_SD_la.mli#L995
  • resto https://gitlab.com/nomadic-labs/resto/-/blob/v1.2/src/resto_directory.mli#L233
  • ppxlib https://github.com/ocaml-ppx/ppxlib/blob/ac7fcfc88d574609b62cc0a38e0de59d03cc96de/src/pp_ast.mli#L53
  • mlpost https://github.com/backtracking/mlpost/blob/0.9/contrib/dot/mlpost_dot.mli#L25
  • capnp-rpc https://github.com/mirage/capnp-rpc/blob/v1.2.3/unix/capnp_rpc_unix.mli#L108
  • ocaml-x509 https://github.com/mirleft/ocaml-x509/blob/v0.11.2/lib/x509.mli#L171

Code blocks

• Should break: Nothing!

• Should not break: 1 entry

  • tezos https://gitlab.com/tezos/tezos/-/blob/v17.3/src/proto_alpha/lib_delegate/test/tenderbrute/lib/tenderbrute.mli#L54

There are also several results that were included in @jonludlam's list due to an incorrect use of odoc's syntax. I'll open separate issues for those that are worth it.

Another thing I spot is that authors often use indentation with tags, when they mean to put it all in the tag block. But odoc can't rely on that...

---

Let's enumerate the possibilities we have (I'm not speaking about the way we would define a delimiter for @-blocks yet).

1. Say that any new block item ends the @-block
   It is good for the 59 "New line"/"Should break" entries.
   This has the advantage of being the least surprising.
2. Say that a blank line ends the @-block.
   It is good for the 59 "New line"/"Should break" entries and the 9 entries in "List"/"Should not break".
   Not too surprising, but a little bit more.
3. Say that only a new paragraph ends an @-block (of course, unless delimited by the to be decided syntax).
   It is good for the 59 "New line"/"Should break" entries and the 9 entries in "List"/"Should not break", and about half of the 9 entries in "New line"/"Should not break" (since those are often not followed by a new paragraph but by a new list / code block).
   The most backward compatible, but quite surprising rule...

I would prefer the solution 1, even though it is the least backward compatible. But I would be fine with solution 2.

Footnotes

1. By hand, that was very boring! ↩

[jonludlam]

Thanks for going through the list, that was a lot of work!

I'm fine with either options 1 or 2, though I lean ever so slightly in favour of 2 rather than 1.

[dbuenzli]

I'd be more in favour of 1. Unless I'm mistaken in the current language:

{ul …}
{| …|}

and

{ul …}

{|…|}

are strictly equivalent. If we adopt 2. that would no longer be the case in @-tag contexts. I don't find this to be a desirable property.

[panglesd]

I have investigated, and finally found something that add grist to my mill!

Ocamlformat (with the --parse-docstring option, which is now on by default) often makes a difference between blocks separated by a new line, and those that are not separated by a new line, in the sense that it preserves it:

Paragraph directly followed by a code block
{[ blibli ]}

Paragraph with a line before the code block

{[ bloblo ]}

Becomes

Paragraph directly followed by a code block
{[
  blibli
]}

Paragraph with a line before the code block

{[
  bloblo
]}

However, it assumes that it is safe to add newlines between lists when needed:

{ul
  {- First item}
  {- Second item}
  {li can also be used}}
{ol
  {- First numbered item}
  {- Second numbered item}
  {li can also be used}}

is formatted as

- First item
- Second item
- can also be used

+ First numbered item
+ Second numbered item
+ can also be used

So, ocamlformat assumed it is safe to add a newline (and it was). Going with option 2., ocamlformat would change the meaning of the docstring when formatting.

This is not a discriminating reason to chose 2. over 1., but that's comforting me in my preference for 1.

[panglesd]

We discussed this during the dev meeting and decided to go for 1:

Any new block ends the @ block!

I'll start implementing this soon.

[panglesd]

While implementing this, I was surprised by how the implicit ending of list light syntax work.

So, blank lines, section headings, closing bracket and tags end a list item:

- A list item
{1 Title not part of the list item}

However, other blocks (eg code blocks, media blocks, ...) do not end the list item:

- A list item
{[A code block, part of the list item]}

A quick search shows that this is indeed used in the wild.

If we implement the tag as planned (takes a single block, without constraint), this creates some strange interaction.

For instance:

@tag
  This is part of the tag
  {[this code block is not]}

@tag
  - This is part of the tag
  {[so is this code block, as part of the first list item]}

This inconsistency is problematic. It seems to make sense to have the same rule for implicit ending of both tags and light syntax list.