Extracting docs from source code

[ladyleet]

Take TS files and convert them into ASTs.
Traverse ASTs to extract out documentation.

[cartant]

Do you have a specific process in mind for extracting the ESDoc-based documentation from the TS source files? Is this going to be an ongoing part of generating the documentation? Or will it be a one-off job?

Internally, ESDoc uses Babel's babylon to parse the source. And within the generated AST, the ESDoc exists in the leadingComment nodes. To parse the ESDoc content from within the comment nodes, ESDoc includes a CommentParser.

The ESDoc comments are preserved in RxJS's JavaScript output, so it's pretty easy to go straight to the distributed JavaScript files and parse those. If you really wanted to, it would also be possible to transpile the TypeScript to JavaScript (on a per-file basis with no syntax or semantic checking) and pass the generated JavaScript to babylon, etc.

[ladyleet]

@cartant I know @jayphelps and @benlesh have been working on this. Guys, have an update?

[ladyleet]

@cartant thanks so much for expressing interest! @jayphelps is working on some of the harder challenges and will comment here for issues for others to take it to the finish line.

[jayphelps]

I'm working on a solution that uses tsc to extract the type signatures from the code itself along with any associated doc comments. It hasn't been as easy as I thought it would be (surprise surprise). Biggest issues have been tsc's lack of support for {@link} and the type aliases not printing as the most derived alias. I think I have a workaround for the {@link} issue; it ain't pretty, but I think it'll work fine.

Because we don't want to generate entries for the entire codebase like we do right now in our esdoc stuff, I'm working off the assumption that we will add a @category <name> tag to things we want included, and then my code will group anything it finds by category. e.g. @category operators, @category observables. Lmk if anyone thinks of a reason this won't work well.

Since there are two different kinds of operators, prototype vs lettable, it's unclear which one (or both?) we would document. Ben was thinking we'd only document one of them.

[manekinekko]

would you guys be interested in using this project?

[jayphelps]

@manekinekko I think we'd use anything that works well for our use case; all of the off the shelf stuff we've looked at so far wasn't flexible enough. The biggest thing is to have operators clearly grouped separate from any other docs that might get generated. e.g. we don't want to give equal placement to random classes/functions and operators. This is one of the reasons our current website isn't great. So having sections like "operators" and "static operators" or whatever.

I've gone through what seems like all of the documentation on it, and it's not clear if this is possible? Perhaps it is if we write our own theme or modify an existing one? It seems very much specific to documenting an angular app, rather than a library?

[cartant]

@jayphelps

It hasn't been as easy as I thought it would be (surprise surprise).

The Programmers’ Credo:

We do these things not because they are easy, but because we thought they were going to be easy.

[niklas-wortmann]

Just a question. Is this still the way to go? I'm not quite sure but I think the current manual approach got some advantages too. For example it is much easier to versionize the documentation or add some improvements to the documentation for special release versions of rxjs. Such advantages keep me thinking about the approach discussed in this issue. Of course it is much more work to do but I think it could also improve the quality of the documentation. What do you think about it? Furthermore I would be interested in the status of this issue? I don't want to frustrate someone by discussing about already done work :)

[ladyleet]

@jwo719 yes - we still will need to do this and half both - manual + generated in the new docs.