Add online help system

[ln2max]

Add an online help system as per #73.
Todo:

• Add man page
• Make documentation conform to man page format or create a separate manpage file that automagically updates from the main documentation
• Add man page installation support for non-debian distributions

Very much a WIP right now. Command-specific help may not be worth doing (and may add unnecessary overhead in terms of keeping documentation updated) if the manpage works well.

[ln2max]

Open questions right now:

1. Is there a less hacky (and more cross-platform-compatible) way to install manpages generated by sphinx? (vs the current manpage install script -- caution, link is to a specific commit)
2. The documentation currently contains a lot of unnecessary information (that's automatically generated from the docstrings in the python files). do we need to keep it the way it is (and have a separate manpage) or can we create a succinct manpage-style central document that contains all necessary information?
3. Do we really need an interactive/online help system (where users can do e.g sudo noma help foo bar) or is a manpage enough?

[AnotherDroog]

@ln2max Thank you for the thought you put into this PR. I'm a big fan of quality docs in the tradition of OpenBSD/FreeBSD and you've addressed particularly sore points.

I've taken care of (1) by adding compatibility for Alpine. Since this covers all our target OS I'd suggest leaving it at that.

(2) You're right that the documentation contains unnecessary information and this extends to the underlying code. Since the bitcoind PR is nearing completion, we are about to see what can be removed from bitcoind.py.

Similarly, we can remove non-critical parts that were added for requirements that are no longer important to the project and thereby bring better focus to the documentation.

I am wondering if private functions will still be autogenerated, this might be another option to pursue.

Concerning the manpage and default help screen, I would suggest keeping those in sync thru Sphinx. Fortunately docopt allows everything outside the usage: and options: to be free-form that we can populate with a well crafted text.

(3) @meeDamian brought up a valid point in #73 and the lack of individual help for each command has been annoying me from the beginning. The work you've done here to print out the docstring is exactly what I was hoping for. A positive side-effect is that we've got an incentive to write understandable help text right next to the code. Bravo!

The only issue I've run into is this:

vagrant@debian-vagrant:/media/noma$ sudo noma help start
Traceback (most recent call last):
  File "/usr/local/bin/noma", line 11, in <module>
    load_entry_point('noma', 'console_scripts', 'noma')()
  File "/media/noma/noma/noma.py", line 120, in main
    node_fn(args)
  File "/media/noma/noma/noma.py", line 107, in node_fn
    _help(args["COMMAND"])
  File "/media/noma/noma/noma.py", line 47, in _help
    help(getattr(node, cmd))
TypeError: getattr(): attribute name must be string

[ln2max]

Thanks for the reply @AnotherDroog ... one thing I still haven't figured out:

right now if you do sudo noma help then it shows a long list of options, many of which are not implemented in the CLI. This is because help is just using the internal Python help() system, which shows all public functions and methods. Some of those functions may be intended for internal use within that module or for use by other modules, they just aren't for users to interact with directly.

In order to get rid of them from the help display, we have 2 options:

1. We can make all non-user-facing functions in the respective modules private/semi-private by prepending an underscore to their names (_my_function()). This will cause them to disappear from the automatically generated help display. BUT, it will also indicate they should not be called from outside the module. (It is still technically possible to call them from outside the module, this is necessary for running tests, but it's really bad form to do so.)
2. We can re-engineer the interactive help system to use manually-composed documentation from sphinx (where that manually-composed documentation only mentions the functionality we want it to, but automatically pulls in docstrings for those functions)... There is probably a way to do this, but I haven't figured it out yet.

[ln2max]

Sphinx is messing up the docstring formatting for the docopts formatted docstring. To fix this, the Sphinx docs recommend the napoleon extension but the relevant documentation page for it is not loading for me:

For this to work, the docstrings must of course be written in correct reStructuredText. You can then use all of the usual Sphinx markup in the docstrings, and it will end up correctly in the documentation. Together with hand-written documentation, this technique eases the pain of having to maintain two locations for documentation, while at the same time avoiding auto-generated-looking pure API documentation.

If you prefer NumPy or Google style docstrings over reStructuredText, you can also enable the napoleon extension. napoleon is a preprocessor that converts your docstrings to correct reStructuredText before autodoc processes them.
https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html

[AnotherDroog]

Thank you for your continued efforts here,

noma help <cmd> works beautifully for me now!

I've tested adding other sections to the main docstring in noma.py, these get rendered properly in the manpage and noma -h. However, the main docstring is missing from noma help as it defaults to help(node) Would you agree that noma -h and noma help should be synonymous?

Apart from that the noma help page looks a bit awkward with :param: and :return: strings littered about. While those fields are missing entirely from man noma.

Now that you've added the napoleon extension, would you suggest changing docstrings to NumPy or Google style everywhere? I can open the napoleon documentation, I'm just not sure what the output is supposed to look like.

[ln2max]

On Wed, Oct 30, 2019 at 02:25:39AM -0700, Another Droog wrote: I've tested adding other sections to the main docstring in `noma.py`, these get rendered properly in the manpage and `noma -h`. However, the main docstring is missing from `noma help` as it defaults to `help(node)` Would you agree that `noma -h` and `noma help` should be synonymous?

Yes, that makes sense, or at least `noma help` should be a superset of `noma -h`.

Apart from that the `noma help` page looks a bit awkward with `:param:` and `:return:` strings littered about. While those fields are missing entirely from `man noma`.

That's odd, `:param` etc should be properly parsed by the Python docstring parser.

Now that you've added the napoleon extension, would you suggest changing docstrings to NumPy or Google style everywhere? I can open the napoleon documentation, I'm just not sure what the output is supposed to look like.

I'm not sure we should be changing anything, except perhaps for removing the `napoleon` plugin! I had been hoping that `napoleon` would just be able to render the docstrings as written (in `docopts` format). The problem here is that we *can't* reformat the docstring which really matters -- the main one describing the command line options -- because `docopts` expects it to be in that format in order to generate the command line parser! So what we really need is a way to get `sphinx` to just render the text as-is, because it's perfectly fine the way it is, instead of parsing the docstring as reStructuredText.