Merge pull request #96 from swcurran/docs-reorg

Updates to docs site generator after aca-py docs reorganization

docs/UpdateRTD.md

@@ -0,0 +1,66 @@
+# Managing Aries Cloud Agent Python `Read The Docs` Documentation
+
+This document describes how to maintain the `Read The Docs` documentation that
+is generated from the ACA-Py code base. As the structure of the ACA-Py code
+evolves, the RTD files need to be regenerated and possibly updated, as described here.
+
+## Generating ACA-Py Read The Docs (RTD) documentation
+
+### Before you start
+
+To test generate and view the RTD documentation locally, you must install [Sphinx](https://www.sphinx-doc.org/en/master/) and the
+[Sphinx RTD theme](https://pypi.org/project/sphinx-rtd-theme/). Follow the instructions on the respective pages to install
+and verify the installation on your system.
+
+### Generate Module Files
+
+To rebuild the project and settings from scratch (you'll need to move the generated index file up a level):
+
+`rm -rf generated; sphinx-apidoc -f -M -o  ./generated ../aries_cloudagent/ $(find ../aries_cloudagent/ -name '*tests*')`
+
+Note that the `find` command that is used to exclude any of the `test` python files from the RTD documentation.
+
+Check the  `git status` in your repo to see if the generator updates, adds or removes any existing RTD modules.
+
+### Reviewing the files locally
+
+To auto-generate the module documentation locally run:
+
+``` bash
+sphinx-build -b html -a -E -c ./ ./ ./_build
+```
+
+Once generated, go into the `_build` folder and open `index.html` in a browser. Note that the `_build` is
+`.gitignore`'d and so will not be part of a git push.
+
+### Look for Errors
+
+This is the hard part; looking for errors in docstrings added by devs. Some tips:
+
+- missing imports (`No module named 'async_timeout'`) can be solved by adding the module to the
+list of `autodoc_mock_imports` in the `conf.py` file in the ACA-Py `docs` folder.
+- Ignore any errors in .md files
+- Ignore the warnings about including `docs/README.md`
+- Ignore an dist-package errors
+
+Other than that, please investigate and fix things that you find. If there are fixes, it's usually
+to adhere to the rules around processing docstrings, and especially around JSON samples.
+
+### Checking for missing modules
+
+The file `index.rst` in the ACA-Py `docs` folder drive the RTD generation. It picks up all the modules
+in the source code, starting from the root `../aries_cloudagent` folder. However, some modules
+are not picked up automatically from the root and have to be manually added to `index.rst`. To do that:
+
+- Get a list of all generated modules by running: `ls generated | grep "aries_cloudagent.[a-z]*.rst"`
+- Compare that list with the modules listed in the "Subpackages" section of the left side menu in your browser, including any listed below the "Submodules".
+
+If any are missing, you likely need to add them to the `index.rst` file in the `toctree` section of the file.
+You will see there are already several instances of that, notably "connections" and "protocols".
+
+### Updating the [readthedocs.org](https://readthedocs.org) site
+
+The RTD documentation is **not** currently auto-generated, so a manual re-generation of the documentation
+is still required.
+
+> TODO: Automate this when new tags are applied to the repository.

docs/assets/README.md

@@ -0,0 +1,16 @@
+# Assets Folder for Documentation
+
+Put any assets (images, source for images, videos, etc.) in this folder to be referenced in the various documents for this repo.
+
+## Plantuml Source and Images
+
+Plantuml diagrams are stored in this folder in source form in files ending in `.puml` and are generated manually using the `./genPlantuml` script. The script uses a docker image from docker-hub and can be run without downloading any dependencies.
+
+If you don't want to use the script, download plantuml and a command line utility and use that for the plantuml generation. I preferred not having any dependencies used (other than docker) and couldn't find
+a nice way to run plantuml headless from a command line.
+
+## To Do
+
+It would be better to use a local `Dockerfile` vs. one found on Docker Hub. The one I did find was simple and straight forward.
+
+I couldn't tell if the svg generation was working so just went with png. Not sure which would be better.

docs/assets/deploymentModel-agent.puml

@@ -0,0 +1,78 @@
+skinparam defaultTextAlignment center
+
+skinparam titleBorderRoundCorner 15
+skinparam titleBorderThickness 2
+skinparam titleBorderColor red
+skinparam titleBackgroundColor White
+
+skinparam monochrome true
+
+skinparam componentStyle uml2
+
+cloud "The Internet\n" {
+
+    component "Distributed\nLedger" as DL
+    component "Other\nAgents" as others
+}
+
+package "Aries Cloud Agent - Python" as vx {
+    package Core as core {
+        component "Transport Plugins" as tp
+        package  "Conductor" as cond {
+            component "Msg Receiver\nGet Thread State" as mr
+            component "Msg Sender\nPut Thread State" as ms
+        }
+        component "Dispatcher" as disp
+        component "Handler API" as hapi
+    }
+    package "Protocols" as prot {
+        component "Protocol 1\nProtocol 2\n.\n.\nProtocol n" as protos
+    }
+    component "Controller\nREST API" as rest
+    package "Handler Plugins" as inthand {
+        component "VDR Manager\nStorage Manager\n.\n.\n." as intmgrs
+    }
+    package "Storage Implementation" as is {
+        database "Secure Storage" as storage
+    }
+    intmgrs --> storage
+
+}
+
+/'
+package "Enterprise Services" {
+    package "Agent Controller" as per {
+        component "HTTP Handler" as http
+        component "Agent\nController\nConfiguration" as config {
+            database "Conf" as confdb
+        }
+        component "Initiator" as init
+        component "Responder" as resp
+        '' database "Configuration\nFiles" as configdb
+        component "Service\nIntegration" as si
+        '' config --> configdb
+        si --> init
+        resp --> si
+        config --> si
+    }
+    component "Service\nBusiness\nLogic" as back
+    back <--> si
+}
+'/
+
+others -down-> tp
+tp --> mr
+ms --> tp
+mr --> disp
+hapi -> intmgrs
+ms --> hapi
+mr --> hapi
+protos --> hapi
+rest -right-> protos
+'' protos --> http
+disp <--> protos
+disp --> ms
+intmgrs -up-> DL
+'' http --> resp
+'' resp --> rest
+'' init --> rest

docs/assets/deploymentModel-controller.puml

@@ -0,0 +1,76 @@
+skinparam defaultTextAlignment center
+
+skinparam titleBorderRoundCorner 15
+skinparam titleBorderThickness 2
+skinparam titleBorderColor red
+skinparam titleBackgroundColor White
+
+skinparam monochrome true
+
+skinparam componentStyle uml2
+
+/'
+cloud "The Internet" {
+    package "DID Method Network" as SN {
+        component "Distributed\nLedger" as DL
+    }
+    component "Other Agents" as others
+}
+'/
+package "Aries Cloud Agent" as ica {
+/'
+    package Core as core {
+        component "Transport Plugins" as tp
+        package  "Conductor" as cond {
+            component "Msg Receiver\nGet Thread State" as mr
+            component "Msg Sender\nPut Thread State" as ms
+        }
+        component "Dispatcher" as disp
+        component "Handler API" as hapi
+    }
+'/
+    package "Protocols" as prot {
+        component "Protocol 1\nProtocol 2\n.\n.\nProtocol n" as protos
+    }
+    component "Controller\nREST API" as rest
+/'
+    package "Handler Plugins" as inthand {
+        component "Storage Manager\nWallet Manager\nNode Pool Manager\n.\n.\n." as intmgrs
+    }
+    package "DID Method SDK" as is {
+        database "Secure Storage" as wallet
+    }
+    intmgrs --> wallet
+'/
+}
+
+
+package "Controller" as per {
+    component "HTTP Handler" as http
+    database "Agent\nController\nConfiguration" as config
+    component "Initiator" as init
+    component "Responder" as resp
+    '' database "Configuration\nFiles" as configdb
+    component "Business\nLogic" as si
+    '' config --> configdb
+    si --> init
+    resp --> si
+    config --> si
+}
+
+'' others -down-> tp
+'' tp --> mr
+'' ms --> tp
+'' mr --> disp
+'' hapi -> intmgrs
+'' ms --> hapi
+'' mr --> hapi
+'' mf --> hapi
+rest -right-> protos
+protos --> http: Web Hooks
+'' disp <--> protos
+'' disp --> ms
+'' intmgrs -up-> DL
+http --> resp
+resp --> rest
+init --> rest

docs/assets/deploymentModel-full.puml

@@ -0,0 +1,37 @@
+skinparam defaultTextAlignment center
+
+skinparam titleBorderRoundCorner 15
+skinparam titleBorderThickness 2
+skinparam titleBorderColor red
+skinparam titleBackgroundColor White
+
+skinparam monochrome true
+
+skinparam componentStyle uml2
+
+cloud "The Internet" {
+    package "DID Method Network" as SN {
+        component "Distributed\nLedger" as DL
+    }
+    component "Other Agents" as others
+}
+
+package "Aries Cloud Agent" as vx {
+    component "Core Capabilities" as core
+    package "DIDComm Protocols" as prot {
+        component "Protocol 1\nProtocol 2\n.\n.\nProtocol n" as protos
+    }
+    component "Controller\nREST API" as rest
+}
+
+package "Controller" as per {
+    component "Application \nBusiness\nLogic" as bl
+}
+
+others -down-> core
+protos -up-> core
+rest -right-> protos
+protos --> bl
+bl --> rest
+core <--> protos
+core -up-> SN

docs/assets/endorse-cred-def.puml

@@ -0,0 +1,75 @@
+@startuml
+' List of actors for our use case
+actor Admin
+participant CredDefRoutes
+participant RevocationRoutes
+participant IndyRevocation
+participant Ledger
+participant TransactionManager
+participant EventBus
+participant OutboundHandler
+participant EndorsedTxnHandler
+boundary OtherAgent
+
+' Sequence for writing a new credential definition
+Admin --> CredDefRoutes: POST /credential-definitions
+group Endorse transaction process
+CredDefRoutes --> Ledger: create_and_send_credential_definition()
+CredDefRoutes --> TransactionManager: create_record()
+CredDefRoutes --> TransactionManager: create_request()
+CredDefRoutes --> OutboundHandler: send_outbound_msg()
+OutboundHandler --> OtherAgent: send_msg()
+OtherAgent --> OtherAgent: endorse_msg()
+EndorsedTxnHandler <-- OtherAgent: send_msg()
+TransactionManager <-- EndorsedTxnHandler: receive_endorse_response()
+TransactionManager <-- EndorsedTxnHandler: complete_transaction()
+Ledger <-- TransactionManager: txn_submit()
+TransactionManager --> TransactionManager: endorsed_txn_post_processing()
+TransactionManager --> EventBus: notify_cred_def_event()
+end
+
+' Create the revocation registry once the credential definition is written
+CredDefRoutes <-- EventBus: on_cred_def_event()
+CredDefRoutes --> IndyRevocation: init_issuer_registry()
+IndyRevocation --> EventBus: notify_revocation_reg_init_event()
+RevocationRoutes <-- EventBus: on_revocation_registry_init_event()
+RevocationRoutes --> RevocationRoutes: generate_tails()
+group Endorse transaction process
+RevocationRoutes --> Ledger:send_revoc_reg_def()
+RevocationRoutes --> TransactionManager: create_record()
+RevocationRoutes --> TransactionManager: create_request()
+RevocationRoutes --> OutboundHandler: send_outbound_msg()
+OutboundHandler --> OtherAgent: send_msg()
+OtherAgent --> OtherAgent: endorse_msg()
+EndorsedTxnHandler <-- OtherAgent: send_msg()
+TransactionManager <-- EndorsedTxnHandler: receive_endorse_response()
+TransactionManager <-- EndorsedTxnHandler: complete_transaction()
+Ledger <-- TransactionManager: txn_submit()
+TransactionManager --> TransactionManager: endorsed_txn_post_processing()
+TransactionManager --> EventBus: notify_revocation_reg_endorsed_event()
+end
+
+' Now create the revocation entry (accumulator)
+RevocationRoutes <-- EventBus: on_revocation_registry_endorsed_event()
+RevocationRoutes --> RevocationRoutes: upload_tails()
+RevocationRoutes --> EventBus: notify_revocation_entry_event()
+RevocationRoutes <-- EventBus: on_revocation_entry_event()
+group Endorse transaction process
+RevocationRoutes --> IndyRevocation: send_entry()
+IndyRevocation --> Ledger: send_entry()
+RevocationRoutes --> TransactionManager: create_record()
+RevocationRoutes --> TransactionManager: create_request()
+RevocationRoutes --> OutboundHandler: send_outbound_msg()
+OutboundHandler --> OtherAgent: send_msg()
+OtherAgent --> OtherAgent: endorse_msg()
+EndorsedTxnHandler <-- OtherAgent: send_msg()
+TransactionManager <-- EndorsedTxnHandler: receive_endorse_response()
+TransactionManager <-- EndorsedTxnHandler: complete_transaction()
+Ledger <-- TransactionManager: txn_submit()
+TransactionManager --> TransactionManager: endorsed_txn_post_processing()
+
+' Notify that the revocation entry is completed (no one listens to this notification yet)
+TransactionManager --> EventBus: notify_revocation_entry_endorsed_event()
+end
+
+@enduml