Merge branch 'chaoss:main' into gsoc-anuj

.github/CODEOWNERS

@@ -1,32 +1,32 @@
-/augur/cli/ @ccarterlandis
-/augur/metrics/ @ccarterlandis @sgoggins
-/augur/housekeeper/ @gabe-heim
-/augur/server.py @ccarterlandis @gabe-heim
-/augur/application.py @ccarterlandis @sgoggins @gabe-heim
-/augur/routes/ @ccarterlandis @gabe-heim @sgoggins
+/augur/cli/ @sgoggins
+/augur/metrics/ @sgoggins @sgoggins
+/augur/housekeeper/ @sgoggins
+/augur/server.py @sgoggins @sgoggins
+/augur/application.py @sgoggins @sgoggins @sgoggins
+/augur/routes/ @sgoggins @sgoggins @sgoggins
 
-/frontend/ @gabe-heim @sgoggins
+/frontend/ @sgoggins @sgoggins
 
-/schema/ @sgoggins @ccarterlandis
+/schema/ @sgoggins @sgoggins
 
 /schema/generate @sgoggins 
 
-/workers/ @ccarterlandis @gabe-heim @sgoggins
+/workers/ @sgoggins @sgoggins @sgoggins
 
-/README.md @sgoggins @ccarterlandis
-/CONTRIBUTING.md @sgoggins @ccarterlandis
-/LICENSE @sgoggins @ccarterlandis
-/CODE_OF_CONDUCT.md @sgoggins @ccarterlandis
+/README.md @sgoggins @sgoggins
+/CONTRIBUTING.md @sgoggins @sgoggins
+/LICENSE @sgoggins @sgoggins
+/CODE_OF_CONDUCT.md @sgoggins @sgoggins
 
-/tests/ @ccarterlandis
-/util/docker @ccarterlandis
-/scripts/ @ccarterlandis
-/docs/ @ccarterlandis
-/.dockerignore @ccarterlandis
-/.travis.yml @ccarterlandis
-*compose.yml @ccarterlandis
-/readthedocs.yml @ccarterlandis
-/Makefile @ccarterlandis
-/tox.ini @ccarterlandis
-*requirements.txt @ccarterlandis
-/setup.py @ccarterlandis @sgoggins
+/tests/ @sgoggins
+/util/docker @sgoggins
+/scripts/ @sgoggins
+/docs/ @sgoggins
+/.dockerignore @sgoggins
+/.travis.yml @sgoggins
+*compose.yml @sgoggins
+/readthedocs.yml @sgoggins
+/Makefile @sgoggins
+/tox.ini @sgoggins
+*requirements.txt @sgoggins
+/setup.py @sgoggins @sgoggins

.gitignore

@@ -170,3 +170,6 @@ workers/clustering_worker/vocabulary_count
 
 #nohup logs
 nohup.out
+
+#vscode 
+.vscode

augur/config.py

@@ -66,6 +66,14 @@
                     "model": "deps",
                     "repo_group_id": 0
                 },
+                {
+                    "delay": 150000,
+                    "given": [
+                        "git_url"
+                    ],
+                    "model": "ossf_scorecard",
+                    "repo_group_id": 0
+                },
                 {
                     "delay": 150000,
                     "given": [

docs/source/development-guide/workers/toc.rst

@@ -43,7 +43,7 @@ Models & Tables they populate-
 
     1.9.  message: Stores the messages on the MR thread.
 
-    1.10: contributors: Stores the information related to each contributor of the project. It's implementation is in the base worker class. `query_gitlab_contribtutors`
+    1.10: contributors: Stores the information related to each contributor of the project. It's implementation is in the base worker class. `query_gitlab_contributors`
 
 2. merge_request_commits:  This model deals with the commit data of each MR-
 

docs/source/docker/docker-compose.rst

@@ -50,6 +50,8 @@ Stopping the containers
 
 To stop the containers, do a keyboard inturrupt while the script is running ``Ctrl+C``. The script will then ask if you want to generate log files to look at later.
 
+If not using the script, the standard method of stopping the containers that you started should work such as ``docker stop`` or ``docker-compose down``
+
 Once you've got your container up and running, checkout out `how to use them <usage.html>`_ 
 
 
@@ -58,24 +60,35 @@ Docker Compose without a script
 
 This section of the documentation details how to use Augur's Docker Compose configuration to get the full stack up and running as fast as possible without the recommended helper script. 
 
+.. warning::
+
+    Don't forget to provide your external database credentials in the ``docker_env.txt`` file. Additionally an ``.env`` file is needed for the ``*.yml`` files' environment variables. Don't forget to set the variables specified specified in these files namely ``AUGUR_DB_TYPE`` and ``AUGUR_DB_HOST``.
+
+    Example docker_env.txt:
+    .. code:: 
+
+        AUGUR_GITHUB_API_KEY=your_key_here
+        AUGUR_DB_SCHEMA_BUILD=0
+        AUGUR_DB_HOST=xx.xxx.xxx.xxx
+        AUGUR_DB_NAME=augur
+        AUGUR_DB_PORT=5432
+        AUGUR_DB_USER=augur
+        AUGUR_DB_PASSWORD=somePassword
+
 
 To run Augur **without** the database container:
 
 .. code-block:: bash
 
     docker-compose -f docker-compose.yml up
 
-.. warning::
-
-    Don't forget to provide your external database credentials in the ``docker_env.txt`` file. Additionally the ``.env`` file is needed for the ``*.yml`` files' environment variables. Don't forget to set the variables specified specified in these files namely ``AUGUR_DB_TYPE`` and ``AUGUR_DB_HOST``.
-
 To run Augur **with** the database container:
 
 .. code-block:: bash
 
     docker-compose -f docker-compose.yml -f database-compose.yml up
 
-If you want to use the ``test_data`` image with the data preloaded, change the ``image`` line of ``database-compose.yml`` to\:
+If you want to use the ``test_data`` image with the data preloaded, change the ``image`` line of ``database-compose.yml`` to:
 
 .. code::
 

docs/source/docker/getting-started.rst

@@ -7,10 +7,10 @@ For the Docker Saavy Who Want to Understand How the Sausage is Made:
 Augur provides several Docker images designed to get you started with our software as quickly as possible. They are:
 
 - ``augurlabs/augur:backend``, our backend data collection and metrics API
-- ``augurlabs/augur:frontend``, our metrics visualization frontend
+- ``augurlabs/augur:frontend``, our metrics visualization frontend (Experimental, will be replaced in the future)
 
 .. warning::
-   The frontend is very out of date and will likely not work with the backend very well. It is still available however.
+   The frontend is very out of date and will likely not work. It is still available however it is in the process of being replaced with an entirely new frontend so the old frontend is not being actively fixed.
 
 - ``augurlabs/augur:database``, an empty PostgreSQL database with the Augur schema installed
 - ``augurlabs/augur:test_data``, a PostgreSQL database loaded with the data used in our testing environment
@@ -34,7 +34,7 @@ Before you get started with Docker, you'll need to set up a PostgreSQL instance
       # TYPE  DATABASE        USER            ADDRESS                 METHOD
       host	all  		all 		0.0.0.0/0 		md5
 
-If you're interested solely in data collection and do not care if your data is not persisted, we recommend using our Docker Compose script. This will start up the backend and frontend containers simultaneously, well as an optional database container; however, if you are looking to collect data long term, we **strongly suggest setting up a persistent database instance**; you can find instructions for doing so `here <../getting-started/database.html>`_. Remember to save off the credentials for your newly minted database; you'll need them shortly. 
+If you're interested solely in data collection, we recommend using our test data with the Docker Compose script. This will start up the backend and frontend containers simultaneously, well as an optional database container; however, if you are looking to collect data long term, we **strongly suggest setting up a persistent database instance**; you can find instructions for doing so `here <../getting-started/database.html>`_. Remember to save off the credentials for your newly minted database; you'll need them shortly. 
 
 If you don't care if your data doesn't get persisted or are doing local development, you can use the database containers we provide. 
 

docs/source/docker/quick-start.rst

@@ -3,13 +3,13 @@ Docker Quick Start
 Before you get off to such a quick start, go ahead and 
   1. create a fork from augur starting at https://github.com/chaoss/augur
   2. clone that fork locally
-  3. Checkout the appropriate branch to work on (see notes below). For right now, to checkout the ``test`` branch.:
+  3. Checkout the appropriate branch to work on (see notes below):
 
   .. code-block:: python
   
-     git checkout test 
+     git checkout dev 
   
-  4. Usually, we'll have you checkout the `dev` branch, but through August 9, 2021, that branch is undergoing refactoring, to be be maintable, and is not reliable for new development 
+  4. Usually, we'll have you checkout the `dev` branch.
   5. Make sure to install all the pre-requisites here: https://oss-augur.readthedocs.io/en/master/getting-started/installation.html#dependencies
 
 
@@ -34,4 +34,3 @@ Before you get off to such a quick start, go ahead and
 
     sudo ./docker-setup.sh
 
-**Detailed Instructions that explain each step, and alternative, are available on the next page**

docs/source/docker/usage.rst

@@ -3,32 +3,6 @@ Interacting with the containers
 
 Once the containers are up and running, you have a few options for interacting with them. They will automatically collect data for your repositories - but how do you add repositories? We're glad you asked!
 
-Using the repo loading UI
---------------------------
-
-.. warning::
-    The frontend is very out of date and will likely not work with the backend very well. It is still available however. The recommended way of accessing augur through docker is through standard api calls at ``localhost:5000``.
-
-
-Augur offers a special graphical interface for loading repository groups when using the Docker containers. This component is called ``augurface``, and is available anytime you are using the ``backend`` and ``frontend`` services together.
-
-To use it, first start the two services (we recommend using `Docker Compose Script <docker-compose.html>`_ for this):
-
-.. code-block:: bash
-
-    # this example uses the docker-setup script
-    $ sudo ./docker-setup.sh
-
-Then, navigate to ``http://localhost:8080/augurface/`` in your browser - **note the trailing slash!** Once you're on this page, you'll need to enter in your Augur API key in the box on the top right. On a default Docker installation, you can use ``docker_key``, but we recommend changing this as soon as possible if you are planning to use the instance long-term. Commands for working with the API keys can be found `here <../getting-started/command-line-interface/db.html>`_.
-
-Once you've entered your API key, you will be able to use the UI to automatically import GitHub organizations as a repo group, or manually create and edit repo groups yourself. Deleting repos or repo groups is not currently supported, but would be a great contribution!
-
-.. warning::
-
-    Because the UI only requires an API key to edit the database, **we recommend that you do not publicly deploy any Docker instance outside your local network or intranet.** The Docker build is intended ONLY for short term data collection usage and local development. We understand if this inconvenciences you, but the functionality is very new to Augur and still needs time to be production ready. **You have been warned.**
-
-If are not using the ``frontend`` service, you can use the `database CLI <../getting-started/command-line-interface/db.html>`_ from within the container to add repos. See below for how to start a shell within the container.
-
 
 Accessing the containers
 ---------------------------
@@ -39,7 +13,7 @@ If you need to access a running container (perhaps to check the worker logs) or
 
     $ docker exec -it <service_name> /bin/bash
 
-You can also step into a running container at every step of the build process and see the status of the container.
+You can also step into a running container at every step of the build process and see the status of the container. (This is typically used for debugging)
 
 First, build the image to output build stages.
 
@@ -66,9 +40,39 @@ By default, the only logs shown by the container are the logs of Augur's main da
     # to watch the logs in real time (like tail -f)
     $ docker-compose logs -f
 
-If you deployed the augur frontend and backend together using the script, you are automtically prompted to ask to save logs to a file once you stop the containers. The logs can be found in ``/var/log/``
+
+As for worker logs. They are currently a work in progress to be made easier to view. In the near future they will automatically populate on the host machine and it will not be neccessary to step inside the container.
+
+Using the repo loading UI [NOT RECOMMENDED]
+-------------------------------------------
+
+.. warning::
+    The frontend is very out of date and will *very likely* not work with the backend. It is still available however. The recommended way of accessing augur through docker is through standard api calls at ``localhost:5000``.
+
+
+Augur offers a special graphical interface for loading repository groups when using the Docker containers. This component is called ``augurface``, and is available anytime you are using the ``backend`` and ``frontend`` services together.
+
+To use it, first start the two services (we recommend using `Docker Compose Script <docker-compose.html>`_ for this):
+
+.. code-block:: bash
+
+    # this example uses the docker-setup script
+    $ sudo ./docker-setup.sh
+
+Then, navigate to ``http://localhost:8080/augurface/`` in your browser - **note the trailing slash!** Once you're on this page, you'll need to enter in your Augur API key in the box on the top right. On a default Docker installation, you can use ``docker_key``, but we recommend changing this as soon as possible if you are planning to use the instance long-term. Commands for working with the API keys can be found `here <../getting-started/command-line-interface/db.html>`_.
+
+Once you've entered your API key, you will be able to use the UI to automatically import GitHub organizations as a repo group, or manually create and edit repo groups yourself. Deleting repos or repo groups is not currently supported, but would be a great contribution!
+
+.. warning::
+
+    Because the UI only requires an API key to edit the database, **we recommend that you do not publicly deploy any Docker instance outside your local network or intranet.** The Docker build is intended ONLY for short term data collection usage and local development. We understand if this inconvenciences you, but the functionality is very new to Augur and still needs time to be production ready. **You have been warned.**
+
+If are not using the ``frontend`` service, you can use the `database CLI <../getting-started/command-line-interface/db.html>`_ from within the container to add repos. See below for how to start a shell within the container.
+
+
 
 Conclusion
 -----------
 
 This wraps up the Docker section of the Augur documentation. We hope it was useful! Happy hacking!
+

docs/source/rest-api/spec.yml

@@ -74,7 +74,7 @@ paths:
             type: array
       tags:
       - utility
-  /repo-groups/:repo_group_id/Committers:
+  /repo-groups/:repo_group_id/committers:
     get:
       description: 'Number of persons opening an issue for the first time. '
       externalDocs: