MVP Roadmap

[UlisesGascon]

Goals

Before starting MVP development, we completed a Proof of Concept (POC) (official presentation, video, source code, slides).

The primary goal of the MVP is to deprecate the old POC and redesign its components based on the feedback collected during and after its presentation.

The MVP will also enable several critical capabilities as we transition toward the product development phase:

• Enable external collaboration: Provide tools, guides, and a clear way to identify tasks for contributors. Avoid requiring high-level access to GitHub APIs or infrastructure access for contributions.
• Support complex workflows: Allow repository content analysis, enabling operations beyond the HTTP API capabilities of the POC.
• Smooth dashboard-website integration: Ensure seamless integration with the website.
• Adopt a relational database: Move away from the local filesystem database in favor of a relational database with features like migrations and seeding for improved ergonomics.
• Establish distinct environments: Create separate development and production environments to store and update calculated data from executors (e.g., long-running analyses like scorecards).
• Generate and manage artifacts: Produce public/private artifacts from the analysis (e.g., results, alerts, tasks) and implement a notification system for relevant stakeholders.

Backlog

✅ Milestone 1: Adopt a relational database and migrate away from the POC

Transition from a local filesystem database to a relational database with robust features like migrations and seeding.

Achieved

• Choose a database (Postgresql) and implement a development workflow (npm scripts (npm run infra:start and npm run infra:stop), adminer...) using Docker (Add local database (postgres) and adminer using Docker #1) (Docs)
• Configure a SQL builder (Knex) and include the option to run migrations (npm run db:migrate), seeds (npm run db:seed), etc... (Add Database Management commands and dependencies #2)(docs)
• Record GitHub orgs when adding a project (Record GitHub orgs when adding a project #21, Ensure that the github organizations are linked to projects #25)
• Add workflow update-github-orgs (Add workflow update-github-orgs  #23) (docs)
• Add workflow upsert-github-repositories (Add workflow upsert-github-repositories #32) (docs)
• Add seeds that allow to run the project without high level permissions (like GH tokens with org scope) (Add a simple seed script #34) (docs)
• Include the table compliance_checks (Add and populate compliance_checks table #36)
• Include the table compliance_checks_alerts (create compliance_checks_alerts table #40)
• Include the table compliance_checks_tasks (create compliance_checks_tasks table #41)
• Include the table compliance_checks_results (create compliance_checks_results table #42)

🔄 Milestone 2: Enable external collaboration

Provide tooling, guides, and accessible workflows to encourage contributor involvement without requiring high-level access.

Achieved

• Add basic tooling for kickoff the ci pipelines including testing and linter (Improve developer experience (ci, tests, linter) #3) (docs)
• Include and update via scripts (npm run db:generate-schema) a database schema dump without the need to run the project locally and enable the option to port the schemas into a different tools (Add support for database schemas #4) (docs, schema)
• Add support to configuration (environmental variables, etc..) (Add support to handle sensitive information (tokens...) #22) (docs)
• Add a basic logger that is muted while running the tests (Add a basic logger handler #24)
• Add a GH pipeline that provide support (tasks, suggestions...) while dealing with migrations or checks (Improve CI pipelines with more and better comments #47) (docs)
• Add CONTRIBUTING.md guide with clear sections on create a good PRs in this project and How to add compliance checks
• Add SECURITY.md
• Add CODEOWNERS
• Add Code of conduct (COC)
• Add issue templates
• Create a solid README.md

Pending

• @UlisesGascon Setup regular working meetings (every 2w) with collaborators in the OpenJS foundation calendar
• @UlisesGascon Onboard the first contributors to the project
• @UlisesGascon Land at least one compliance check developed by the community (Carlos, Iñigo, Tobias, Alpi...)

✅ Milestone 3: Improve website integration

Ensure seamless integration between the dashboard and the OpenJS Security Program Standards website.

Achieved

• Support website release 2.0 by creating a script to download the content of compliance_checks in json format (feat: add script to export compliance_checks table #51)

🔄 Milestone 4: Enable artifacts generation

Support the creation of artifacts such as reports, analysis results, and alerts, with mechanisms for notifying stakeholders.

Achieved

• Include all the artifacts generation to an specific folder (/output/*) and avoid commit then by include a .gitignore rule

Pending

• @UlisesGascon Add an script to send the generated artifacts to a private artifact directory (TBC)
• @UlisesGascon Migrate the legacy POC scripts/templates to the MVP and ensure that we can at least generate the same markdown samples but using the database this time
• @inigomarquinez brainstorm and generated web rich resources that can be embedded into the website (public dashboard information) as a POC

🔄 Milestone 5: Ensure compatibility with diverse checks

Support various types of checks, including manual tasks, automated workflows, and complex executors like long-running analyses.

Achieved

• Add CLI and project addition capabilities (Add CLI and project addition capabilities #5) (Docs)
• Add JSON Schemas validations for all the GH APIs endpoints in use (Add GitHub Org API query response validation #20, Add JSON schema validation to the GH API responses #27 and Add JSON Schema for Github API list Organization repositories #28)
• Add CLI command check list (Add cli command check list #37) (docs)
• Add CLI command check run (Add CLI command check run #38) (docs)
• Add workflow run-all-checks (Add workflow run-all-checks #39) (docs)
• Land the first automated check (githubOrgMFA), including CLI integration (Add check for githubOrgMFA #44) and business logic (Add Compliance check: githubOrgMFA #43) with unit and integration tests in place. Also website manual addition of the query used by the dashboard (Add evaluation criteria for githubOrgMFA website#9)
• Land a compliance check that relies on manual validation
• Add support to run workflows that require long execution times using containers and child processes (Include OSSF Scorecard analysis using containers #62)

Pending

• @UlisesGascon Create issues to split workload for adding compliance checks
• @UlisesGascon build and land a complex compliance check that requires to run a child process against all the repositories like the OSSF scorecard or similar.

🔄 Milestone 6: Create development/production environments

Establish environments for storing and updating calculated data generated during execution.

Achieved
Not started yet

Pending

• @UlisesGascon create a cloud resource that acts as artifact container
• @UlisesGascon create an external database for production and staging environment
• @UlisesGascon ensure that the database can be connected easily from the local environment so the complex checks can run in a distributed environment
• @UlisesGascon to provide resources from his local selfhost environment for computational work