feat: add node mono repo post processor (#1471)

* feat: add node mono repo post processor

docker/owlbot/nodejs_mono_repo/Dockerfile

@@ -0,0 +1,55 @@
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Version 0.2.0
+
+# build from the root of this repo:
+# docker build -t gcr.io/repo-automation-bots/owlbot-nodejs -f docker/owlbot/nodejs/Dockerfile .
+FROM python:3.10.5-buster
+
+WORKDIR /
+
+###################### Install nodejs.
+RUN curl -sL https://deb.nodesource.com/setup_15.x | bash -
+RUN apt-get install -y nodejs
+
+###################### Install synthtool's requirements.
+COPY requirements.txt /synthtool/requirements.txt
+RUN pip install -r /synthtool/requirements.txt
+
+# Put synthtool in the PYTHONPATH so owlbot.py scripts will find it.
+ENV PYTHONPATH="/synthtool"
+
+# Tell synthtool to pull templates from this docker image instead of from
+# the live repo.
+ENV SYNTHTOOL_TEMPLATES="/synthtool/synthtool/gcp/templates"
+
+# Copy synthtool.
+COPY synthtool /synthtool/synthtool
+COPY docker /synthtool/docker
+COPY post-processor-changes.txt /post-processor-changes.txt
+
+# Update permissions so non-root users won't see errors.
+RUN find /synthtool -exec chmod a+r {} \;
+RUN find /synthtool -type d -exec chmod a+x {} \;
+
+# Install dependencies used for post processing:
+# * gts/typescript are used for linting.
+# * google-gax is used for compiling protos.
+RUN cd /synthtool && mkdir node_modules && npm i [email protected] [email protected] [email protected] \
+    [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] \
+    [email protected] [email protected] [email protected] [email protected]
+
+ENTRYPOINT [ "/bin/bash" ]
+CMD [ "/synthtool/docker/owlbot/nodejs_mono_repo/entrypoint.sh" ]

docker/owlbot/nodejs_mono_repo/cloudbuild.yaml

@@ -0,0 +1,21 @@
+steps:
+    # Write the latest commit message to a file in the workspace.
+    - name: 'gcr.io/cloud-builders/git'
+      entrypoint: '/bin/sh'
+      args:
+        - '-c'
+        - 'git log -1 --format="%s%n%n%b%nSource-Link: https://github.com/googleapis/synthtool/commit/%H" > post-processor-changes.txt'
+    # Build the docker image.
+    - name: 'gcr.io/cloud-builders/docker'
+      args: [ 'build',
+        '-t', 'gcr.io/repo-automation-bots/owlbot-nodejs-mono-repo:$SHORT_SHA',
+        '-t', 'gcr.io/repo-automation-bots/owlbot-nodejs-mono-repo:latest',
+        '-t', 'gcr.io/cloud-devrel-public-resources/owlbot-nodejs-mono-repo:$SHORT_SHA',
+        '-t', 'gcr.io/cloud-devrel-public-resources/owlbot-nodejs-mono-repo:latest',
+        '-f', 'docker/owlbot/nodejs-mono-repo/Dockerfile', '.' ]
+    # Push the docker image.
+images:
+  - gcr.io/repo-automation-bots/owlbot-nodejs-mono-repo:$SHORT_SHA
+  - gcr.io/repo-automation-bots/owlbot-nodejs-mono-repo:latest
+  - gcr.io/cloud-devrel-public-resources/owlbot-nodejs-mono-repo:$SHORT_SHA
+  - gcr.io/cloud-devrel-public-resources/owlbot-nodejs-mono-repo:latest

docker/owlbot/nodejs_mono_repo/entrypoint.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -ex
+
+if [ -f owlbot.py ]; then
+    python owlbot.py
+else
+    python -m synthtool.languages.node_mono_repo
+fi

synthtool/gcp/common.py

@@ -24,7 +24,7 @@
 
 from synthtool import shell, _tracked_paths
 from synthtool.gcp import partials
-from synthtool.languages import node
+from synthtool.languages import node, node_mono_repo
 from synthtool.log import logger
 from synthtool.sources import git, templates
 
@@ -326,6 +326,33 @@ def node_library(self, **kwargs) -> Path:
 
         return self._generic_library("node_library", **kwargs)
 
+    def node_mono_repo_library(self, relative_dir, **kwargs) -> Path:
+        # TODO: once we've migrated all Node.js repos to either having
+        #  .repo-metadata.json, or excluding README.md, we can remove this.
+        if not os.path.exists(Path(relative_dir, "/.repo-metadata.json").resolve()):
+            self.excludes.append("README.md")
+            if "samples/README.md" not in self.excludes:
+                self.excludes.append("samples/README.md")
+
+        kwargs["metadata"] = node_mono_repo.template_metadata(relative_dir)
+
+        ignore_src_index = [
+            "yes" for f in self.excludes if fnmatch.fnmatch("src/index.ts", f)
+        ]
+        # generate root-level `src/index.ts` to export multiple versions and its default clients
+        if (
+            "versions" in kwargs
+            and "default_version" in kwargs
+            and not ignore_src_index
+        ):
+            node_mono_repo.generate_index_ts(
+                versions=kwargs["versions"],
+                default_version=kwargs["default_version"],
+                relative_dir=relative_dir,
+            )
+
+        return self._generic_library("node_mono_repo_library", **kwargs)
+
     def php_library(self, **kwargs) -> Path:
         return self._generic_library("php_library", **kwargs)
 

synthtool/gcp/templates/node_mono_repo_library/.eslintignore

@@ -0,0 +1,7 @@
+**/node_modules
+**/coverage
+test/fixtures
+build/
+docs/
+protos/
+samples/generated/

synthtool/gcp/templates/node_mono_repo_library/.eslintrc.json

@@ -0,0 +1,3 @@
+{
+  "extends": "./node_modules/gts"
+}

synthtool/gcp/templates/node_mono_repo_library/.gitattributes

@@ -0,0 +1,4 @@
+*.ts text eol=lf
+*.js text eol=lf
+protos/* linguist-generated
+**/api-extractor.json  linguist-language=JSON-with-Comments

synthtool/gcp/templates/node_mono_repo_library/.mocharc.js

@@ -0,0 +1,29 @@
+// Copyright 2020 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+const config = {
+  "enable-source-maps": true,
+  "throw-deprecation": true,
+  "timeout": 10000,
+  "recursive": true
+}
+if (process.env.MOCHA_THROW_DEPRECATION === 'false') {
+  delete config['throw-deprecation'];
+}
+if (process.env.MOCHA_REPORTER) {
+  config.reporter = process.env.MOCHA_REPORTER;
+}
+if (process.env.MOCHA_REPORTER_OUTPUT) {
+  config['reporter-option'] = `output=${process.env.MOCHA_REPORTER_OUTPUT}`;
+}
+module.exports = config

synthtool/gcp/templates/node_mono_repo_library/.nycrc

@@ -0,0 +1,24 @@
+{
+  "report-dir": "./.coverage",
+  "reporter": ["text", "lcov"],
+  "exclude": [
+    "**/*-test",
+    "**/.coverage",
+    "**/apis",
+    "**/benchmark",
+    "**/conformance",
+    "**/docs",
+    "**/samples",
+    "**/scripts",
+    "**/protos",
+    "**/test",
+    "**/*.d.ts",
+    ".jsdoc.js",
+    "**/.jsdoc.js",
+    "karma.conf.js",
+    "webpack-tests.config.js",
+    "webpack.config.js"
+  ],
+  "exclude-after-remap": false,
+  "all": true
+}

synthtool/gcp/templates/node_mono_repo_library/.prettierignore

@@ -0,0 +1,6 @@
+**/node_modules
+**/coverage
+test/fixtures
+build/
+docs/
+protos/

synthtool/gcp/templates/node_mono_repo_library/.prettierrc.js

@@ -0,0 +1,17 @@
+// Copyright 2020 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+module.exports = {
+  ...require('gts/.prettierrc.json')
+}

synthtool/gcp/templates/node_mono_repo_library/CODE_OF_CONDUCT.md

@@ -0,0 +1,94 @@
+<!-- # Generated by synthtool. DO NOT EDIT! !-->
+# Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of
+experience, education, socio-economic status, nationality, personal appearance,
+race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+*   Using welcoming and inclusive language
+*   Being respectful of differing viewpoints and experiences
+*   Gracefully accepting constructive criticism
+*   Focusing on what is best for the community
+*   Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+*   The use of sexualized language or imagery and unwelcome sexual attention or
+    advances
+*   Trolling, insulting/derogatory comments, and personal or political attacks
+*   Public or private harassment
+*   Publishing others' private information, such as a physical or electronic
+    address, without explicit permission
+*   Other conduct which could reasonably be considered inappropriate in a
+    professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, or to ban temporarily or permanently any
+contributor for other behaviors that they deem inappropriate, threatening,
+offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+This Code of Conduct also applies outside the project spaces when the Project
+Steward has a reasonable belief that an individual's behavior may have a
+negative impact on the project or its community.
+
+## Conflict Resolution
+
+We do not believe that all conflict is bad; healthy debate and disagreement
+often yield positive results. However, it is never okay to be disrespectful or
+to engage in behavior that violates the project’s code of conduct.
+
+If you see someone violating the code of conduct, you are encouraged to address
+the behavior directly with those involved. Many issues can be resolved quickly
+and easily, and this gives people more control over the outcome of their
+dispute. If you are unable to resolve the matter for any reason, or if the
+behavior is threatening or harassing, report it. We are dedicated to providing
+an environment where participants feel welcome and safe.
+
+Reports should be directed to *[email protected]*, the
+Project Steward(s) for *Google Cloud Client Libraries*. It is the Project Steward’s duty to
+receive and address reported violations of the code of conduct. They will then
+work with a committee consisting of representatives from the Open Source
+Programs Office and the Google Open Source Strategy team. If for any reason you
+are uncomfortable reaching out to the Project Steward, please email
+[email protected].
+
+We will investigate every complaint, but you may not receive a direct response.
+We will use our discretion in determining when and how to follow up on reported
+incidents, which may range from not taking action to permanent expulsion from
+the project and project-sponsored spaces. We will notify the accused of the
+report and provide them an opportunity to discuss it before any action is taken.
+The identity of the reporter will be omitted from the details of the report
+supplied to the accused. In potentially harmful situations, such as ongoing
+harassment or threats to anyone's safety, we may take action without notice.
+
+## Attribution
+
+This Code of Conduct is adapted from the Contributor Covenant, version 1.4,
+available at
+https://www.contributor-covenant.org/version/1/4/code-of-conduct.html