-
Notifications
You must be signed in to change notification settings - Fork 84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add node mono repo post processor #1471
Merged
Merged
Changes from 11 commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
650b64c
initial commit of copied files
sofisl a7cf09b
feat: add nodejs-mono-repo post-processor
sofisl 6add7f9
run lint
sofisl cd2927d
run lint
sofisl 8dfc752
run lint
sofisl c875d45
address comments
sofisl 1545f75
address comments
sofisl 31028cb
Merge branch 'master' into nodeMonoRepoPostProcessor
sofisl b27e430
address comments
sofisl 7e81775
fix iterating path
sofisl 1449d08
delete extra file
sofisl 8006717
Merge branch 'master' into nodeMonoRepoPostProcessor
sofisl File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
# Copyright 2021 Google LLC | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# https://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
# Version 0.2.0 | ||
|
||
# build from the root of this repo: | ||
# docker build -t gcr.io/repo-automation-bots/owlbot-nodejs -f docker/owlbot/nodejs/Dockerfile . | ||
FROM python:3.10.5-buster | ||
|
||
WORKDIR / | ||
|
||
###################### Install nodejs. | ||
RUN curl -sL https://deb.nodesource.com/setup_15.x | bash - | ||
RUN apt-get install -y nodejs | ||
|
||
###################### Install synthtool's requirements. | ||
COPY requirements.txt /synthtool/requirements.txt | ||
RUN pip install -r /synthtool/requirements.txt | ||
|
||
# Put synthtool in the PYTHONPATH so owlbot.py scripts will find it. | ||
ENV PYTHONPATH="/synthtool" | ||
|
||
# Tell synthtool to pull templates from this docker image instead of from | ||
# the live repo. | ||
ENV SYNTHTOOL_TEMPLATES="/synthtool/synthtool/gcp/templates" | ||
|
||
# Copy synthtool. | ||
COPY synthtool /synthtool/synthtool | ||
COPY docker /synthtool/docker | ||
COPY post-processor-changes.txt /post-processor-changes.txt | ||
|
||
# Update permissions so non-root users won't see errors. | ||
RUN find /synthtool -exec chmod a+r {} \; | ||
RUN find /synthtool -type d -exec chmod a+x {} \; | ||
|
||
# Install dependencies used for post processing: | ||
# * gts/typescript are used for linting. | ||
# * google-gax is used for compiling protos. | ||
RUN cd /synthtool && mkdir node_modules && npm i [email protected] [email protected] [email protected] \ | ||
[email protected] [email protected] [email protected] [email protected] [email protected] [email protected] \ | ||
[email protected] [email protected] [email protected] [email protected] | ||
|
||
ENTRYPOINT [ "/bin/bash" ] | ||
CMD [ "/synthtool/docker/owlbot/nodejs_mono_repo/entrypoint.sh" ] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
steps: | ||
# Write the latest commit message to a file in the workspace. | ||
- name: 'gcr.io/cloud-builders/git' | ||
entrypoint: '/bin/sh' | ||
args: | ||
- '-c' | ||
- 'git log -1 --format="%s%n%n%b%nSource-Link: https://github.com/googleapis/synthtool/commit/%H" > post-processor-changes.txt' | ||
# Build the docker image. | ||
- name: 'gcr.io/cloud-builders/docker' | ||
args: [ 'build', | ||
'-t', 'gcr.io/repo-automation-bots/owlbot-nodejs-mono-repo:$SHORT_SHA', | ||
'-t', 'gcr.io/repo-automation-bots/owlbot-nodejs-mono-repo:latest', | ||
'-t', 'gcr.io/cloud-devrel-public-resources/owlbot-nodejs-mono-repo:$SHORT_SHA', | ||
'-t', 'gcr.io/cloud-devrel-public-resources/owlbot-nodejs-mono-repo:latest', | ||
'-f', 'docker/owlbot/nodejs-mono-repo/Dockerfile', '.' ] | ||
# Push the docker image. | ||
images: | ||
- gcr.io/repo-automation-bots/owlbot-nodejs-mono-repo:$SHORT_SHA | ||
- gcr.io/repo-automation-bots/owlbot-nodejs-mono-repo:latest | ||
- gcr.io/cloud-devrel-public-resources/owlbot-nodejs-mono-repo:$SHORT_SHA | ||
- gcr.io/cloud-devrel-public-resources/owlbot-nodejs-mono-repo:latest |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
#!/bin/bash | ||
# Copyright 2021 Google LLC | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# https://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
set -ex | ||
|
||
if [ -f owlbot.py ]; then | ||
python owlbot.py | ||
else | ||
python -m synthtool.languages.node_mono_repo | ||
fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
**/node_modules | ||
**/coverage | ||
test/fixtures | ||
build/ | ||
docs/ | ||
protos/ | ||
samples/generated/ |
3 changes: 3 additions & 0 deletions
3
synthtool/gcp/templates/node_mono_repo_library/.eslintrc.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
{ | ||
"extends": "./node_modules/gts" | ||
} |
4 changes: 4 additions & 0 deletions
4
synthtool/gcp/templates/node_mono_repo_library/.gitattributes
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
*.ts text eol=lf | ||
*.js text eol=lf | ||
protos/* linguist-generated | ||
**/api-extractor.json linguist-language=JSON-with-Comments |
29 changes: 29 additions & 0 deletions
29
synthtool/gcp/templates/node_mono_repo_library/.mocharc.js
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
// Copyright 2020 Google LLC | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
const config = { | ||
"enable-source-maps": true, | ||
"throw-deprecation": true, | ||
"timeout": 10000, | ||
"recursive": true | ||
} | ||
if (process.env.MOCHA_THROW_DEPRECATION === 'false') { | ||
delete config['throw-deprecation']; | ||
} | ||
if (process.env.MOCHA_REPORTER) { | ||
config.reporter = process.env.MOCHA_REPORTER; | ||
} | ||
if (process.env.MOCHA_REPORTER_OUTPUT) { | ||
config['reporter-option'] = `output=${process.env.MOCHA_REPORTER_OUTPUT}`; | ||
} | ||
module.exports = config |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
{ | ||
"report-dir": "./.coverage", | ||
"reporter": ["text", "lcov"], | ||
"exclude": [ | ||
"**/*-test", | ||
"**/.coverage", | ||
"**/apis", | ||
"**/benchmark", | ||
"**/conformance", | ||
"**/docs", | ||
"**/samples", | ||
"**/scripts", | ||
"**/protos", | ||
"**/test", | ||
"**/*.d.ts", | ||
".jsdoc.js", | ||
"**/.jsdoc.js", | ||
"karma.conf.js", | ||
"webpack-tests.config.js", | ||
"webpack.config.js" | ||
], | ||
"exclude-after-remap": false, | ||
"all": true | ||
} |
6 changes: 6 additions & 0 deletions
6
synthtool/gcp/templates/node_mono_repo_library/.prettierignore
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
**/node_modules | ||
**/coverage | ||
test/fixtures | ||
build/ | ||
docs/ | ||
protos/ |
17 changes: 17 additions & 0 deletions
17
synthtool/gcp/templates/node_mono_repo_library/.prettierrc.js
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
// Copyright 2020 Google LLC | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// https://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
module.exports = { | ||
...require('gts/.prettierrc.json') | ||
} |
94 changes: 94 additions & 0 deletions
94
synthtool/gcp/templates/node_mono_repo_library/CODE_OF_CONDUCT.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,94 @@ | ||
<!-- # Generated by synthtool. DO NOT EDIT! !--> | ||
# Code of Conduct | ||
|
||
## Our Pledge | ||
|
||
In the interest of fostering an open and welcoming environment, we as | ||
contributors and maintainers pledge to making participation in our project and | ||
our community a harassment-free experience for everyone, regardless of age, body | ||
size, disability, ethnicity, gender identity and expression, level of | ||
experience, education, socio-economic status, nationality, personal appearance, | ||
race, religion, or sexual identity and orientation. | ||
|
||
## Our Standards | ||
|
||
Examples of behavior that contributes to creating a positive environment | ||
include: | ||
|
||
* Using welcoming and inclusive language | ||
* Being respectful of differing viewpoints and experiences | ||
* Gracefully accepting constructive criticism | ||
* Focusing on what is best for the community | ||
* Showing empathy towards other community members | ||
|
||
Examples of unacceptable behavior by participants include: | ||
|
||
* The use of sexualized language or imagery and unwelcome sexual attention or | ||
advances | ||
* Trolling, insulting/derogatory comments, and personal or political attacks | ||
* Public or private harassment | ||
* Publishing others' private information, such as a physical or electronic | ||
address, without explicit permission | ||
* Other conduct which could reasonably be considered inappropriate in a | ||
professional setting | ||
|
||
## Our Responsibilities | ||
|
||
Project maintainers are responsible for clarifying the standards of acceptable | ||
behavior and are expected to take appropriate and fair corrective action in | ||
response to any instances of unacceptable behavior. | ||
|
||
Project maintainers have the right and responsibility to remove, edit, or reject | ||
comments, commits, code, wiki edits, issues, and other contributions that are | ||
not aligned to this Code of Conduct, or to ban temporarily or permanently any | ||
contributor for other behaviors that they deem inappropriate, threatening, | ||
offensive, or harmful. | ||
|
||
## Scope | ||
|
||
This Code of Conduct applies both within project spaces and in public spaces | ||
when an individual is representing the project or its community. Examples of | ||
representing a project or community include using an official project e-mail | ||
address, posting via an official social media account, or acting as an appointed | ||
representative at an online or offline event. Representation of a project may be | ||
further defined and clarified by project maintainers. | ||
|
||
This Code of Conduct also applies outside the project spaces when the Project | ||
Steward has a reasonable belief that an individual's behavior may have a | ||
negative impact on the project or its community. | ||
|
||
## Conflict Resolution | ||
|
||
We do not believe that all conflict is bad; healthy debate and disagreement | ||
often yield positive results. However, it is never okay to be disrespectful or | ||
to engage in behavior that violates the project’s code of conduct. | ||
|
||
If you see someone violating the code of conduct, you are encouraged to address | ||
the behavior directly with those involved. Many issues can be resolved quickly | ||
and easily, and this gives people more control over the outcome of their | ||
dispute. If you are unable to resolve the matter for any reason, or if the | ||
behavior is threatening or harassing, report it. We are dedicated to providing | ||
an environment where participants feel welcome and safe. | ||
|
||
Reports should be directed to *[email protected]*, the | ||
Project Steward(s) for *Google Cloud Client Libraries*. It is the Project Steward’s duty to | ||
receive and address reported violations of the code of conduct. They will then | ||
work with a committee consisting of representatives from the Open Source | ||
Programs Office and the Google Open Source Strategy team. If for any reason you | ||
are uncomfortable reaching out to the Project Steward, please email | ||
[email protected]. | ||
|
||
We will investigate every complaint, but you may not receive a direct response. | ||
We will use our discretion in determining when and how to follow up on reported | ||
incidents, which may range from not taking action to permanent expulsion from | ||
the project and project-sponsored spaces. We will notify the accused of the | ||
report and provide them an opportunity to discuss it before any action is taken. | ||
The identity of the reporter will be omitted from the details of the report | ||
supplied to the accused. In potentially harmful situations, such as ongoing | ||
harassment or threats to anyone's safety, we may take action without notice. | ||
|
||
## Attribution | ||
|
||
This Code of Conduct is adapted from the Contributor Covenant, version 1.4, | ||
available at | ||
https://www.contributor-covenant.org/version/1/4/code-of-conduct.html |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since this logic is pretty much identical to
def node_library
, I might just makenode_library
take an optionalrelative_dir
parameter, and then default to./
unless there's an alternaterelative_dir
provided.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Normally, I think it would be better to simply modify the existing code rather than duplicating it. However, I think in this case it's better to duplicate it, because: 1) it makes it easier to read and debug (just follow the node_mono_repo naming), and 2) we are eventually migrating to a mono repo, and eventually all the split repo paths will be deprecated. Given that it's only a few helper methods that are truly being copied over, I'd opt for copying over adding a new path to a helper method (this goes for other helper methods that are similar as well). Furthermore, if there are more specific changes we need to make for mono-repos, it will be easier to add to a mono-repo only path rather than making sure it doesn't affect split repos.