Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add littlequery service #14

Open
wants to merge 80 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
b053787
services diff gha
BBBmau Feb 15, 2024
15b29ce
add needs
BBBmau Feb 15, 2024
bf4d864
use setup-go@v3
BBBmau Feb 15, 2024
1b02395
add artifacts
BBBmau Feb 15, 2024
2e46cd8
remove checkout
BBBmau Feb 15, 2024
39f413d
update directory
BBBmau Feb 16, 2024
4f4fd37
update common-copy.yaml
BBBmau Feb 16, 2024
ae84cde
update common-copy.yaml
BBBmau Feb 16, 2024
4cbb813
remove data print
BBBmau Feb 16, 2024
e3114e8
add pull-request check on teamcitydiff
BBBmau Feb 16, 2024
205b550
update paths in GHA
BBBmau Feb 16, 2024
3c16df3
add exit code in diff_check
BBBmau Feb 16, 2024
b1cd675
update exit output
BBBmau Feb 16, 2024
5971c66
add exit code diff_check.go
BBBmau Feb 16, 2024
38cfdb1
services_beta diff check support
BBBmau Feb 16, 2024
3e0e56b
beta services support args
BBBmau Feb 16, 2024
590b54b
add arg for services kt file
BBBmau Feb 16, 2024
a9e631f
services_ga.kt in gha
BBBmau Feb 16, 2024
25819d8
move diff_check to tools folder
BBBmau Feb 16, 2024
8604188
remove go command
BBBmau Feb 16, 2024
fdcc36e
provide path for artifacts download
BBBmau Feb 16, 2024
96a8d3d
add cd in Build Provider
BBBmau Feb 16, 2024
dcde4c7
update diff_check_beta.yml
BBBmau Feb 16, 2024
71b7997
fix paths
BBBmau Feb 16, 2024
8d1a003
yml typo beta
BBBmau Feb 16, 2024
95ad364
directory testing
BBBmau Feb 16, 2024
af8422e
directory testing
BBBmau Feb 16, 2024
8ef04c9
fix directory issues with commands
BBBmau Feb 17, 2024
d1f73ba
refactor tools/teamcity-diff-check/main.go
BBBmau Feb 17, 2024
8567836
directory check
BBBmau Feb 18, 2024
63c8c08
typo
BBBmau Feb 18, 2024
3ae5129
remove common copy
BBBmau Feb 18, 2024
36ceb3b
generate both providers into one gha
BBBmau Feb 18, 2024
136bd4b
type artifacts name
BBBmau Feb 18, 2024
6177b5c
add -o flag into artifacts download
BBBmau Feb 18, 2024
f461781
use merge multiple artifacts
BBBmau Feb 19, 2024
88c2054
use artifact@v4
BBBmau Feb 19, 2024
4b1d6d8
use mmv1 directory for services file
BBBmau Feb 20, 2024
a977f86
remove .zip
BBBmau Feb 20, 2024
8578505
output missing services from diff
BBBmau Feb 20, 2024
7d7026c
find artifacts folder
BBBmau Feb 20, 2024
4ba11ee
use beta provider as only artifact
BBBmau Feb 20, 2024
94912ce
include google ga and beta in gha
BBBmau Feb 20, 2024
cd2dd3f
artifact name
BBBmau Feb 20, 2024
6f0a087
-o flag in artifacts
BBBmau Feb 20, 2024
c8806a4
output stdout from go list command
BBBmau Feb 20, 2024
0836e95
diff test
BBBmau Feb 20, 2024
04a3fe5
output beta main.go
BBBmau Feb 20, 2024
8b95460
remove err check
BBBmau Feb 20, 2024
00ce67a
remove services print / refactor for final review
BBBmau Feb 20, 2024
03a02c3
output cleanup
BBBmau Feb 20, 2024
f1b1897
uncomment go list err
BBBmau Feb 21, 2024
a3f8c03
regex assert
BBBmau Feb 21, 2024
4f19264
add check for new services in PR / set googleServices as groundtruth …
BBBmau Feb 22, 2024
2b3e2c5
add actions/checkout
BBBmau Feb 22, 2024
1fe2a90
logic fix
BBBmau Feb 22, 2024
99509da
add needs: check-pr
BBBmau Feb 22, 2024
7b5741b
add mmv1/products for pull-request event
BBBmau Feb 22, 2024
b577bf3
set force depth 0
BBBmau Feb 22, 2024
aa41aba
Update .github/workflows/teamcity-services-diff-check.yml
BBBmau Feb 22, 2024
187f356
Update .github/workflows/teamcity-services-diff-check.yml
BBBmau Feb 22, 2024
3ea8f5b
use text file for services, add if statement in gha
BBBmau Feb 22, 2024
453d637
directory typo
BBBmau Feb 22, 2024
aae6e15
typo
BBBmau Feb 22, 2024
7c89158
typo in grep
BBBmau Feb 22, 2024
898f6f2
quotations on if statement
BBBmau Feb 22, 2024
5ffb26b
unquote GITHUB_OUTPUT
BBBmau Feb 22, 2024
2fe8124
add quotes on output.services
BBBmau Feb 22, 2024
7589004
if test
BBBmau Feb 22, 2024
b56ccce
echo output
BBBmau Feb 22, 2024
48e22aa
github_output error fix
BBBmau Feb 22, 2024
4c16b09
services.outputs
BBBmau Feb 22, 2024
d78d651
proper if statement syntax gpush
BBBmau Feb 22, 2024
1a32160
invalid 0 format
BBBmau Feb 22, 2024
3fcb1ba
add missing assignment
BBBmau Feb 22, 2024
1266bc1
fix openfile error
BBBmau Feb 22, 2024
8a54067
remove types in pull_request trigger
BBBmau Feb 22, 2024
a65db8c
send txt files to correct directory
BBBmau Feb 22, 2024
f1eeb17
check services directory for ls command
BBBmau Feb 22, 2024
98176eb
add product.yaml
BBBmau Feb 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions .github/workflows/teamcity-services-diff-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
name: TeamCity Services Diff Check
permissions: read-all

on:
workflow_dispatch:
pull_request:
paths:
- '.github/workflows/teamcity-services-diff-check.yml'
- 'mmv1/third_party/terraform/.teamcity/components/inputs/services_ga.kt'
- 'mmv1/third_party/terraform/.teamcity/components/inputs/services_beta.kt'
- 'mmv1/products/**'
jobs:
check-pr:
runs-on: ubuntu-22.04
outputs:
services: ${{steps.services.outputs.services}}
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- id: services
run: |
newServices=$(($(git diff --name-only --diff-filter=A origin/main HEAD | grep -P "mmv1/products/.*/product.yaml" | wc -l)))
echo "services=$newServices" >> "${GITHUB_OUTPUT}"
if [ "$newServices" = "0" ];then
echo "No new service found."
fi
terraform-provider-google:
if: ${{needs.check-pr.outputs.services != '0'}}
needs: check-pr
uses: ./.github/workflows/build-downstream.yml
with:
repo: 'terraform-provider-google'

terraform-provider-google-beta:
if: ${{needs.check-pr.outputs.services != '0'}}
needs: check-pr
uses: ./.github/workflows/build-downstream.yml
with:
repo: 'terraform-provider-google-beta'

teamcity-services-diff-check:
needs: [terraform-provider-google, terraform-provider-google-beta]
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3

- name: Download built artifacts - GA provider
uses: actions/download-artifact@v2
with:
name: artifact-terraform-provider-google
path: artifacts

- name: Unzip the artifacts and delete the zip
run: |
unzip -o artifacts/output.zip -d ./provider
rm artifacts/output.zip

- name: Download built artifacts - Beta provider
uses: actions/download-artifact@v2
with:
name: artifact-terraform-provider-google-beta
path: artifacts

- name: Unzip the artifacts and delete the zip
run: |
unzip -o artifacts/output.zip -d ./provider
rm artifacts/output.zip

- name: Setup Go
uses: actions/setup-go@v3
with:
go-version: '^1.20'

- name: Cache Go modules and build cache
uses: actions/cache@v3
with:
path: |
~/go/pkg/mod
~/.cache/go-build
key: ${{ runner.os }}-test-terraform-provider-google-${{hashFiles('go.sum','google-*/transport/**','google-*/tpgresource/**','google-*/acctest/**','google-*/envvar/**','google-*/sweeper/**','google-*/verify/**') }}
restore-keys: |
${{ runner.os }}-test-terraform-provider-google-${{ hashFiles('go.sum') }}
${{ runner.os }}-test-terraform-provider-google-

- name: Diff Check
run: |
ls provider/google/services > tools/teamcity-diff-check/services_ga.txt
ls provider/google-beta/services > tools/teamcity-diff-check/services_beta.txt
cd tools/teamcity-diff-check
go run main.go -service_file=services_ga
go run main.go -service_file=services_beta

22 changes: 22 additions & 0 deletions mmv1/products/littlequery/product.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Copyright 2022 Google Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

--- !ruby/object:Api::Product
name: LittleQuery

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😂 love the name

display_name: Little Query
versions:
- !ruby/object:Api::Product::Version
name: beta
base_url: https://workstations.googleapis.com/v1beta/
scopes:
- https://www.googleapis.com/auth/cloud-platform
92 changes: 92 additions & 0 deletions tools/teamcity-diff-check/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
package main

import (
"bufio"
"flag"
"fmt"
"io"
"os"
"regexp"
)

var serviceFile = flag.String("service_file", "services_ga", "kotlin service file to be parsed")

func serviceDifference(gS, tS []string) []string {
t := make(map[string]struct{}, len(tS))
for _, s := range tS {
t[s] = struct{}{}
}

var diff []string
for _, s := range gS {
if _, found := t[s]; !found {
diff = append(diff, s)
}
}

return diff
}

func main() {
flag.Parse()

file, err := os.Open(*serviceFile + ".txt")
if err != nil {
fmt.Println(err)
return
}
defer file.Close()

googleServices := []string{}
scanner := bufio.NewScanner(file)
for scanner.Scan() {
googleServices = append(googleServices, scanner.Text())
}

////////////////////////////////////////////////////////////////////////////////

f, err := os.Open(fmt.Sprintf("../../mmv1/third_party/terraform/.teamcity/components/inputs/%s", *serviceFile+".kt"))
if err != nil {
panic(err)
}

// Get the file size
stat, err := f.Stat()
if err != nil {
fmt.Println(err)
return
}

// Read the file into a byte slice
bs := make([]byte, stat.Size())
_, err = bufio.NewReader(f).Read(bs)
if err != nil && err != io.EOF {
fmt.Println(err)
return
}

// Regex pattern captures "services" from *serviceFile.
pattern := regexp.MustCompile(`(?m)"(?P<service>\w+)"\sto\s+mapOf`)

template := []byte("$service")

dst := []byte{}
teamcityServices := []string{}

// For each match of the regex in the content.
for _, submatches := range pattern.FindAllSubmatchIndex(bs, -1) {
service := pattern.Expand(dst, template, bs, submatches)
teamcityServices = append(teamcityServices, string(service))
}
if len(teamcityServices) == 0 {
fmt.Fprintf(os.Stderr, "teamcityServices error: regex produced no matches.\n")
os.Exit(1)
}

if diff := serviceDifference(googleServices, teamcityServices); len(diff) != 0 {
fmt.Fprintf(os.Stderr, "error: diff in %s\n", *serviceFile)
fmt.Fprintf(os.Stderr, "Missing Services: %s\n", diff)
os.Exit(1)
}

}
Loading