Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Initial implementation #1

Merged
merged 41 commits into from
Nov 21, 2024
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
235ceb1
feat: Initial implementation
jachym-tousek-keboola Oct 29, 2024
b508a58
build: Copy provisioning from object encryptor
jachym-tousek-keboola Nov 1, 2024
bfc0b65
build: Change cloud IDs
jachym-tousek-keboola Nov 1, 2024
21d23dc
build: Rename identifiers
jachym-tousek-keboola Nov 5, 2024
17cef60
fix: Minor issues
jachym-tousek-keboola Nov 5, 2024
d926b7c
test: Add GCP test
jachym-tousek-keboola Nov 7, 2024
3e34422
test: Add AWS test
jachym-tousek-keboola Nov 7, 2024
26ee0fd
fix: Cache encryptor
jachym-tousek-keboola Nov 7, 2024
8b9a03a
test: Add Azure test
jachym-tousek-keboola Nov 7, 2024
22a2345
refactor: Simplify scripts
jachym-tousek-keboola Nov 7, 2024
b0a0822
refactor: AzureEncryptor
jachym-tousek-keboola Nov 7, 2024
ced6ed6
build: Setup Azure key using terraform
jachym-tousek-keboola Nov 7, 2024
2792b72
fix: AzureEncryptor issues
jachym-tousek-keboola Nov 7, 2024
a4be920
build: Change cloud IDs
jachym-tousek-keboola Nov 8, 2024
345b8e1
build: Setup Azure key using terraform
jachym-tousek-keboola Nov 8, 2024
7692f97
build: Implement CI workflow
jachym-tousek-keboola Nov 8, 2024
76185cf
style: Rename variables
jachym-tousek-keboola Nov 12, 2024
3b19a16
build: Enable linters
jachym-tousek-keboola Nov 12, 2024
5307d68
build: Use pkg directory
jachym-tousek-keboola Nov 12, 2024
f4dc217
perf: Remove type casting
jachym-tousek-keboola Nov 12, 2024
aeea8c8
style: Rename test functions
jachym-tousek-keboola Nov 18, 2024
cf0482a
refactor: Extract functions
jachym-tousek-keboola Nov 18, 2024
24af764
refactor: Rename classes
jachym-tousek-keboola Nov 18, 2024
807da0e
refactor: Remove env variables prefix
jachym-tousek-keboola Nov 18, 2024
3beaab4
refactor: Simplify metadata
jachym-tousek-keboola Nov 18, 2024
045dc49
docs: Minor fixes
jachym-tousek-keboola Nov 18, 2024
0722f4b
style: Trim whitespace
jachym-tousek-keboola Nov 18, 2024
cf8fa70
build: Use aws-actions/configure-aws-credentials
jachym-tousek-keboola Nov 18, 2024
5627c5e
build: Use AWS session token
Halama Nov 18, 2024
407f88f
Merge pull request #2 from keboola/martin-aws-credentials
jachym-tousek-keboola Nov 19, 2024
9f85d23
fix: Incorrect paths
jachym-tousek-keboola Nov 19, 2024
bcb2cf2
refactor: Move code to internal packages
jachym-tousek-keboola Nov 19, 2024
c5a6b56
build: Use OIDC for GCP
jachym-tousek-keboola Nov 19, 2024
8b9a63c
refactor: Move internal packages
jachym-tousek-keboola Nov 19, 2024
359a031
build: Update terraform providers for ci
jachym-tousek-keboola Nov 19, 2024
8d52e4f
build: Update terraform providers for local
jachym-tousek-keboola Nov 20, 2024
5928d6b
build: Simplify GCP OIDC
Halama Nov 20, 2024
d6fb63d
Merge pull request #4 from keboola/martin-gcp-oidc
jachym-tousek-keboola Nov 20, 2024
5b07c93
fix: Makefile uses `TEST_ARGS`
Matovidlo Nov 19, 2024
a72591f
Merge pull request #3 from keboola/mv-makefile-adjustements
jachym-tousek-keboola Nov 20, 2024
0d7d4fc
docs: Add readme file
jachym-tousek-keboola Nov 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
root = true

[*]
end_of_line = lf
insert_final_newline = true

[{*.go,Makefile}]
indent_style = tab

[*.{yml,yaml,tf,sh}]
indent_style = space
indent_size = 2
trim_trailing_whitespace = true
38 changes: 38 additions & 0 deletions .github/workflows/push.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: GitHub Actions
on: [ push ]
concurrency: ci

env:
AZURE_TENANT_ID: 9b85ee6f-4fb0-4a46-8cb7-4dcc6b262a89
AZURE_CLIENT_ID: 018b2e1a-41f3-48cf-a3b6-dd93f74c6d2f
AZURE_CLIENT_SECRET: ${{ secrets.TEST_AZURE_CLIENT_SECRET }}
AZURE_KEY_VAULT_URL: https://ci-go-cloud-encrypt.vault.azure.net/
AZURE_KEY_NAME: ci-go-cloud-encrypt
AWS_REGION: eu-central-1
AWS_ACCESS_KEY_ID: AKIA4DHGP53NXOCD6Y6B
AWS_SECRET_ACCESS_KEY: ${{ secrets.TEST_AWS_SECRET_ACCESS_KEY }}
AWS_KMS_KEY_ID: f14ac86a-dc61-4544-b1f7-6312773765f0
AWS_ROLE_ID: arn:aws:iam::831559560923:role/ci-go-cloud-encrypt-role
GCP_KMS_KEY_ID: projects/go-team-ci/locations/global/keyRings/ci-go-cloud-encrypt/cryptoKeys/ci-go-cloud-encrypt
jobs:
tests:
runs-on: ubuntu-latest
steps:
- name: Check out the repo
uses: actions/checkout@v4

- name: Google Login
uses: google-github-actions/auth@v2
with:
credentials_json: ${{ secrets.TEST_GCP_SERVICE_ACCOUNT_KEY }}
export_environment_variables: true
create_credentials_file: true

- name: Build image
run: |
docker login --username "$DOCKERHUB_USER" --password "$DOCKERHUB_TOKEN"
docker compose -f docker-compose.ci.yml build

- name: Run tests
run: |
docker compose -f docker-compose.ci.yml run ci go test ./...
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
!.gitkeep

var/*
vendor/*

.env.local
30 changes: 30 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
FROM golang:1.23

ENV HOME=/my-home
ENV GOCACHE=/tmp/cache/go
ENV GOMODCACHE=/tmp/cache/go-mod
ENV GOFLAGS="-mod=mod"
ENV PATH="$PATH:$GOPATH/bin"

# Install editor
RUN apt-get update && apt-get install -y nano
ENV EDITOR=nano

# Install tools
RUN mkdir -p /tmp/build
COPY Makefile /tmp/build/Makefile
COPY scripts /tmp/build/scripts
RUN cd /tmp/build && make tools && rm -rf /tmp/build

# Set prompt
RUN mkdir -p ~ && \
echo 'PS1="\w > "' > ~/.bashrc

# Fix permissions
RUN mkdir -p $GOPATH && chmod -R 777 $GOPATH && \
mkdir -p $GOCACHE && chmod -R 777 $GOCACHE && \
mkdir -p $GOMODCACHE && chmod -R 777 $GOMODCACHE && \
mkdir -p $HOME && chmod -R 777 $HOME

WORKDIR /code/
CMD ["/bin/bash"]
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) Keboola :(){:|:&};: s.r.o.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
14 changes: 14 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
tools:
bash ./scripts/tools.sh

lint:
bash ./scripts/lint.sh

fix:
bash ./scripts/fix.sh

tests:
gotestsum --no-color=false --format testname -- -timeout 600s -p 8 -parallel 8 -v -race -coverprofile=/tmp/profile.out ./pkg/...
jachym-tousek-keboola marked this conversation as resolved.
Show resolved Hide resolved

godoc:
godoc -http=0.0.0.0:6060
135 changes: 135 additions & 0 deletions build/ci/golangci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
run:
timeout: 15m
concurrency: 8
max-issues-per-linter: 0
# Maximum count of issues with the same text.
max-same-issues: 0
tests: true # check test files
modules-download-mode: mod

output:
sort-results: true

linters-settings:
# Prevent updating goa generated code due to linter update
misspell:
ignore-words:
- Statuser
# Errors from the project can be returned without wrapping
wrapcheck:
ignorePackageGlobs:
- github.com/keboola/go-cloud-encrypt/*
# Gci - improved version of goimports
gci:
sections:
- standard # Captures all standard packages if they do not match another section.
- default # Contains all imports that could not be matched to another section type.
- prefix(github.com/keboola/go-cloud-encrypt)

gocyclo:
# minimal code complexity to report, 30 by default (but we recommend 10-20)
min-complexity: 10

dupl:
threshold: 500

# Forbidden constructions
forbidigo:
# Instead of matching the literal source code, use real package name.
analyze-types: true
forbid:
# No debug statements
- p: ^(fmt\.Print.*|print|println)$
msg: Debug statements are forbidden, use a logger, not debug statements.

stylecheck:
checks:
- all

exhaustive:
check-generated: true
default-signifies-exhaustive: true

# https://golangci-lint.run/usage/linters
linters:
disable-all: true
enable:
- asciicheck
- bodyclose
- contextcheck
- dogsled
- dupl
- durationcheck
- errcheck
- errname
- errorlint
- exhaustive
- copyloopvar
- forbidigo
- gci
- gochecknoglobals
- gochecknoinits
- goconst
- gocritic
- godot
- godox
- gofumpt
- goheader
- gomodguard
- goprintffuncname
- gosec
- gosimple
- govet
- importas
- ineffassign
- makezero
- nakedret
- nilerr
- noctx
- predeclared
- promlinter
- rowserrcheck
- sqlclosecheck
- staticcheck
- stylecheck
- tagliatelle
- thelper
- tparallel
- paralleltest
- unconvert
- unparam
- unused
- wastedassign
- whitespace
# DISABLED
#- goimports # replaced with gci
#- gofmt # replaced with gofumpt
#- nolintlint # strange behavior
#- gomoddirectives # allow replace directive in go.mod
#- misspell - broken, rewrites code
# TODO
- funlen
#- gocyclo
- gocognit
- cyclop
- nestif
#- lll
#- gomnd

issues:
max-same-issues: 25

# Excluding configuration per-path, per-linter, per-text and per-source
exclude-rules:
# Exclude some linters from running on tests files.
- path: _test\.go
linters:
- gocyclo
- goconst
- gosec
- gochecknoglobals
- errcheck
- errorlint
- dupl
- dogsled
- bodyclose
19 changes: 19 additions & 0 deletions docker-compose.ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
services:
ci:
build: .
environment:
AZURE_TENANT_ID:
AZURE_CLIENT_ID:
AZURE_CLIENT_SECRET:
AZURE_KEY_VAULT_URL:
AZURE_KEY_NAME:
AWS_REGION:
AWS_ACCESS_KEY_ID:
AWS_SECRET_ACCESS_KEY:
AWS_KMS_KEY_ID:
AWS_ROLE_ID:
GCP_KMS_KEY_ID:
GOOGLE_APPLICATION_CREDENTIALS: /code/var/gcp-private-key.json
volumes:
- ./:/code:z
- $GOOGLE_APPLICATION_CREDENTIALS:/code/var/gcp-private-key.json
10 changes: 10 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
services:
dev:
build: .
volumes:
- ./:/code:z
- cache:/tmp/cache
Matovidlo marked this conversation as resolved.
Show resolved Hide resolved
env_file: .env.local

volumes:
cache:
75 changes: 75 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
module github.com/keboola/go-cloud-encrypt

go 1.23.2

require (
cloud.google.com/go/kms v1.20.0
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.0
github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azkeys v1.3.0
github.com/aws/aws-sdk-go-v2/config v1.28.1
github.com/aws/aws-sdk-go-v2/service/kms v1.37.2
github.com/dgraph-io/ristretto/v2 v2.0.0-alpha
github.com/keboola/go-utils v1.2.0
github.com/pkg/errors v0.9.1
github.com/stretchr/testify v1.9.0
)

require (
cloud.google.com/go v0.115.1 // indirect
cloud.google.com/go/auth v0.9.3 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.4 // indirect
cloud.google.com/go/compute/metadata v0.5.0 // indirect
cloud.google.com/go/iam v1.2.1 // indirect
cloud.google.com/go/longrunning v0.6.1 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.16.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/internal v1.1.0 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.3.1 // indirect
github.com/aws/aws-sdk-go-v2 v1.32.3 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.17.42 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.18 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.22 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.22 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.3 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.24.3 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.3 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.32.3 // indirect
github.com/aws/smithy-go v1.22.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/golang-jwt/jwt/v5 v5.2.1 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/google/s2a-go v0.1.8 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.4 // indirect
github.com/googleapis/gax-go/v2 v2.13.0 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.54.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.54.0 // indirect
go.opentelemetry.io/otel v1.29.0 // indirect
go.opentelemetry.io/otel/metric v1.29.0 // indirect
go.opentelemetry.io/otel/trace v1.29.0 // indirect
golang.org/x/crypto v0.28.0 // indirect
golang.org/x/net v0.30.0 // indirect
golang.org/x/oauth2 v0.23.0 // indirect
golang.org/x/sync v0.8.0 // indirect
golang.org/x/sys v0.26.0 // indirect
golang.org/x/text v0.19.0 // indirect
golang.org/x/time v0.6.0 // indirect
google.golang.org/api v0.197.0 // indirect
google.golang.org/genproto v0.0.0-20240903143218-8af14fe29dc1 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240903143218-8af14fe29dc1 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 // indirect
google.golang.org/grpc v1.66.2 // indirect
google.golang.org/protobuf v1.34.2 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
Loading