Skip to content

Commit 5e0f125

Browse files
author
Yao Adzaku
committed
initial commit
0 parents  commit 5e0f125

17 files changed

+961
-0
lines changed

‎.gitignore

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
/.GOPATH
2+
/bin
3+
vendor/
4+
vendor/
5+
NOTES

‎Dockerfile

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
FROM golang:latest
2+
3+
RUN mkdir -p /go/src/github.com/rpip/dba
4+
WORKDIR /go/src/github.com/rpip/dba
5+
ADD . /go/src/github.com/rpip/dba
6+
7+
RUN go get github.com/Masterminds/glide github.com/k0kubun/pp
8+
RUN make deps && make build
9+
10+
ENTRYPOINT ["./bin/dba"]
11+
12+
CMD ["--help"]

‎HACKING.md

+112
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
## HACKING DBA
2+
3+
Below is a sample DBA config file.
4+
5+
```hcl
6+
db "library" {
7+
type = "mysql"
8+
dsn = "dba:dba123@(:3306)/dbatest?charset=utf8&parseTime=True&loc=Local"
9+
verbose = true
10+
11+
table "user" {
12+
// defaults to 'id'
13+
primary_key = "user_id"
14+
15+
updates {
16+
first_name = "${first_name()}"
17+
last_name = "${first_name()}"
18+
username = "${username()}"
19+
bio = "${paragraph()}"
20+
age = "${digits_n(2)}"
21+
gender = "${gender_abbrev()}"
22+
}
23+
}
24+
25+
table "product" {
26+
27+
updates {
28+
name = "${product()}"
29+
price = "${digits_n(5)}"
30+
merchant = "${company()}"
31+
brand = "${brand()}"
32+
}
33+
34+
}
35+
36+
table "blog" {
37+
title = "${title()}"
38+
body = "${paragraphs_n(5)}"
39+
published = true
40+
}
41+
}
42+
43+
```
44+
45+
DBA parses the config above into an internal config format. Below is dump of the Go data structure.
46+
47+
```go
48+
dba.Config{
49+
Databases: []*dba.Database{
50+
&dba.Database{
51+
meta: dba.meta{
52+
Name: "library",
53+
Scope: "database",
54+
Metadata: {
55+
"dsn": "dba:dba123@(:3306)/dbatest?charset=utf8&parseTime=True&loc=Local",
56+
"verbose": true,
57+
"type": "mysql",
58+
},
59+
},
60+
Tables: []*dba.Table{
61+
&dba.Table{
62+
meta: dba.meta{
63+
Name: "user",
64+
Scope: "table",
65+
Metadata: {
66+
"primary_key": "user_id",
67+
},
68+
},
69+
Updates: {
70+
"bio": "${paragraph()}",
71+
"age": "${digits_n(2)}",
72+
"gender": "${gender_abbrev()}",
73+
"first_name": "${first_name()}",
74+
"last_name": "${first_name()}",
75+
"username": "${username()}",
76+
},
77+
},
78+
&dba.Table{
79+
meta: dba.meta{
80+
Name: "product",
81+
Scope: "table",
82+
Metadata: map[string]interface {}{},
83+
},
84+
Updates: {
85+
"name": "${product()}",
86+
"price": "${digits_n(3)}",
87+
"merchant": "${company()}",
88+
"brand": "${brand()}",
89+
},
90+
},
91+
&dba.Table{
92+
meta: dba.meta{
93+
Name: "blog",
94+
Scope: "table",
95+
Metadata: {
96+
"title": "${title()}",
97+
"body": "${paragraphs_n(5)}",
98+
"published": true,
99+
},
100+
},
101+
Updates: map[string]interface {}{},
102+
},
103+
},
104+
DB: (*sql.DB)(nil),
105+
},
106+
},
107+
}
108+
```
109+
110+
Each table implements the `Anonymizer` interface. As we iterate through the tables in the config, we call the the `Anonymize` function on each table object and it constructs an UPDATE SQL query string from the updates declared in the config. This SQL query is then run against the delcared DB connection. That simple.
111+
112+
The lib folder contains the core of DBA. The goal is to enable usage of DBA as a library within other applications. The `main.go` script in the root folder contains the generated command line tool.

‎LICENSE

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
The MIT License (MIT)
2+
3+
Copyright (c) [2017] [Yao Adzaku]
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎Makefile

+51
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Porject name
2+
PROJECT = dba
3+
# Set an output prefix, which is the local directory if not specified
4+
PREFIX?=$(shell pwd)
5+
BUILDTAGS=
6+
GLIDE = $(shell which glide)
7+
8+
.PHONY: clean all fmt vet lint build test install static deps docker
9+
.DEFAULT: default
10+
11+
all: clean build fmt lint test vet install
12+
13+
build:
14+
@echo "+ $@"
15+
@go build -tags "$(BUILDTAGS) cgo" -o bin/$(PROJECT) .
16+
@echo "DBA saved in ./bin/dba"
17+
18+
static:
19+
@echo "+ $@"
20+
CGO_ENABLED=1 go build -tags "$(BUILDTAGS) cgo static_build" -ldflags "-w -extldflags -static" -o reg .
21+
22+
fmt:
23+
@echo "+ $@"
24+
@gofmt -s -l . | grep -v vendor | tee /dev/stderr
25+
26+
lint:
27+
@echo "+ $@"
28+
@golint ./... | grep -v vendor | tee /dev/stderr
29+
30+
test: fmt lint vet
31+
@echo "+ $@"
32+
@go test -v -tags "$(BUILDTAGS) cgo" $(shell go list ./... | grep -v vendor)
33+
34+
vet:
35+
@echo "+ $@"
36+
@go vet $(shell go list ./... | grep -v vendor)
37+
38+
clean:
39+
@echo "+ $@"
40+
@rm -rf reg bin
41+
42+
install:
43+
@echo "+ $@"
44+
@go install .
45+
46+
deps:
47+
@echo "Installing dependencies..."
48+
@$(GLIDE) install
49+
50+
docker:
51+
@docker build . -t $(PROJECT)

‎README.md

+94
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# DBA - Database Anonymizer
2+
3+
Tool for anonymizing database records
4+
5+
*work in progress*
6+
7+
## Installation
8+
9+
For now, you need Go and the [glide](https://github.com/Masterminds/glide) build to install dba. Downloadable binaries will be available soon.
10+
11+
``` shell
12+
λ git clone github.com/rpip/dba
13+
λ cd dba && make deps && make
14+
λ ./bin/dba --help
15+
usage: dba [<flags>] <conf>
16+
17+
Anonymize database records.
18+
19+
Flags:
20+
-h, --help Show context-sensitive help (also try --help-long and --help-man).
21+
-v, --verbose Verbose mode.
22+
--version Show application version.
23+
24+
Args:
25+
<conf> Config file.
26+
27+
```
28+
29+
## Usage
30+
31+
DBA uses the HCL config language from HashiCorp to describe database parameters and table updates. It's the same config language used in products like Terraform. It's quite flexible, easy to read and as such makes the config seem like a little DSL. It supports basic data types and ternary operations. You can [read more here](https://www.terraform.io/docs/configuration/index.html).
32+
33+
``` hcl
34+
db "library" {
35+
type = "mysql"
36+
dsn = "dba:dba123@(:3306)/dbatest?charset=utf8&parseTime=True&loc=Local"
37+
verbose = true
38+
39+
table "user" {
40+
// defaults to 'id'
41+
primary_key = "user_id"
42+
43+
updates {
44+
first_name = "${first_name()}"
45+
last_name = "${first_name()}"
46+
username = "${username()}"
47+
bio = "${paragraph()}"
48+
age = "${digits_n(2) * 2}"
49+
gender = "${gender_abbrev()}"
50+
role = "${row.role == '0' ? 8 : 30}"
51+
}
52+
}
53+
54+
table "product" {
55+
56+
updates {
57+
name = "${product()}"
58+
price = "${digits_n(5)}"
59+
merchant = "${company()}"
60+
brand = "${brand()}"
61+
}
62+
63+
}
64+
}
65+
```
66+
67+
```shell
68+
λ ./bin/dba sample.conf.hcl
69+
```
70+
71+
## TODO
72+
- [] test suite
73+
- [] more fake data generators, eg: date, time
74+
- [] support sqlite, Postgres
75+
- [] atomic operation:
76+
```hcl
77+
atomic {
78+
field = value
79+
field = value
80+
}```
81+
- [] Progress indictaor
82+
- [] Batch updates: build a long query string to run all table updates in one go
83+
- [] documentation
84+
85+
## Development
86+
87+
Pull requests are welcome. Please make sure the build succeeds and the test suite passes. Also see `HACKING.md` for info on the internals of dba.
88+
89+
You can also build a Docker image to test this:
90+
91+
```shell
92+
λ make docker
93+
λ docker run -i -t dba sample.conf.hcl
94+
```

‎glide.lock

+49
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎glide.yaml

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
package: github.com/rpip/dba
2+
import:
3+
- package: gopkg.in/alecthomas/kingpin.v2
4+
version: ^2.2.3
5+
- package: github.com/alecthomas/template
6+
- package: github.com/hashicorp/hcl
7+
- package: github.com/go-sql-driver/mysql
8+
version: ^1.3.0
9+
- package: github.com/hashicorp/hil
10+
- package: github.com/mitchellh/mapstructure
11+
- package: github.com/icrowley/fake
12+
- package: github.com/corpix/uarand

0 commit comments

Comments
 (0)