Skip to content
This repository has been archived by the owner on Aug 31, 2022. It is now read-only.

[multi-DB] Part 5: Golang API changes and replacement #24

Merged
merged 7 commits into from
Nov 22, 2019
Merged

[multi-DB] Part 5: Golang API changes and replacement #24

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
824ee76
[multi-DB] Part 5: Golang API changes and replacement
dzhangalibaba Nov 8, 2019
1051d14
try sudo
dzhangalibaba Nov 8, 2019
84ee240
fix build issue which not using committed codes
dzhangalibaba Nov 8, 2019
113256c
add GetSeparator and dynamiclly update Target_value/name
dzhangalibaba Nov 11, 2019
049e6ec
fix typo
dzhangalibaba Nov 11, 2019
d47d1a7
using GetDbId()
dzhangalibaba Nov 12, 2019
b0fb1c7
using go modules to manage local pkgs
dzhangalibaba Nov 13, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,20 @@ export GOPATH=/tmp/go
endif

INSTALL := /usr/bin/install
DBDIR := /var/run/redis/sonic-db/

all: sonic-telemetry

sonic-telemetry:
Copy link
Contributor

@qiluo-msft qiluo-msft Nov 12, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sonic-telemetry [](start = 0, length = 15)

I am not familiar with golang.
Seems this is a build target, why not use go build directly, instead of go get. Then we don't need mkdir and cp #Closed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go build still require ${GOPATH}/src as source codes workspace, we still need mkdir and cp

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that you can run go build without a target, it just compile *.go in current directory.

In reply to: 344968232 [](ancestors = 344968232)

Copy link
Contributor Author

@dzhangalibaba dzhangalibaba Nov 13, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did some search online, looks after go version 1.11, go modules are introduced to solve the $GOPATH problem. We can use go mod init and go install to remove the dependency on GOPATH. Also looks our go version in docker is 1.11.5 which is updated this July. Then we are able to build using local source files without cp. Makefile is updated.

/usr/local/go/bin/go get -v github.com/Azure/sonic-telemetry/telemetry
/usr/local/go/bin/go get -v github.com/Azure/sonic-telemetry/dialout/dialout_client_cli
/usr/local/go/bin/go mod init github.com/Azure/sonic-telemetry
/usr/local/go/bin/go install github.com/Azure/sonic-telemetry/telemetry
/usr/local/go/bin/go install github.com/Azure/sonic-telemetry/dialout/dialout_client_cli

check:
/usr/local/go/bin/go get -v -t github.com/Azure/sonic-telemetry/gnmi_server/...
/usr/local/go/bin/go test -v ${GOPATH}/src/github.com/Azure/sonic-telemetry/gnmi_server
sudo mkdir -p ${DBDIR}
sudo cp ./testdata/database_config.json ${DBDIR}
/usr/local/go/bin/go test -v github.com/Azure/sonic-telemetry/gnmi_server
/usr/local/go/bin/go test -v github.com/Azure/sonic-telemetry/dialout/dialout_client

install:
$(INSTALL) -D ${GOPATH}/bin/telemetry $(DESTDIR)/usr/sbin/telemetry
Expand All @@ -24,4 +28,6 @@ deinstall:

clean:
rm -fr ${GOPATH}
rm -rf go.mod
rm -rf go.sum

11 changes: 6 additions & 5 deletions dialout/dialout_client/dialout_client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"fmt"
spb "github.com/Azure/sonic-telemetry/proto"
sdc "github.com/Azure/sonic-telemetry/sonic_data_client"
sdcfg "github.com/Azure/sonic-telemetry/sonic_db_config"
"github.com/go-redis/redis"
log "github.com/golang/glog"
gpb "github.com/openconfig/gnmi/proto/gnmi"
Expand Down Expand Up @@ -641,23 +642,23 @@ func processTelemetryClientConfig(ctx context.Context, redisDb *redis.Client, ke
// read configDB data for telemetry client and start publishing service for client subscription
func DialOutRun(ctx context.Context, ccfg *ClientConfig) error {
clientCfg = ccfg
dbn := spb.Target_value["CONFIG_DB"]
dbn := sdcfg.GetDbId("CONFIG_DB")

var redisDb *redis.Client
if sdc.UseRedisLocalTcpPort == false {
redisDb = redis.NewClient(&redis.Options{
Network: "unix",
Addr: sdc.Default_REDIS_UNIXSOCKET,
Addr: sdcfg.GetDbSock("CONFIG_DB"),
Password: "", // no password set
DB: int(dbn),
DB: dbn,
DialTimeout: 0,
})
} else {
redisDb = redis.NewClient(&redis.Options{
Network: "tcp",
Addr: sdc.Default_REDIS_LOCAL_TCP_PORT,
Addr: sdcfg.GetDbTcpAddr("CONFIG_DB"),
Password: "", // no password set
DB: int(dbn),
DB: dbn,
DialTimeout: 0,
})
}
Expand Down
16 changes: 7 additions & 9 deletions dialout/dialout_client/dialout_client_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ import (
"time"

sds "github.com/Azure/sonic-telemetry/dialout/dialout_server"
spb "github.com/Azure/sonic-telemetry/proto"
sdc "github.com/Azure/sonic-telemetry/sonic_data_client"
sdcfg "github.com/Azure/sonic-telemetry/sonic_db_config"
gclient "github.com/openconfig/gnmi/client/gnmi"
)

Expand Down Expand Up @@ -95,12 +95,11 @@ func runServer(t *testing.T, s *sds.Server) {
}

func getRedisClient(t *testing.T) *redis.Client {
dbn := spb.Target_value["COUNTERS_DB"]
rclient := redis.NewClient(&redis.Options{
Network: "tcp",
Addr: "localhost:6379",
Addr: sdcfg.GetDbTcpAddr("COUNTERS_DB"),
Password: "", // no password set
DB: int(dbn),
DB: sdcfg.GetDbId("COUNTERS_DB"),
DialTimeout: 0,
})
_, err := rclient.Ping().Result()
Expand All @@ -125,12 +124,11 @@ func exe_cmd(t *testing.T, cmd string) {
}

func getConfigDbClient(t *testing.T) *redis.Client {
dbn := spb.Target_value["CONFIG_DB"]
rclient := redis.NewClient(&redis.Options{
Network: "tcp",
Addr: "localhost:6379",
Addr: sdcfg.GetDbTcpAddr("CONFIG_DB"),
Password: "", // no password set
DB: int(dbn),
DB: sdcfg.GetDbId("CONFIG_DB"),
DialTimeout: 0,
})
_, err := rclient.Ping().Result()
Expand All @@ -157,7 +155,7 @@ func loadConfigDB(t *testing.T, rclient *redis.Client, mpi map[string]interface{
func prepareConfigDb(t *testing.T) {
rclient := getConfigDbClient(t)
defer rclient.Close()
rclient.FlushDb()
rclient.FlushDB()

fileName := "../../testdata/COUNTERS_PORT_ALIAS_MAP.txt"
countersPortAliasMapByte, err := ioutil.ReadFile(fileName)
Expand All @@ -179,7 +177,7 @@ func prepareConfigDb(t *testing.T) {
func prepareDb(t *testing.T) {
rclient := getRedisClient(t)
defer rclient.Close()
rclient.FlushDb()
rclient.FlushDB()
//Enable keysapce notification
os.Setenv("PATH", "$PATH:/usr/bin:/sbin:/bin:/usr/local/bin:/usr/local/Cellar/redis/4.0.8/bin")
cmd := exec.Command("redis-cli", "config", "set", "notify-keyspace-events", "KEA")
Expand Down
16 changes: 7 additions & 9 deletions gnmi_server/server_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ import (
"testing"
"time"
// Register supported client types.
spb "github.com/Azure/sonic-telemetry/proto"
sdc "github.com/Azure/sonic-telemetry/sonic_data_client"
sdcfg "github.com/Azure/sonic-telemetry/sonic_db_config"
gclient "github.com/jipanyang/gnmi/client/gnmi"
)

Expand Down Expand Up @@ -155,12 +155,11 @@ func runServer(t *testing.T, s *Server) {
}

func getRedisClient(t *testing.T) *redis.Client {
dbn := spb.Target_value["COUNTERS_DB"]
rclient := redis.NewClient(&redis.Options{
Network: "tcp",
Addr: "localhost:6379",
Addr: sdcfg.GetDbTcpAddr("COUNTERS_DB"),
Password: "", // no password set
DB: int(dbn),
DB: sdcfg.GetDbId("COUNTERS_DB"),
DialTimeout: 0,
})
_, err := rclient.Ping().Result()
Expand All @@ -171,12 +170,11 @@ func getRedisClient(t *testing.T) *redis.Client {
}

func getConfigDbClient(t *testing.T) *redis.Client {
dbn := spb.Target_value["CONFIG_DB"]
rclient := redis.NewClient(&redis.Options{
Network: "tcp",
Addr: "localhost:6379",
Addr: sdcfg.GetDbTcpAddr("CONFIG_DB"),
Password: "", // no password set
DB: int(dbn),
DB: sdcfg.GetDbId("CONFIG_DB"),
DialTimeout: 0,
})
_, err := rclient.Ping().Result()
Expand All @@ -203,7 +201,7 @@ func loadConfigDB(t *testing.T, rclient *redis.Client, mpi map[string]interface{
func prepareConfigDb(t *testing.T) {
rclient := getConfigDbClient(t)
defer rclient.Close()
rclient.FlushDb()
rclient.FlushDB()

fileName := "../testdata/COUNTERS_PORT_ALIAS_MAP.txt"
countersPortAliasMapByte, err := ioutil.ReadFile(fileName)
Expand All @@ -225,7 +223,7 @@ func prepareConfigDb(t *testing.T) {
func prepareDb(t *testing.T) {
rclient := getRedisClient(t)
defer rclient.Close()
rclient.FlushDb()
rclient.FlushDB()
//Enable keysapce notification
os.Setenv("PATH", "/usr/bin:/sbin:/bin:/usr/local/bin")
cmd := exec.Command("redis-cli", "config", "set", "notify-keyspace-events", "KEA")
Expand Down
41 changes: 11 additions & 30 deletions proto/sonic.pb.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 4 additions & 13 deletions sonic_data_client/db_client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import (
log "github.com/golang/glog"

spb "github.com/Azure/sonic-telemetry/proto"
sdcfg "github.com/Azure/sonic-telemetry/sonic_db_config"
"github.com/go-redis/redis"
gnmipb "github.com/openconfig/gnmi/proto/gnmi"
"github.com/workiva/go-datastructures/queue"
Expand All @@ -24,8 +25,6 @@ const (
// indentString represents the default indentation string used for
// JSON. Two spaces are used here.
indentString string = " "
Default_REDIS_UNIXSOCKET string = "/var/run/redis/redis.sock"
Default_REDIS_LOCAL_TCP_PORT string = "localhost:6379"
)

// Client defines a set of methods which every client must implement.
Expand Down Expand Up @@ -304,15 +303,7 @@ func GetTableKeySeparator(target string) (string, error) {
return "", fmt.Errorf("%v not a valid path target", target)
}

var separator string
switch target {
case "CONFIG_DB":
separator = "|"
case "STATE_DB":
separator = "|"
default:
separator = ":"
}
var separator string = sdcfg.GetDbSeparator(target)
return separator, nil
}

Expand All @@ -325,7 +316,7 @@ func useRedisTcpClient() {
if UseRedisLocalTcpPort {
redisDb = redis.NewClient(&redis.Options{
Network: "tcp",
Addr: Default_REDIS_LOCAL_TCP_PORT,
Addr: sdcfg.GetDbTcpAddr(dbName),
Password: "", // no password set
DB: int(dbn),
DialTimeout: 0,
Expand All @@ -345,7 +336,7 @@ func init() {

redisDb = redis.NewClient(&redis.Options{
Network: "unix",
Addr: Default_REDIS_UNIXSOCKET,
Addr: sdcfg.GetDbSock(dbName),
Password: "", // no password set
DB: int(dbn),
DialTimeout: 0,
Expand Down
Loading