Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Dapr InvokeService and InvokeBinding API #362

Merged
merged 10 commits into from
Dec 14, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion cmd/layotto_multiple_api/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ package main
import (
"encoding/json"
"fmt"
"mosn.io/layotto/pkg/grpc/dapr"
"mosn.io/layotto/pkg/grpc/default_api"
helloworld_api "mosn.io/layotto/pkg/integrate/api/helloworld"
"os"
Expand Down Expand Up @@ -162,6 +163,10 @@ func NewRuntimeGrpcServer(data json.RawMessage, opts ...grpc.ServerOption) (mgrp
default_api.NewGrpcAPI,
// a demo to show how to register your own API
helloworld_api.NewHelloWorldAPI,
// support Dapr API
// Currently it only support Dapr's InvokeService and InvokeBinding API.
// Note: this feature is still in Alpha state and we don't recommend that you use it in your production environment.
dapr.NewDaprAPI_Alpha,
),
// Hello
runtime.WithHelloFactory(
Expand Down Expand Up @@ -228,7 +233,7 @@ func NewRuntimeGrpcServer(data json.RawMessage, opts ...grpc.ServerOption) (mgrp
// State
runtime.WithStateFactory(
runtime_state.NewFactory("in-memory", func() state.Store {
return mock_state.NewInMemoryStateStore()
return mock_state.New(loggerForDaprComp)
}),
runtime_state.NewFactory("redis", func() state.Store {
return state_redis.NewRedisStateStore(loggerForDaprComp)
Expand Down
3 changes: 3 additions & 0 deletions codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,7 @@ ignore:
# and https://docs.codecov.com/docs/unexpected-coverage-changes
- "pkg/filter/network/tcpcopy/persistence"
- "pkg/filter/network/tcpcopy/strategy"
- "pkg/grpc/dapr/proto/common/v1"
- "pkg/grpc/dapr/proto/runtime/v1"
- "pkg/grpc/dapr/dapr_api_unimplement.go"
- "pkg/wasm/watcher.go"
7 changes: 7 additions & 0 deletions components/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ github.com/apache/dubbo-go-hessian2 v1.7.0/go.mod h1:7rEw9guWABQa6Aqb8HeZcsYPHsO
github.com/apache/thrift v0.12.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
github.com/apache/thrift v0.13.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=
github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da h1:8GUt8eRujhVEGZFFEjBj46YV4rDjvGrNxb0KMWYkL2I=
github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY=
github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8=
github.com/aryann/difflib v0.0.0-20170710044230-e206f873d14a/go.mod h1:DAHtR1m6lCRdSC2Tm3DSWRPvIPr6xNKyeHdqDQSQT+A=
Expand Down Expand Up @@ -363,16 +364,20 @@ github.com/grpc-ecosystem/grpc-gateway v1.14.6/go.mod h1:zdiPV4Yse/1gnckTHtghG4G
github.com/grpc-ecosystem/grpc-gateway v1.16.0 h1:gmcG1KaJ57LophUzW0Hy8NmPhnMZb4M0+kPpLofRdBo=
github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
github.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q=
github.com/hashicorp/consul/api v1.3.0 h1:HXNYlRkkM/t+Y/Yhxtwcy02dlYwIaoxzvxPnS+cqy78=
github.com/hashicorp/consul/api v1.3.0/go.mod h1:MmDNSzIMUjNpY/mQ398R4bk2FnqQLoPndWW5VkKPlCE=
github.com/hashicorp/consul/sdk v0.1.1/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8=
github.com/hashicorp/consul/sdk v0.3.0/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-cleanhttp v0.5.1 h1:dH3aiDG9Jvb5r5+bYHsikaOUIpcM0xvgMXVoDkXMzJM=
github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
github.com/hashicorp/go-hclog v0.0.0-20180709165350-ff2cf002a8dd/go.mod h1:9bjs9uLqI8l75knNv3lV1kA55veR+WUPSiKIWcQHudI=
github.com/hashicorp/go-immutable-radix v1.0.0 h1:AKDB1HM5PWEA7i4nhcpwOrO2byshxBjXVn/J/3+z5/0=
github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60=
github.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM=
github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk=
github.com/hashicorp/go-plugin v1.0.1/go.mod h1:++UyYGoz3o5w9ZzAdZxtQKrWWP+iqPBn3cQptSMzBuY=
github.com/hashicorp/go-rootcerts v1.0.0 h1:Rqb66Oo1X/eSV1x66xbDccZjhJigjg0+e82kpwzSwCI=
github.com/hashicorp/go-rootcerts v1.0.0/go.mod h1:K6zTfqpRlCUIjkwsN4Z+hiSfzSTQa6eBIzfwKfwNnHU=
github.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU=
github.com/hashicorp/go-syslog v1.0.0 h1:KaodqZuhUoZereWVIYmpUgZysurB1kBLX2j0MwMrUAE=
Expand All @@ -382,12 +387,14 @@ github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/b
github.com/hashicorp/go-version v1.2.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1 h1:0hERBMJE1eitiLkihrMvRVBYAkpHzc/J3QdDN+dAcgU=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=
github.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ=
github.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I=
github.com/hashicorp/serf v0.8.2 h1:YZ7UKsJv+hKjqGVUUbtE3HNj79Eln2oQ75tniF6iPt0=
github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc=
github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb/go.mod h1:+NfK9FKeTrX5uv1uIXGdwYDTeHna2qgaIlx54MXqjAM=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
Expand Down
3 changes: 2 additions & 1 deletion components/hello/helloworld/helloworld_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
package helloworld

import (
"context"
"testing"

"mosn.io/layotto/components/hello"
Expand All @@ -32,7 +33,7 @@ func TestHelloWorld(t *testing.T) {
Name: "Layotto",
}

resp, _ := hs.Hello(req)
resp, _ := hs.Hello(context.Background(), req)
if resp.HelloString != "Hi, Layotto" {
t.Fatalf("hello output failed")
}
Expand Down
11 changes: 0 additions & 11 deletions components/lock/etcd/etcd_lock_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -242,14 +242,3 @@ func startEtcdServer(dir string, port int) (*embed.Etcd, error) {
<-e.Server.ReadyNotify()
return e, nil
}

func Test_addPathSeparator(t *testing.T) {
p := addPathSeparator("")
assert.Equal(t, p, "/")
p = addPathSeparator("l8")
assert.Equal(t, p, "/l8/")
p = addPathSeparator("/l8")
assert.Equal(t, p, "/l8/")
p = addPathSeparator("l8/")
assert.Equal(t, p, "/l8/")
}
33 changes: 33 additions & 0 deletions components/pkg/utils/etcd_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
/*
* Copyright 2021 Layotto Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package utils

import (
"github.com/stretchr/testify/assert"
"testing"
)

func Test_addPathSeparator(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is there no test logic related to etcd, as this file is actually etcd_test.go?

p := addPathSeparator("")
assert.Equal(t, p, "/")
p = addPathSeparator("l8")
assert.Equal(t, p, "/l8/")
p = addPathSeparator("/l8")
assert.Equal(t, p, "/l8/")
p = addPathSeparator("l8/")
assert.Equal(t, p, "/l8/")
}
1 change: 1 addition & 0 deletions docs/zh/_sidebar.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@
- [file API设计文档](zh/design/file/file-design.md)
- [FaaS 设计文档](zh/design/faas/faas-poc-design.md)
- [API插件](zh/design/api_plugin/design.md)
- [支持Dapr API](zh/design/api_plugin/dapr_api.md)
- 贡献指南
- [新手攻略:从零开始成为 Layotto 贡献者](zh/development/start-from-zero.md)
- [文档贡献指南](zh/development/contributing-doc.md)
Expand Down
40 changes: 40 additions & 0 deletions docs/zh/design/api_plugin/dapr_api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
## TLDR
本提案想让开源Layotto 既支持 Layotto API,又支持Dapr API。类似于“Minio 既支持Minio API,又支持AWS S3 API”

## 想解决的问题
1. 目前,我们尽量保证Layotto API 里各个字段定义的和Dapr一样,但用户真正关心的是两者sdk能否复用。虽然我们努力保证proto字段一致,但只要不能复用sdk就没解决用户的问题,还给自己增加维护成本。
比如:
![image](https://user-images.githubusercontent.com/26001097/145837477-00fc5cd8-32eb-4ce9-bbfb-6e590172fce8.png)

因此,我们想让Layotto直接支持Dapr的grpc API (一模一样,包括 package名),对于用户来说,他可以用Dapr sdk在两者之间自由切换,不用担心被厂商绑定。

2. 另一方面,还需要有一定扩展性。我们在生产落地的过程中发现目前的Dapr API没法完全满足需求,难免要对API做一些扩展。扩展的API已经加到了现在的 Layotto API里,提案提给了 Dapr 社区、但还在等社区慢慢接受,比如config API,比如Lock API

## 方案
### Layotto API on Dapr API
![image](https://user-images.githubusercontent.com/26001097/145838604-e3a0caad-9473-4092-a2c6-0cc46c972790.png)
1. Layotto 会启动一个grpc服务器,前阵子刚加了个API插件功能,我们可以通过 API插件的形式注册一个Dapr API 插件;
2. 另一方面,保留Layotto API。Layotto接收到Layotto API请求后,翻译成Dapr API,然后按Dapr API处理。
这样的好处有:
- 复用代码
- Layotto API 可以按生产需求进行扩展,比如支持Lock API,configuration API等;做了扩展后可以提给Dapr 社区,再慢慢讨论,即使最终讨论的结果和原始方案不一样,也只是影响最终做出来的Dapr API,不会影响已经用上Layotto API的用户。

### 用户价值
对于用户来说:

- 如果用户担心厂商绑定,可以只用Dapr API,可以用同一套Dapr sdk在Dapr 和Layotto 之间迁移,减少用户疑虑;
- 如果用户相信我们的落地经验、愿意用Layotto API,那他们可以用Layotto API,代价是没法用同一个sdk在两个sidecar之间迁移

### Q&A
#### 想给Dapr API加字段怎么加
##### 想加个字段(field)
比如想给layotto api加个abc字段,可以通过metadata或者grpc头把这个字段传给dapr API
dapr API的实现再把这个字段透传给组件,组件解析这个字段

##### 不只加字段,还要加一些逻辑、机制(mechanism)
比如layotto api加个abc字段,如果abc==true,那么runtime走一段特殊逻辑

这种情况要修改Dapr API的实现,加一段if else

#### 想加新API怎么加
加在layotto API上,新API不需要复用Dapr API;等Dapr接收提案后再修改实现,layotto API不变
4 changes: 2 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ go 1.14
require (
github.com/agrea/ptr v0.0.0-20180711073057-77a518d99b7b
github.com/alicebob/miniredis/v2 v2.13.3
github.com/dapr/components-contrib v1.4.0-rc2
github.com/dapr/components-contrib v1.5.1-rc.1
github.com/dapr/kit v0.0.2-0.20210614175626-b9074b64d233
github.com/fsnotify/fsnotify v1.4.9
github.com/gammazero/workerpool v1.1.2
Expand All @@ -14,12 +14,12 @@ require (
github.com/google/uuid v1.2.0
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0
github.com/json-iterator/go v1.1.11
github.com/phayes/freeport v0.0.0-20171002181615-b8543db493a5
github.com/pkg/errors v0.9.1
github.com/shirou/gopsutil v3.21.3+incompatible
github.com/stretchr/testify v1.7.0
github.com/urfave/cli v1.22.1
github.com/valyala/fasthttp v1.28.0
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519 // indirect
golang.org/x/net v0.0.0-20211005001312-d4b1ae081e3b // indirect
golang.org/x/text v0.3.7 // indirect
google.golang.org/grpc v1.39.0
Expand Down
Loading