Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(generator): generate less code and doc #823

Merged
merged 15 commits into from
Oct 28, 2022
10 changes: 5 additions & 5 deletions components/rpc/invoker/mosn/channel/httpchannel.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,11 +83,11 @@ func newHttpChannel(config ChannelConfig) (rpc.Channel, error) {
return nil, err
}
// the goroutine model is:
// request goroutine ---> localTcpConn ---> mosn
// ^ |
// | |
// | |
// hstate <-- readloop goroutine <------
// request goroutine ---> localTcpConn ---> mosn
// ^ |
// | |
// | |
// hstate(net.Pipe) <-- readloop goroutine <---
return localTcpConn, nil
},
// stateFunc
Expand Down
6 changes: 5 additions & 1 deletion docs/zh/blog/code/layotto-rpc/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,8 +148,12 @@ func (stm *StageManager) runStartStage() {

### 0x01 Dubbo-go-sample client 发起请求

根据 [Dubbo Json Rpc Example](https://mosn.io/layotto/#/zh/start/rpc/dubbo_json_rpc)例子运行如下命令
根据 [Dubbo Json Rpc Example](https://mosn.io/layotto/#/zh/start/rpc/dubbo_json_rpc) 例子运行如下命令

```shell
go run demo/rpc/dubbo_json_rpc/dubbo_json_client/client.go -d '{"jsonrpc":"2.0","method":"GetUser","params":["A003"],"id":9527}'
```

使用 Layotto 对 App 提供的 Grpc API InvokeService 发起 RPC 调用,经过数据填充和连接建立等流程,最终通过 Grpc clientStream 中调用 SendMsg 向 Layotto 发送数据,具体流程如下。

```go
Expand Down
8 changes: 4 additions & 4 deletions docs/zh/design/rpc/rpc设计文档.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
RPC 设计文档

### API
layotto rpc接口与Dapr保持一致.
### API 设计
[layotto rpc API](https://github.com/mosn/layotto/blob/f70cdc619693ad762cf809daf0579403c341def1/spec/proto/runtime/v1/runtime.proto#L19https://github.com/mosn/layotto/blob/f70cdc619693ad762cf809daf0579403c341def1/spec/proto/runtime/v1/runtime.proto#L19) 与Dapr保持一致.

### 核心抽象
为了与pb定义解耦,添加了一层RPC核心抽象.

- invoker: 提供完整对RPC能力, 目前只对接了Mosn
- callback:before/after filter, 可以在请求执行前后执行自定义的逻辑
- invoker: 提供完整的 RPC能力, 目前只对接了Mosn
- callback:before/after filter, 可以在请求执行前后执行自定义的逻辑(例如添加请求头,例如协议转换)
- channel:发送请求,接收响应,负责与不同传输协议交互

由于Mosn已经有了完整的RPC能力支持,layotto只提供了非常轻量的RPC框架
Expand Down
17 changes: 11 additions & 6 deletions etc/script/generate-code.sh
Original file line number Diff line number Diff line change
Expand Up @@ -36,16 +36,15 @@ generateSdkAndSidecar() {
return 0
fi

# 1. create directory
mkdir _output/tmp

# 2. generate code
protoc -I . \
--go_out . --go_opt=paths=source_relative \
--go-grpc_out=. \
--go-grpc_opt=require_unimplemented_servers=false,paths=source_relative \
--p6_out _output/tmp --p6_opt=paths=source_relative \
${protos}

# move code to the right places
# 3. move code to the right places
# sdk
mv _output/tmp/client/client_generated.go sdk/go-sdk/client/client_generated.go
mv _output/tmp/grpc/context_generated.go pkg/grpc/context_generated.go
Expand All @@ -69,14 +68,20 @@ res=$(ls -d spec/proto/extension/v1/*/)
toGenerate=()
idx=0
for r in $res; do
# echo $r
# ignore empty directory
if test $(ls ${r}*.proto|wc -l) -eq 0; then
echo "[Warn] Directory ${r} is empty. Ignore it."
continue
fi

# generate .pb.go
docker run --rm \
-v $project_path/$r:/api/proto \
layotto/protoc

# check if it needs sdk & sidecar generation
protos=$(ls ${r}*.proto)
for proto in ${protos}; do
# check if it needs sdk & sidecar generation
needGenerate "${proto}"
if test $? -eq $true; then
echo "${proto} needs code generation."
Expand Down
31 changes: 21 additions & 10 deletions etc/script/generate-doc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,11 @@ generateQuickstart() {
fi

# 6. add the quickstart doc into the test-quickstart.sh
addQuickstartIntoCI "docs/en/start/${nickname}/start.md"
addQuickstartIntoCI "docs/zh/start/${nickname}/start.md"
# check no `@exclude` tag
if [ $(grep "@exclude skip ci_generator" "${proto_path}/${proto_name}" | wc -l) -eq 0 ]; then
addQuickstartIntoCI "docs/en/start/${nickname}/start.md"
addQuickstartIntoCI "docs/zh/start/${nickname}/start.md"
fi

# 7. clean up
rm "${proto_path}/${nickname}"
Expand All @@ -117,14 +120,21 @@ echo "===========> Generating docs for ${proto_path_extension}"
res=$(cd $proto_path_extension && ls -d *)
for directory in $res; do
echo "===========> Generating the API reference for ${proto_path_extension}/${directory}"
# 1.1. generate the API reference

# 1.1. ignore empty directory
if test $(ls ${proto_path_extension}/${directory}/*.proto |wc -l) -eq 0; then
echo "[Warn] Directory ${directory} is empty. Ignore it."
continue
fi

# 1.2. generate the API reference
docker run --rm \
-v ${project_path}/docs/api/v1:/out \
-v ${project_path}/${proto_path_extension}/${directory}:/protos \
-v ${tpl_path}:/tpl \
pseudomuto/protoc-gen-doc --doc_opt=/tpl/api_ref_html.tmpl,${directory}.html

# 1.2. generate the quickstart document
# 1.3. generate the quickstart document
# find all protos
protos=$(cd ${proto_path_extension}/${directory} && ls *.proto)
for p in ${protos}; do
Expand Down Expand Up @@ -154,13 +164,19 @@ done
cd $project_path
sidebar_zh=docs/zh/api_reference/README.md
sidebar=docs/en/api_reference/README.md
echo "===========> Updating the sidebar"
echo "===========> Updating the API reference index"
# delete existing lines
# -i "" is for compatibility with MacOS. See https://blog.csdn.net/dawn_moon/article/details/8547408
sed -i "" '/.*: \[.*\]\(.*\)/d' $sidebar_zh
sed -i "" '/.*: \[.*\]\(.*\)/d' $sidebar
# reinsert the reference lines
for r in $res; do
# ignore empty directory
if test $(ls ${proto_path_extension}/${r}/*.proto |wc -l) -eq 0; then
echo "[Warn] Directory ${r} is empty. Ignore it."
continue
fi
# insert
echo "$r: [spec/proto/extension/v1/$r](https://mosn.io/layotto/api/v1/$r.html) \n" >>$sidebar_zh
echo "$r: [spec/proto/extension/v1/$r](https://mosn.io/layotto/api/v1/$r.html) \n" >>$sidebar
done
Expand All @@ -171,8 +187,3 @@ sed -i "" '$d' $sidebar
# 4. update the sidebar
cd $project_path
# TODO

cd $project_path
# generate index for api references
#idx=$(cd docs && ls api/v1/*)
#echo $idx > docs/api/extensions.txt
Loading