proposal: Trusted Protocal and Trust Native Protocal

[seeflood]

What would you like to be added:
Layotto not only supports dapr api, but also supports all "trusted protocols",e.g. redis protocal and aws s3 protocal ( #510 is working on it)

Specifically, we can divide layotto API into three types:

• Standard API. This is what we already have.
• Trusted Protocal. For example , we add an "aws s3 API".
• Trust Native Protocal. For example, layotto support redis native protocal

I will submit a PR to add support for redis API.

Why is this needed:
That's a long story. To be short, current dapr api can not meet all the needs of applications in the real world while maintaining portability, but protocals like redis protocal and aws s3 protocal can.
For more details, I have a presentation for it .

[seeflood]

Subtasks:

• s3 API. Assigned feat: implement oss interface #556

• Redis API. 设计实现 Redis API #608

• Redis native protocal [feature request] Let MOSN act as a Redis proxy, like envoy mosn#2054

• Kafka API.
  Alicloud SLS component can implement Kafka-like API too.

• Design transaction message API (like RocketMQ's) for pubsub

• Delay message API for pubsub proposal: pubsub api add delay message support #617

• Design a "feature discovery" mechanism

---

chinese:

• 设计实现 s3 API. assigned feat: implement oss interface #556
• 设计实现 Redis API. 设计实现 Redis API #608
• Redis native protocal
• 设计实现 Kafka API
  有了这个API后，我们可以用阿里云 SLS 组件来实现这个 API
• 设计实现事务消息 API
• 设计实现延迟消息 API proposal: pubsub api add delay message support #617
• Design a "feature discovery" mechanism

[seeflood]

we can do some research on eventmesh and openmessaging

[kevinten10]

Does the Redis API need to be defined in the proto file?

We have tried to define the interface of Redis API on capa-java before, mainly referring to jedis, but there are many jedis custom objects which are very complicated.

[seeflood]

Does the Redis API need to be defined in the proto file?

I think so. It's like "the gRPC version of redis binding api", and as a gRPC api, it needs a proto.
From this perspective, the interface of Redis API on capa-java is "the in-process version of redis binding API"

We have tried to define the interface of Redis API on capa-java before, mainly referring to jedis, but there are many jedis custom objects which are very complicated.

I read your code and it's very good. Could u give some examples on "jedis custom objects which are very complicated"?
By the way, did you already use these interfaces in your production environment?

[azhsmesos]

Delay message API for pubsub 这儿有人assigned吗，等我目前的pr 如果成功 merge后想尝试下这个的实现

[seeflood]

@azhsmesos 欢迎，目前没人 assigned
因为这个功能比较重要，需要先做设计（写个设计 proposal)，大家讨论后达成一致了再开始编码~
可以先想想怎么设计，如果不太清楚的话我帮着写个初版设计proposal也行

[kevinten10]

I read your code and it's very good. Could u give some examples on "jedis custom objects which are very complicated"?
By the way, did you already use these interfaces in your production environment?

我给出的Redis API链接里，剔除了"非常复杂的jedis自定义对象”的部分，因为这部分是跟jedis的jar耦合在一起的，在开源版本的设计中我不希望它们强耦合。

举个例子，在上述定义的完全版的Redis API NativeKeyCommands中，有这样的API：

import redis.clients.jedis.params.SortingParams;

List<byte[]> sort(byte[] key, SortingParams sortingParameters);

• SortingParams

By the way, did you already use these interfaces in your production environment?

是的，包含了上述Jedis复杂对象部分的API，就是目前在生产上使用的Redis API。它基于jedis，在java-sdk中可以良好工作。

[kevinten10]

以下是使用到的"非常复杂的jedis自定义对象”：

NativeGeoCommands.java

import redis.clients.jedis.GeoCoordinate;

import redis.clients.jedis.GeoRadiusResponse;

import redis.clients.jedis.GeoUnit;

import redis.clients.jedis.params.geo.GeoRadiusParam;

NativeHashCommands.java

import redis.clients.jedis.ScanParams;

import redis.clients.jedis.ScanResult;

NativeKeyCommands.java

import redis.clients.jedis.SortingParams;

NativeListCommands.java

import redis.clients.jedis.BinaryClient;

NativeSetCommands.java

import redis.clients.jedis.ScanParams;

import redis.clients.jedis.ScanResult;

NativeSortedSetCommands.java

import redis.clients.jedis.ScanParams;

import redis.clients.jedis.ScanResult;

import redis.clients.jedis.Tuple;

import redis.clients.jedis.params.sortedset.ZAddParams;

NativeStringCommands.java

import redis.clients.jedis.BitOP;

可以在 https://github.com/redis/jedis/tree/master/src/main/java/redis/clients/jedis 中找到这些对象，不过由于jar包版本区别，可能和上述import目录不同。

[seeflood]

Update:
I just found that dapr is developing "capability API" for feature discovery.
dapr/dapr#4367
dapr/dapr#4621

Personally I don't want to put the discovery logic into sidebar

[azhsmesos]

我不是很理解咱们api规范是啥样的，比如事物消息，我我这儿有一份思路就是，可以类似rocketMQ那样，利用两阶段提交+TCC补偿机制实现，比较重要考虑的问题就是可能在两阶段提交中途宕机导致事物回滚不成功，这儿要么就是consumer端去进行tcc补偿，但是感觉对业务侵入性比较大，可不可以考虑利用redis+lua的scriptLoad脚本实现，只要redis不宕机就可以保证事物的正常执行

[kevinten10]

@azhsmesos Hi~ 您说的是关于事务消息API的设计思路对么~

但是感觉对业务侵入性比较大

我不是很清楚事务方面有没有更简洁易用的api呢，但感觉侵入性越大越难可移植呢

或许也可以参考一下dapr的一个提案中对transaction的设计：dapr/dapr#3354

[seeflood]

I submitted an issue in MOSN mosn/mosn#2054
We can also support redis native protocal