Proposal: make layotto more friendly in Cloud Native Ecosystem

[Xunzhuo]

Hi community :)
I am willing to propose and push the process of containerization and cloud native compatibility in layotto, some major tasks I can imagine:

Refactor the Dockerfile

I read the structure of dockerfile/makefile for now, generally great, but I think we can optimize them better and more clean.

Provide Docker-Compose

If users want to start to use and run some demos (As I know, layotto contains many interesting demos, but they are not for Quick Start, because I will do many steps to run layotto up)
According to docs, there are 15 demos for now, I list them in below, I will write each one of them with a docker-compose, for users. they can just run docker-compose up to have a quick start to run a demo:)

API

API	status	quick start	components	desc
State	✅	demo	list	Write/Query the data of the Key/Value model
Pub/Sub	✅	demo	list	Publish/Subscribe message through various Message Queue
Service Invoke	✅	demo	list	Call Service through MOSN (another istio data plane)
Config	✅	demo	list	Write/Query/Subscribe the config through various Config Center
Lock	✅	demo	list	Distribute lock implementation
Sequencer	✅	demo	list	Distribube auto increment ID generator
File	✅	TODO	list	File API implementation
Binding	✅	TODO	TODO	Transparent data transmission API

Extendability

feature	status	quick start	desc
API plugin	✅	demo	You can add your own API !

Actuator

feature	status	quick start	desc
Health Check	✅	demo	Query health state of app and components in Layotto
Metadata Query	✅	demo	Query metadata in Layotto/app

Traffic Control

feature	status	quick start	desc
TCP Copy	✅	demo	Dump the tcp traffic received by Layotto into local file system
Flow Control	✅	demo	limit access to the APIs provided by Layotto

Multilingual programming based on WASM

feature	status	quick start	desc
Go (TinyGo)	✅	demo	Compile Code written by TinyGo to *.wasm and run in Layotto
Rust	TODO	TODO	Compile Code written by Rust to *.wasm and run in Layotto
AssemblyScript	TODO	TODO	Compile Code written by AssemblyScript to *.wasm and run in Layotto

FaaS (Layotto + WebAssembly + k8s)

feature	status	quick start	desc
Go (TinyGo)	✅	demo	Compile Code written by TinyGo to *.wasm and run in Layotto And Scheduled by k8s.
Rust	TODO	TODO	Compile Code written by Rust to *.wasm and run in Layotto And Scheduled by k8s.
AssemblyScript	TODO	TODO	Compile Code written by AssemblyScript to *.wasm and run in Layotto And Scheduled by k8s.

Service Mesh

feature	status	quick start	desc
istio	✅	demo	As the data plane of istio

Provide Helm Charts for layotto

When running in a cluster, deploying app with helm is a welcomed and convenient approach. Helm provides the ability to deploy and manage apps in kubernetes or other container orchestration, which makes user easy to install/upgrade/uninstall apps. So design/create/write one/more helm charts is necessary. I would like to provide this more cloud native way to install layotto in cluster

Provide Operator to add/update/delete laytto and watch its events

Operator is the other dimension to manage resources in cluster, I would like to create/write a operator, with layotto CRD, to install layotto in cluster:)

Provide more powerful CI by GitHub Workflows in Project Infa Area

Some child tasks I can imagine:

• lint && style check for golang
• auto build binaries across platform out by GitHub Workflow
• auto build image and push image to container registry like docker Hub
  This is useful when releasing or developing layotto:
  The release workflow like this:
  Create a tag -> trigger the workflow -> build image with release tag -> push to image
  The dev workflow like this:
  Merge a PR -> trigger the workflow -> build image with release tag -> push to image

The whole work is big, if u forks think it is acceptable, I will spare my time to finish this whole work, separate this work into a number of PRs in the next few months:)

[seeflood]

Thanks for your proposal ! These are really important and can make Layotto more cloud native :)
Looking forward to your pull request.
If you find there is too much to do, you can also split them into different tasks and add the tasks into the community task list so that others can help.

[seeflood]

@Xunzhuo Hi , are u working on it?
@stulzq 老师 can help do kubernetes integration. Can we split this proposal into different sub tasks?

• Refactor the Dockerfile. Develop a "basic" dockerfile #628
• Provide Docker-Compose
• deploy layotto in kubernetes (with Helm ,or some other approaches)
• Provide Operator to add/update/delete laytto and watch its events
• Provide more powerful CI by GitHub Workflows in Project Infa Area

We can discuss this great proposal recently in the first community meeting.
What do you think?

[Xunzhuo]

Yes, I am working on this, but due to the busy work recently, I will start to do the jobs ASAP.

[stulzq]

@Xunzhuo Hi , are u working on it? @stulzq 老师 can help do kubernetes integration. Can we split this proposal into different sub tasks?

• Refactor the Dockerfile
• Provide Docker-Compose
• deploy layotto in kubernetes (with Helm ,or some other approaches)
• Provide Operator to add/update/delete laytto and watch its events
• Provide more powerful CI by GitHub Workflows in Project Infa Area

We can discuss this great proposal recently in the first community meeting. What do you think?

okkk

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue or help wanted) or other activity occurs. Thank you for your contributions.

[seeflood]

Regarding to deployment in k8s:
I just read through mosn's tutorials again and surprisedly found that mosn can be deployed using istio's toolchain, at very little cost. The only thing mosn has to do is to build a proxyv2 image with mosn in it.
So......
If we build a proxyv2 image with layotto in it, we can reuse istio's toolchains to deploy the sidecar in k8s , too ?

Ref:
https://mosn.io/docs/quick-start/istio/
https://istio.io/latest/blog/2020/mosn-proxy/

[seeflood]

Conclusion of community meeting discussion:

• Regarding docker image: when no start-up arguments passed to layotto, layotto should try to read the configuration file from an ABSOLUTE path by default.
  With this feature, we can mount the configuration file using ConfigMap in k8s

• Regarding helm chart: currently we only focus on using istio to deploy layotto. In the future, we can add a demo to show how to deploy layotto using 北极星 . So we won't develop helm chart .

[seeflood]

@Xunzhuo Hi, can I submit a "help wanted" issue as a subtask of this proposal?
The issue aims to develop a "basic" dockerfile, which is common and decoupled with the config.json, so that it can be used with any config.json files

[Xunzhuo]

Sure @seeflood

[seeflood]

I saw microsoft launched a project https://github.com/Azure/draft to auto-generate Kubernetes manifests, Helm charts, Kustomize configuration, and other artifacts.
Maybe it can help us. I haven't looked into this in details

docs:
https://docs.microsoft.com/en-us/azure/aks/draft

---

And the idea to write a command line tool to auto-generate github workflow is very smart. We can encapsulate our workflows with a cli tool and auto-generate workflows for other projects.
I also found azure has some interesting repos for github workflow reuse:
https://github.com/Azure/actions-workflow-samples
https://github.com/Azure/actions

[Xunzhuo]

Thanks @seeflood, I will take a look in the details in the next week.