Application: Research feasibility and design for Java Host

applications/research-feasibiliy-java-host.md

@@ -0,0 +1,216 @@
+# Java Host Research Proposal
+
+- **Project Name:** Research feasibility of Polkadot Host in Java
+- **Team Name:** [LimeChain](https://github.com/LimeChain)
+- **Payment Address:** eth:0x1F7683228Ee9Bc65335374eA5c92B81C74131404 (USDC/USDT/DAI)
+- **[Level](https://github.com/w3f/Grants-Program/tree/master#level_slider-levels):** 3
+
+# Project Overview **📄**
+
+## Overview
+
+Over the past several years the Polkadot and Substrate community has put a lot of effort into growing the diversity of
+both Host and Runtime implementations. Firstly, in addition to the Substrate runtime environment,
+currently, [an alternative framework for building Substrate-compatible runtimes in Go is being developed (by another team at LimeChain)](https://polkadot.polkassembly.io/motion/292)
+. On top of that, multiple different language host implementations have been built, and are currently maintained and
+improved:
+
+- Go: [gossamer](https://github.com/ChainSafe/gossamer) by ChainSafe
+- C++: [kagome](https://github.com/soramitsu/kagome) by Soramitsu
+- Rust: [polkadot](https://github.com/paritytech/polkadot) by Parity
+- Rust Light Client: [smoldot](https://github.com/paritytech/smoldot) by Parity
+
+While the ecosystem is growing with multiple alternatives for host implementation, adding a host implementation in Java
+will further enhance the diversity, and will enable a whole new subset of participants to enter the Polkadot ecosystem.
+
+If we try to judge the possible diversity increase, we can look no further than the Ethereum ecosystem as an example of
+client diversity. Currently, around 18% of the consensus nodes run the Java-based consensus
+client ([Teku](https://consensys.net/knowledge-base/ethereum-2/teku/)) and around 6% of the users run the java execution
+client ([Hyperledger Besu](https://besu.hyperledger.org/en/stable/)).
+
+Furthermore, such a host written in java has the chance of attracting the attention of the Java development communities
+and Java-centric organisations. These ecosystems and organisations usually consist of well-established organisations and
+communities. Some of the key advantages of Java-based applications include:
+
+- Platform independence: Java code can be run on any device or operating system that supports a Java Virtual Machine (
+  JVM), making it a highly portable language. This is particularly valuable for enterprises, which often have complex,
+  distributed IT environments with a mix of different operating systems and hardware platforms.
+- Scalability and performance: Java is a high-performance language that is designed to be scalable and efficient. It has
+  a number of features, such as just-in-time compilation and garbage collection, that make it well-suited for
+  large-scale, resource-intensive applications. The team at Limechain is a contributor
+  to [Hedera Hashgraph](https://hedera.com/) - a high performant BFT-based ledger network primarily built in Java.
+- Strong community and support: As a widely-used language with a large and active community of developers, Java has a
+  wealth of resources and support available for developers. This includes support on the most popular libraries, as well
+  as a wealth of online resources and communities where developers can go for help and advice.
+- Enterprise-grade libraries and tools: Java has a rich ecosystem of libraries and tools that are specifically designed
+  for enterprise-grade applications. This includes things like enterprise application frameworks, data access libraries,
+  and security tools, all of which make it easier for developers to build high-quality, enterprise-grade applications.
+  One such, relevant example is the Java version of the wasm execution environment - wasmer
+    - [wasmer-java](https://github.com/wasmerio/wasmer-java).
+
+In summary, having a host implementation in Java will further expand the ecosystem and enhance diversity. If the state
+of the blockchain networks, and the history of software engineering for that matter, has taught us anything, it is that
+the bigger and more diverse the ecosystem is - in terms of both people and software, the more resilient it is. With this
+research, we want to contribute to increasing both of these.
+
+## Project Details
+
+### Research & PoC development
+
+**The end goal of this initiative is to create a host implementation in Java**. Going directly for a host implementation
+could prove a challenging task, thus spending the time to abstract and formalise the different building blocks and their
+implementation feasibility in Java is the rational first step.
+
+The end goal of the research as part of this grant is to prove the feasibility and formalise the architecture of Java
+Host written in Java.
+
+We would focus on clearing out the unknowns, mitigating the risks and doing feasibility research for several components
+that hold the biggest risks:
+
+- **Runtime Execution Environment** - Verify that Java can be a viable and performant execution environment for WASM,
+  and especially the WebAssembly MVP with the domain-specific API that Polkadot uses. In order to clear out this
+  unknown, the team will create a PoC to prove or disprove the feasibility of a Java-based execution environment. The
+  library to be tested is `wasmer-java`. The following functionalities of `wasmer-java` will be tested:
+  - Load `wasm` (WebAssemblyMVP) bytecode for execution
+  - Import Host APIs within the Runtime
+    - Verify that all `crypto` and hashing primitives can be implemented in Java
+  - Load `__heap_base` exported from the `WASM` runtime
+  - Allocate/Deallocate memory buffer for transferring data between Host <> Runtime
+  - Set data in the shared memory buffer for sending data to Runtime
+  - Call exported from the WASM Bytecode function with data arguments
+  - Read and parse the returned data of the WASM Runtime from the shared memory buffer 
+- **PolkaJ** - https://github.com/emeraldpay/polkaj has a lot of the required building blocks for a Java Host. The team
+  will verify or disprove the feasibility of the `SCALE` codec implementation as-well as the WASM Wrapper implementation
+  of https://github.com/w3f/schnorrkel. As an alternative implementation of `schnorrkel` we will
+  explore https://github.com/debuggor/schnorrkel-java as-well.
+- **libp2p** - `yamux` and [kademlia](https://github.com/libp2p/specs/blob/master/kad-dht/README.md) are not yet supported
+  in the [JVM](https://github.com/libp2p/jvm-libp2p) implementation. We will assess the scope of the missing libraries
+  and will contact the team to discuss potential implementation on their side or contributing on our.
+
+Furthermore, the project will include multiple research outcomes, detailing and specifying the way that the different
+components of light and full nodes can be built.
+
+# Team **👥**
+
+### Team members
+
+- Daniel Ivanov
+- Georgi Spasov
+- Viktor Todorov
+
+### Contact
+
+- **Contact Name:** Christian Veselinov, Zhivko Todorov, George Spasov
+- **Contact Email:** [chris@limechain.tech](mailto:chris@limechain.tech)
+  , [zhivko@limechain.tech](mailto:zhivko@limechain.tech), [george@limechain.tech](mailto:george@limechain.tech)
+- **Website:** [https://limechain.tech](https://limechain.tech/)
+
+### Legal Structure
+
+- **Registered Address:** Bulgaria, Dragan Tsankov 23A, 1113, Sofia, Bulgaria
+- **Registered Legal Entity:** LimeLabs Ltd.
+
+### Team's experience
+
+At LimeChain, we feel confident in and have a lot of development hours invested in the Polkadot ecosystem. Some of the
+ecosystem work we’ve done involves the creation
+of [Runtime from scratch](https://github.com/LimeChain/as-substrate-runtime) and evolving it into fully
+capable [frameworks for building Substrate Runtimes](https://github.com/LimeChain/subsembly).
+
+We have extensive experience in building Java clients, given that we are contributors to Hedera Hashgraph, where we are
+part of the development of the network client in Java. We also feel experienced in building developer tooling,
+specifically within Rust/WebAssembly ([matchstick](https://github.com/limeChain/matchstick/)). All of this emphasizes
+that the knowledge/expertise of the company spans across the required technological stacks - Substrate, WebAssembly and
+Java.
+
+- [https://github.com/LimeChain/subsembly](https://github.com/LimeChain/subsembly)
+- [https://github.com/LimeChain/subsembly-core](https://github.com/LimeChain/subsembly-core)
+- [https://github.com/LimeChain/as-scale-codec](https://github.com/LimeChain/as-scale-codec)
+- [https://github.com/LimeChain/as-substrate-runtime](https://github.com/LimeChain/as-substrate-runtime)
+- [https://github.com/LimeChain/gosemble](https://github.com/LimeChain/gosemble)
+
+### Team Code Repos
+
+- [https://github.com/LimeChain](https://github.com/LimeChain)
+
+Please also provide the GitHub accounts of all team members. If they contain no activity, references to projects hosted
+elsewhere or live are also fine.
+
+- [https://github.com/daniel-k-ivanov](https://github.com/daniel-k-ivanov)
+- [https://github.com/Perseverance](https://github.com/Perseverance)
+- [https://github.com/vikinatora](https://github.com/vikinatora)
+
+### Team LinkedIn Profiles (if available)
+
+- [https://www.linkedin.com/in/daniel-k-ivanov/](https://www.linkedin.com/in/daniel-k-ivanov/)
+- [https://www.linkedin.com/in/george-spasov/](https://www.linkedin.com/in/george-spasov/)
+- [https://www.linkedin.com/in/viktor-todorov-8a7434122/](https://www.linkedin.com/in/viktor-todorov-8a7434122/)
+
+# **Development Status 📖**
+
+As part of our current research, we’ve compared the Polkadot host specification and requirements with the currently
+available tools and libraries in Java needed to implement it. While the preliminary research shows positive results,
+several important aspects of the creation of a host need to be further proven, tested or researched:
+
+- The feasibility of using Java as an execution environment for WebAssembly MVP
+    - Another popular Polkadot Host implementation - Gossamer - uses wasmer as an execution environment for the runtime
+      WebAssembly. [Wasmer-java](https://github.com/wasmerio/wasmer-java) is an available option but needs to be tested
+      out against WebAssembly MVP and the domain-specific APIs.
+- The overall feasibility of the creation of the different modules and the rough architecture (storage, consensus,
+  networking, RPCs, etc) in order to create light and full client nodes.
+
+# Development Roadmap **🔩**
+
+Described below are the steps we think are necessary to get a deep understanding of how the Java host can be built:
+
+1. Review and build a PoC for `wasmer-java` against WebAssembly MVP
+    1. In order to prove the feasibility of Java being an execution environment we will create a small POC showcasing
+       the ability of Java to communicate with a WebAssembly MVP binary
+2. Validate / Invalidate the applicability of [PolkaJ](https://github.com/emeraldpay/polkaj)
+    1. Reusing the `SCALE` codec implementation
+    2. Reusing the `schnorrkel` of PolkaJ or [schnorrkel-java](https://github.com/debuggor/schnorrkel-java)
+3. Research the feasibility of [libp2p-jvm](https://github.com/libp2p/jvm-libp2p)
+    1. Discuss timelines on implementing `yamux` and `kademlia` with Protocol Labs or potential contribution
+4. Research the feasibility of the different modules needed in order to create a light client
+5. Research the feasibility of the different modules needed in order to create a full host
+6. Propose an architecture and sensible roadmap for implementing a Java Host, based on the previous steps
+
+### Overview
+
+- **Total Estimated Duration:** 6 working weeks
+- **Full-Time Equivalent (FTE):** 2
+- **Total Costs:** $36,000
+
+### Milestone 1 — Research the feasibility of Polkadot Host in Java
+
+- **Estimated duration:** 4 working weeks
+- **FTE:** 2
+- **Costs:** $24,000
+
+This milestone will:
+
+| Number | Deliverable | Specification |
+| --- | --- | --- |
+| 0a. | License | Apache 2.0 |
+| 0b. | Documentation | We will provide Markdown documentation of the whole research, explaining the necessary steps needed to resolve the technical challenges of the creation of a Java-based light client and full host. |
+| 1a.  | PoC | Research and outline the feasibility of using `wasmer-java` for the Runtime Execution Environment for WebAssembly MVP. PoC will be built in order to verify the feasibility. |
+| 1b. | Testing Guide | We will provide a testing guide of the PoC of a Java-based WASM execution environment |
+| 1c. | Docker | We will provide a Dockerfile(s) that can be used to test the PoC. |
+| 2. | Research | Research and verify the feasibility of reusing PolkaJ libraries for various components such as `SCALE`, `schnorrkel` and `RPC` API. |
+| 2. | Research | Research and outline the design of the p2p module of the Java Polkadot Host. Asses the implementation of `yamux` and `kademlia`. Contact the Protocol Labs team.|
+| 3. | Research | Research and outline the design of the storage module (Host and Runtime) of the Java Polkadot Host. |
+| 4. | Research | Research and outline the design of the RPC API of the Java Polkadot Host. |
+| 5. | Research | Research and outline the design of the modules and APIs for the Java Polkadot Host to be part of the consensus-reaching processes. |
+| 6. | Documentation | Java Polkadot Host design document will be produced that combines the research findings of the previous deliverables and outlines a high-level specification for the creation of a Java-based Host. |
+| 7. | Documentation | Java Polkadot Host design document will be produced that combines the research findings of the previous deliverables and outlines a high-level specification for the creation of a Java-based Light-client. |
+
+# Future Plans
+
+Having this research will give us a clear understanding of how Java light and full clients can be built. Completing it
+will become the foundational block for the creation of the Java-based Host which would be the next step for the team
+after delivering this research.
+
+# Additional Information **➕**
+
+LimeChain has been a long-time contributor to the Substrate ecosystem mainly focused on developer tooling. Due to our
+involvement in the space, we are working with various clients, developing smart contracts and working on parachains.