From 1b1fc392d64ad00695063dd4c817dd2496858931 Mon Sep 17 00:00:00 2001 From: Sabine Proll Date: Fri, 28 Apr 2023 11:08:06 +0200 Subject: [PATCH 1/4] proposal: static analysis tool for runtime pallets --- applications/sarp-basic-functionality.md | 140 +++++++++++++++++++++++ 1 file changed, 140 insertions(+) create mode 100644 applications/sarp-basic-functionality.md diff --git a/applications/sarp-basic-functionality.md b/applications/sarp-basic-functionality.md new file mode 100644 index 00000000000..ccfbe9e2836 --- /dev/null +++ b/applications/sarp-basic-functionality.md @@ -0,0 +1,140 @@ +# SARP - A Static Analysis Tool for Runtime Pallets + +- **Team Name:** Supercomputing Systems AG +- **Payment Address:** FsnxqJnqWVNMZZgxaQdhaCk9c5sL3WSggRCRqp1qEzk1L2i (KSM) +- **[Level](https://github.com/w3f/Grants-Program/tree/master#level_slider-levels):** 2 + +## Project Overview :page_facing_up: + +This application is a response to the RFP [Static Analysis for Runtime Pallets](https://github.com/w3f/Grants-Program/blob/master/docs/RFPs/Open/Static-Analysis-for-Runtime-Pallets.md) + +### Overview + +[Runtime Pallets](https://docs.substrate.io/learn/runtime-development/) are modules for writing the business logic of blockchains in [Substrate](https://github.com/paritytech/substrate) (a Rust framework for building blockchains). These are usually concise pieces of standalone code with relatively few dependencies and clear specifications, hence tractable targets for performing static analysis and verification. The code quality of a runtime pallet is crucial, as even minor defects can result in major exploits like DoS attacks or the stealing of funds by a malicious party. A static code analysis can help to automate the auditing processes and prevent introduction of defects throughout the software life-cycle. + +Therefore we would like to develop a tool - SARP (Static Analysis tool for Runtime Pallets) to perform static analysis with reasonable soundness guarantees. In particular, we would like to target vunerability classes that are detectable using dataflow analysis techniques like *tag analysis* and *taint analysis*. + +Our team has no prior knowledge in static code analysis, but has a good understanding of substrate and Rust. + +### Project Details + +We will base our work on [MIRAI](https://github.com/facebookexperimental/MIRAI/) and extend it with checks on substrate pallets. For details see [Milestone 1](#milestone-1---basic-setup--2-usecases) + +### Ecosystem Fit + +The tool will help any team developing substrate pallets. It can further be integrated in the CI pipelines of the teams, providing a continuous quality check on the pallet code. + +In the long term it could be interesting to connect the work done here with the new emerging auditing DAOs (like [Fuzzland](https://github.com/w3f/Grants-Program/blob/master/applications/FuzzLand.md) or [QRUCIAL DAO](https://github.com/w3f/Grants-Program/blob/master/applications/QRUCIAL_DAO.md)). + + +## Team :busts_in_silhouette: + +### Team members + +- Sabine Proll: Project Lead +- Bigna Härdi: Developer +- Edith Chevrier: Developer +- Thomas Niederberger: Developer + +### Contact + +- **Contact Name:** Sabine Proll +- **Contact Email:** Sabine.Proll@scs.ch | info@scs.ch +- **Website:** https://www.scs.ch + +### Legal Structure + +- **Registered Address:** Technoparkstrasse 1, 8005 Zürich, Switzerland +- **Registered Legal Entity:** Supercomputing Systems AG + +### Team's experience + +Supercomputing Systems AG is a contractor with 130 engineers, working in the fields of software, electronics and system design. Profound know-how, solid methodological competence as well as efficient project management are the foundation of our success. Within the company we have a team of 5 blockchain developers, who have experience in the Polkadot ecosystem. + +Our blockchain team has been a contributor to the ecoysystem since 2019. We started with grants from the Web3 Foundation to build the basis for [Integritee](https://github.com/integritee-network) (see our grants from waves [1](https://github.com/w3f/General-Grants-Program/blob/master/grants/speculative/substrate_sgx_proposal.md), [3](https://github.com/w3f/General-Grants-Program/blob/master/grants/speculative/substrate-api-client.md) and [5](https://github.com/w3f/General-Grants-Program/blob/master/grants/speculative/SubstraTEE-extension-pack1.md)). After that, our team has worked for Integritee and Encointer as a contractor. Recently the team received grants from the Kusama treasury for maintaining and improving the [substrate-api-client](https://github.com/scs/substrate-api-client), see our proposals for [Nov 22 - Jan 23](https://kusama.subsquare.io/referenda/referendum/26) and [Feb 23 - Apr 23](https://kusama.subsquare.io/referenda/referendum/88). + +### Team Code Repos + +The team has mainly worked on the following repositories + +- [Substrate Api Client](https://github.com/scs/substrate-api-client) +- [Integritee Worker](https://github.com/integritee-network/worker) +- [Encointer Sidechain](https://github.com/encointer/community-sidechain) + +Github accounts of the team members + +- https://github.com/masapr +- https://github.com/haerdib +- https://github.com/echevrier +- https://github.com/Niederb + + +### Team LinkedIn Profiles + +- https://www.linkedin.com/in/sabine-proll-5a7118153 +- https://www.linkedin.com/in/bigna-h%C3%A4rdi-736bb21a9 +- https://www.linkedin.com/in/edith-chevrier-90233297 +- https://www.linkedin.com/in/thomas-niederberger-6057b71a7 + +## Development Status :open_book: + +We will base our work on [MIRAI](https://github.com/facebookexperimental/MIRAI) and the RFP [Static Analysis for Runtime Pallets](https://github.com/w3f/Grants-Program/blob/master/docs/RFPs/Open/Static-Analysis-for-Runtime-Pallets.md) + +We have not started to work on this. + +## Development Roadmap :nut_and_bolt: + +### Overview + +- **Total Estimated Duration:** 1,5 months +- **Full-Time Equivalent (FTE):** 0,8 FTE +- **Total Costs:** 30.000 USD + +### Milestone 1 - Basic Setup + 2 Usecases + +- **Estimated duration:** 1,5 months +- **FTE:** 0,8 FTE +- **Costs:** 30.000 USD + + + +This project contains the basic project setup and a first set of checks on the following vulnerability classes: + +* [incorrect origin](https://github.com/bhargavbh/MIRAI/blob/main/substrate_examples/incorrect-origin/description.md) of dispatchable functions. +* [unsigned transaction](https://github.com/bhargavbh/MIRAI/blob/main/substrate_examples/unsigned-transaction/description.md) validation. + +The goal is to show the feasibility of the approach and to lay a basis for future development. Therefore we leave tasks with little risks to future work packages (see also [Future Plans](#future-plans)). + +#### Deliverables + +| Number | Deliverable | Specification | +|--------|------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| 0a. | License | Apache 2.0 | +| 0b. | User Documentation | We will provide both **inline documentation** of the code and a basic **tutorial** that explains how to use the tool on a substrate pallet.| +| 0c. | Testing and Testing Guide | Each check, that the tool can perform, will be fully covered by comprehensive unit tests to ensure functionality and robustness. In the testing guide, we will describe how to run these tests. | +| 0d. | Docker | We will provide a Dockerfile(s) that can be used to test all the functionality delivered with this milestone. | +| 1. | Tool | A robust static analysis tool that works on Substrate runtime pallets. The tool will provide at least one check on each vulnerability class. See also [Limitations](#Limitations). | +| 2. | Vulnerability Documentation | For each of the above stated vulnerability classes we will document, how this vulnerability is covered by our checks and limitations thereof. | +| 3. | Engagement | Engage with teams at Web3 Foundation and Parity to prioritise vulnerability classes. | + +#### Limitations +The possibilities of a static code analysis are limited. On the one hand, the checks can generate many false positives. On the other hand, it might not be possible to catch all vulnerabilities within a class of vulnerabilities. Because of this we will document the limits of the tool and closely cooperate with the teams at Web3 Foundation and Parity to prioritize the features and their implementation. + +## Future Plans + +The next steps for the tool would be to: + +* improve the usability, by providing + * means to surpress warnings + * a comprehensive user tutorial, incl. documentation on the risks of each vulnerability +* add more features including checks on the following vulnerability classes: + * tracking bad randomness: ensure bad randomness does not leak into sensitive functions. + * detect panics statically to avoid potential DoS attacks: these include [unsafe arithmetic operations](https://github.com/bhargavbh/MIRAI/blob/main/substrate_examples/arithmetic-overflow/description.md), access outside bounds, assertion failures, etc. + * tracking unsanitised input leakage for sensitive functions. + +Once we have a tool with a good feature set and basic usability features, we want to promote it to the Polkadot developers. Once the tool is in use, we hope to receive feedback on further features and improvements by the developers. + +## Additional Information :heavy_plus_sign: + +**How did you hear about the Grants Program?** +We have previously received grants by the Web3 Foundation for other projects (substratee and substrate-api-client). \ No newline at end of file From 38006eeff0bb4d5d3bc093b8f197f75bc44ad3ba Mon Sep 17 00:00:00 2001 From: Sabine Proll Date: Tue, 2 May 2023 15:39:17 +0200 Subject: [PATCH 2/4] - split into 2 milestones (research & implementation) - changed payment to USDC - changed license to MIT (because MIRAI uses MIT) --- applications/sarp-basic-functionality.md | 45 ++++++++++++++++++------ 1 file changed, 34 insertions(+), 11 deletions(-) diff --git a/applications/sarp-basic-functionality.md b/applications/sarp-basic-functionality.md index ccfbe9e2836..a77fa004970 100644 --- a/applications/sarp-basic-functionality.md +++ b/applications/sarp-basic-functionality.md @@ -1,7 +1,7 @@ # SARP - A Static Analysis Tool for Runtime Pallets -- **Team Name:** Supercomputing Systems AG -- **Payment Address:** FsnxqJnqWVNMZZgxaQdhaCk9c5sL3WSggRCRqp1qEzk1L2i (KSM) +- **Team Name:** Supercomputing Systems AG (SCS) +- **Payment Address:** 0xd24622311a22470353bd21d9bcd9e02ba0cfebbe (USDC) - **[Level](https://github.com/w3f/Grants-Program/tree/master#level_slider-levels):** 2 ## Project Overview :page_facing_up: @@ -18,7 +18,7 @@ Our team has no prior knowledge in static code analysis, but has a good understa ### Project Details -We will base our work on [MIRAI](https://github.com/facebookexperimental/MIRAI/) and extend it with checks on substrate pallets. For details see [Milestone 1](#milestone-1---basic-setup--2-usecases) +We will base our work on [MIRAI](https://github.com/facebookexperimental/MIRAI/) and extend it with checks on substrate pallets. For details see the [Development Roadmap](#development-roadmap-nut_and_bolt) ### Ecosystem Fit @@ -90,26 +90,49 @@ We have not started to work on this. - **Full-Time Equivalent (FTE):** 0,8 FTE - **Total Costs:** 30.000 USD -### Milestone 1 - Basic Setup + 2 Usecases - -- **Estimated duration:** 1,5 months -- **FTE:** 0,8 FTE -- **Costs:** 30.000 USD -This project contains the basic project setup and a first set of checks on the following vulnerability classes: +#### Vulnerability Classes +For this project we want to address the following vulnerability classes: * [incorrect origin](https://github.com/bhargavbh/MIRAI/blob/main/substrate_examples/incorrect-origin/description.md) of dispatchable functions. * [unsigned transaction](https://github.com/bhargavbh/MIRAI/blob/main/substrate_examples/unsigned-transaction/description.md) validation. -The goal is to show the feasibility of the approach and to lay a basis for future development. Therefore we leave tasks with little risks to future work packages (see also [Future Plans](#future-plans)). +These will be part of both milestones. + +### Milestone 1 - Research + +- **Estimated duration:** 0,5 months +- **FTE:** 0,8 FTE +- **Costs:** 10.000 USD + +In milestone 1 we want to investigate how the above stated [vulnerability classes](#vulnerability-classes), can be detected by extending MIRAI. + +#### Deliverables + +| Number | Deliverable | Specification | +|--------|-----------------------------|--------------------------------------------------------------------------------------------------------------| +| 0a. | License | MIT | +| 1. | Prototype Code | Prototype code to approach the above two stated vulnerability classes. | +| 2. | Documentation | Technical documentation, describing the approach we plan to implement in milestone 2, incl. its limitations. | +| 3. | Engagement | Engage with teams at Web3 Foundation and Parity for prioritization. | + + + +### Milestone 2 - Basic Setup + 2 Usecases + +- **Estimated duration:** 1 month +- **FTE:** 0,8 FTE +- **Costs:** 20.000 USD + +After the research in milestone 1, we want to implement a first simple version of the tool, together with tests and documentation. The goal is to show the feasibility of the approach and to lay a basis for future development. Therefore we leave tasks with little risks to future work packages (see also [Future Plans](#future-plans)). #### Deliverables | Number | Deliverable | Specification | |--------|------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| 0a. | License | Apache 2.0 | +| 0a. | License | MIT | | 0b. | User Documentation | We will provide both **inline documentation** of the code and a basic **tutorial** that explains how to use the tool on a substrate pallet.| | 0c. | Testing and Testing Guide | Each check, that the tool can perform, will be fully covered by comprehensive unit tests to ensure functionality and robustness. In the testing guide, we will describe how to run these tests. | | 0d. | Docker | We will provide a Dockerfile(s) that can be used to test all the functionality delivered with this milestone. | From 15bba04da42f8c8da8aa883277659eb1c75cefbc Mon Sep 17 00:00:00 2001 From: Sabine Proll Date: Fri, 5 May 2023 11:58:11 +0200 Subject: [PATCH 3/4] adjusted deliverables of milestones after review --- applications/sarp-basic-functionality.md | 32 +++++++++++++----------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/applications/sarp-basic-functionality.md b/applications/sarp-basic-functionality.md index a77fa004970..f5fd6698a9d 100644 --- a/applications/sarp-basic-functionality.md +++ b/applications/sarp-basic-functionality.md @@ -111,12 +111,14 @@ In milestone 1 we want to investigate how the above stated [vulnerability classe #### Deliverables -| Number | Deliverable | Specification | -|--------|-----------------------------|--------------------------------------------------------------------------------------------------------------| -| 0a. | License | MIT | -| 1. | Prototype Code | Prototype code to approach the above two stated vulnerability classes. | -| 2. | Documentation | Technical documentation, describing the approach we plan to implement in milestone 2, incl. its limitations. | -| 3. | Engagement | Engage with teams at Web3 Foundation and Parity for prioritization. | +| Number | Deliverable | Specification | +|--------|-----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| 0a. | License | MIT | +| 0b. | User Documentation | We will provide a basic **tutorial** that explains how to use the tool on a substrate pallet. | +| 0c. | Testing and Testing Guide | A first set of tests will be provided, together with a testing guide, that describes how to run them. | +| 1. | Prototype Code | Prototype code to approach the above two stated vulnerability classes. | +| 2. | Documentation | Technical documentation
  • describing the approach we plan to implement in milestone 2, incl. its limitations.
  • with (interesting) examples of the vulnerability classes.
| +| 3. | Engagement | Engage with teams at Web3 Foundation and Parity for prioritization. | @@ -130,15 +132,15 @@ After the research in milestone 1, we want to implement a first simple version o #### Deliverables -| Number | Deliverable | Specification | -|--------|------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| 0a. | License | MIT | -| 0b. | User Documentation | We will provide both **inline documentation** of the code and a basic **tutorial** that explains how to use the tool on a substrate pallet.| -| 0c. | Testing and Testing Guide | Each check, that the tool can perform, will be fully covered by comprehensive unit tests to ensure functionality and robustness. In the testing guide, we will describe how to run these tests. | -| 0d. | Docker | We will provide a Dockerfile(s) that can be used to test all the functionality delivered with this milestone. | -| 1. | Tool | A robust static analysis tool that works on Substrate runtime pallets. The tool will provide at least one check on each vulnerability class. See also [Limitations](#Limitations). | -| 2. | Vulnerability Documentation | For each of the above stated vulnerability classes we will document, how this vulnerability is covered by our checks and limitations thereof. | -| 3. | Engagement | Engage with teams at Web3 Foundation and Parity to prioritise vulnerability classes. | +| Number | Deliverable | Specification | +|--------|------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| 0a. | License | MIT | +| 0b. | User Documentation | We will provide both **inline documentation** of the code and a basic **tutorial** that explains how to use the tool on a substrate pallet. | +| 0c. | Testing and Testing Guide | Each check, that the tool can perform, will be fully covered by comprehensive unit tests to ensure functionality and robustness. In the testing guide, we will describe how to run these tests. | +| 0d. | Docker | We will provide a Dockerfile(s) that can be used to test all the functionality delivered with this milestone. | +| 1. | Tool | A robust static analysis tool that works on Substrate runtime pallets. Our aim is to provide features, that address all of the above stated vulnerability classes. See also [Limitations](#Limitations). | +| 2. | Vulnerability Documentation | For each of the above stated vulnerability classes we will document, how this vulnerability is covered by our checks and limitations thereof. | +| 3. | Engagement | Engage with teams at Web3 Foundation and Parity to prioritise vulnerability classes. | #### Limitations The possibilities of a static code analysis are limited. On the one hand, the checks can generate many false positives. On the other hand, it might not be possible to catch all vulnerabilities within a class of vulnerabilities. Because of this we will document the limits of the tool and closely cooperate with the teams at Web3 Foundation and Parity to prioritize the features and their implementation. From 40481235c3023abc779fd0fcb9962901f6f7d0f2 Mon Sep 17 00:00:00 2001 From: Sabine Proll Date: Mon, 8 May 2023 15:13:32 +0200 Subject: [PATCH 4/4] removed milestone 2 --- applications/sarp-basic-functionality.md | 46 ++++++------------------ 1 file changed, 11 insertions(+), 35 deletions(-) diff --git a/applications/sarp-basic-functionality.md b/applications/sarp-basic-functionality.md index f5fd6698a9d..485712cd299 100644 --- a/applications/sarp-basic-functionality.md +++ b/applications/sarp-basic-functionality.md @@ -2,7 +2,7 @@ - **Team Name:** Supercomputing Systems AG (SCS) - **Payment Address:** 0xd24622311a22470353bd21d9bcd9e02ba0cfebbe (USDC) -- **[Level](https://github.com/w3f/Grants-Program/tree/master#level_slider-levels):** 2 +- **[Level](https://github.com/w3f/Grants-Program/tree/master#level_slider-levels):** 1 ## Project Overview :page_facing_up: @@ -86,9 +86,9 @@ We have not started to work on this. ### Overview -- **Total Estimated Duration:** 1,5 months +- **Total Estimated Duration:** 0,5 months - **Full-Time Equivalent (FTE):** 0,8 FTE -- **Total Costs:** 30.000 USD +- **Total Costs:** 10.000 USD @@ -99,7 +99,6 @@ For this project we want to address the following vulnerability classes: * [incorrect origin](https://github.com/bhargavbh/MIRAI/blob/main/substrate_examples/incorrect-origin/description.md) of dispatchable functions. * [unsigned transaction](https://github.com/bhargavbh/MIRAI/blob/main/substrate_examples/unsigned-transaction/description.md) validation. -These will be part of both milestones. ### Milestone 1 - Research @@ -121,41 +120,18 @@ In milestone 1 we want to investigate how the above stated [vulnerability classe | 3. | Engagement | Engage with teams at Web3 Foundation and Parity for prioritization. | - -### Milestone 2 - Basic Setup + 2 Usecases - -- **Estimated duration:** 1 month -- **FTE:** 0,8 FTE -- **Costs:** 20.000 USD - -After the research in milestone 1, we want to implement a first simple version of the tool, together with tests and documentation. The goal is to show the feasibility of the approach and to lay a basis for future development. Therefore we leave tasks with little risks to future work packages (see also [Future Plans](#future-plans)). - -#### Deliverables - -| Number | Deliverable | Specification | -|--------|------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| 0a. | License | MIT | -| 0b. | User Documentation | We will provide both **inline documentation** of the code and a basic **tutorial** that explains how to use the tool on a substrate pallet. | -| 0c. | Testing and Testing Guide | Each check, that the tool can perform, will be fully covered by comprehensive unit tests to ensure functionality and robustness. In the testing guide, we will describe how to run these tests. | -| 0d. | Docker | We will provide a Dockerfile(s) that can be used to test all the functionality delivered with this milestone. | -| 1. | Tool | A robust static analysis tool that works on Substrate runtime pallets. Our aim is to provide features, that address all of the above stated vulnerability classes. See also [Limitations](#Limitations). | -| 2. | Vulnerability Documentation | For each of the above stated vulnerability classes we will document, how this vulnerability is covered by our checks and limitations thereof. | -| 3. | Engagement | Engage with teams at Web3 Foundation and Parity to prioritise vulnerability classes. | - -#### Limitations -The possibilities of a static code analysis are limited. On the one hand, the checks can generate many false positives. On the other hand, it might not be possible to catch all vulnerabilities within a class of vulnerabilities. Because of this we will document the limits of the tool and closely cooperate with the teams at Web3 Foundation and Parity to prioritize the features and their implementation. - ## Future Plans The next steps for the tool would be to: -* improve the usability, by providing - * means to surpress warnings - * a comprehensive user tutorial, incl. documentation on the risks of each vulnerability -* add more features including checks on the following vulnerability classes: - * tracking bad randomness: ensure bad randomness does not leak into sensitive functions. - * detect panics statically to avoid potential DoS attacks: these include [unsafe arithmetic operations](https://github.com/bhargavbh/MIRAI/blob/main/substrate_examples/arithmetic-overflow/description.md), access outside bounds, assertion failures, etc. - * tracking unsanitised input leakage for sensitive functions. +1. Implement a first simple version of the tool, together with tests and documentation. +2. Improve the usability, by providing + * means to surpress warnings + * a comprehensive user tutorial, incl. documentation on the risks of each vulnerability +3. Add more features including checks on the following vulnerability classes: + * tracking bad randomness: ensure bad randomness does not leak into sensitive functions. + * detect panics statically to avoid potential DoS attacks: these include [unsafe arithmetic operations](https://github.com/bhargavbh/MIRAI/blob/main/substrate_examples/arithmetic-overflow/description.md), access outside bounds, assertion failures, etc. + * tracking unsanitised input leakage for sensitive functions. Once we have a tool with a good feature set and basic usability features, we want to promote it to the Polkadot developers. Once the tool is in use, we hope to receive feedback on further features and improvements by the developers.