ScoutCoinFabrik: Milestone 2

[valeriacaracciolo]

Project Abstract

This is the second milestone of the project ScoutCoinFabrik. The pull request associated to the the first milestone is the following:
#1490

Scout is an extensible open-source tool (or set of tools) to assist Rust Polkadot / Kusama smart contract developers to detect common security issues and deviations from best practices. The objective of this second milestone is the delivery of a prototype tool after the realization of a proof of concept in milestone 1. The repository associated to this project can be found in the following link:
https://github.com/CoinFabrik/web3-grant

Grant level

• Level 1: Up to $10,000, 2 approvals
• Level 2: Up to $30,000, 3 approvals
• Level 3: Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

• The application template has been copied and aptly renamed (project_name.md).
• I have read the application guidelines.
• Payment details have been provided (bank details via email or BTC, Ethereum (USDC/DAI) or Polkadot/Kusama (USDT) address in the application).
• The software delivered for this grant will be released under an open-source license specified in the application.
• The initial PR contains only one commit (squash and force-push if needed).
• The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
• I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @_______:matrix.org (change the homeserver if you use a different one)

[semuelle]

Thank you for the application, @valeriacaracciolo. We will look into it as soon as possible.

[keeganquigley]

Thanks for the follow-up application @valeriacaracciolo, happy to see it. A couple of initial comments:

• Since this is a new grant, it would technically be a single milestone, so you can go ahead and change the label to "Milestone 1" instead of "2" to avoid confusion when it comes time to submit it.
• Are you able to expand a bit (in the application) on what the prototype would look like? Would this just be a tool written in Rust? The more technical details you can provide, the better.
• What kind of functionality will the CLI have?
• Can you elaborate on what functionalities the VSCode extension would provide as well? (e.g. code completion, diagnostic errors, hover content, etc.) Would it be compatible with other ink! extensions such as this one for semantic analysis?

[valeriacaracciolo]

Hello Keegan, we will be updating our application tomorrow, but I just wanted to send over our replies to your inquiries, in case you have more comments:

Since this is a new grant, it would technically be a single milestone, so you can go ahead and change the label to "Milestone 1" instead of "2" to avoid confusion when it comes time to submit it.

Ok. We will modify this in the application file. We will add a note or reference to connect this application to the previous milestone.

Are you able to expand a bit on what the prototype would look like? Would this just be a tool written in Rust? The more technical details you can provide, the better.

We are to build a prototype that can analyze Rust code to detect vulnerabilities in ink! smart contracts and possibly in pallets and other pieces of code. This builds over the proof-of-concept tool [mention] (we've built and delivered as part of a previous Web3 Foundation grant) by: a) Moving from a proof-of-concept (PoC) tool to a robust tool that integrates with a popular IDE (VSCode), includes a CLI, etc, b) We will improve the precision of the detectors we included in the PoC reducing the rate of false positives, and c) We will add more detectors to have reasonable coverage of the relevant security vulnerabilities that happen in smart contracts.

What kind of functionality will the CLI have?

For this prototype, we want to develop a simple command line interface like the one used in other static analyzers from other blockchains (eg: Slither <https://github.com/crytic/slither>, Rustle <https://github.com/blocksecteam/rustle>). In particular, we will develop the possibility to run the prototype on smart contract files or directories. The base command will be: cargo scout file_name.rs We will also include options for running subsets of detectors and triggering errors for CI/CD workflows.

Can you elaborate on what functionalities the VSCode extension would provide as well? (e.g. code completion, diagnostic errors, hover content, etc.) Would it be compatible with other ink! extensions such as this one <https://w3f.github.io/Grants-Program/applications/ink-analyzer#future-plans> for semantic analysis?

Our VSCode development will list security issues, highlight issues with squiggles and hover-over descriptions. We will seek compatibility of this development with other relevant ink! extensions such as Ink! Analyzer <https://w3f.github.io/Grants-Program/applications/ink-analyzer#future-plans>. Thank you so much for all your help Keegan Best,

…

On 13 Apr 2023, at 21:31, Keegan | W3F ***@***.***> wrote: Thanks for the follow-up application @valeriacaracciolo <https://github.com/valeriacaracciolo>, happy to see it. A couple of initial comments: Since this is a new grant, it would technically be a single milestone, so you can go ahead and change the label to "Milestone 1" instead of "2" to avoid confusion when it comes time to submit it. Are you able to expand a bit on what the prototype would look like? Would this just be a tool written in Rust? The more technical details you can provide, the better. What kind of functionality will the CLI have? Can you elaborate on what functionalities the VSCode extension would provide as well? (e.g. code completion, diagnostic errors, hover content, etc.) Would it be compatible with other ink! extensions such as this one <https://w3f.github.io/Grants-Program/applications/ink-analyzer#future-plans> for semantic analysis? — Reply to this email directly, view it on GitHub <#1666 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ARBOA7SUASDPYBUSVYVUE2DXBBIBBANCNFSM6AAAAAAWZLEN3Q>. You are receiving this because you were mentioned.

[keeganquigley]

Thanks for the answers @valeriacaracciolo I will go ahead and mark the application as ready for review so other committee members can comment. Feel free to ping me once the application has been updated.

[keeganquigley]

Thanks for making the changes @valeriacaracciolo I'd be happy to support it.

[Noc2]

I'm happy to go ahead with it as well.

[Noc2]

I will also share the application again with the rest of the team.

[valeriacaracciolo]

Awesome, thank you so much David Best,

…

On 8 May 2023, at 15:48, David Hawig ***@***.***> wrote: I will also share the application again with the rest of the team. — Reply to this email directly, view it on GitHub <#1666 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ARBOA7SD4JCEN5WPBI5T5X3XFD2RLANCNFSM6AAAAAAWZLEN3Q>. You are receiving this because you were mentioned.

[semuelle]

Sorry for the late reply here, @valeriacaracciolo. You did some useful work in your first grant and generally I think it makes sense to continue.

[valeriacaracciolo]

Hello Sebastian! Thanks for your inquiries. We have just replied to both of them. Please let us know if you have further comments. Best,

…

On 8 May 2023, at 17:55, Sebastian Müller ***@***.***> wrote: @semuelle requested changes on this pull request. Sorry for the late reply here, @valeriacaracciolo <https://github.com/valeriacaracciolo>. You did some useful work in your first grant and generally I think it makes sense to continue. In applications/ScoutCoinFabrik_2.md <#1666 (comment)>: > +- **Estimated duration:** 5 weeks +- **FTE:** 5 +- **Costs:** 30,000 USD + +| Number | Deliverable | Specification | +| -----: | ----------- | ------------- | +| 0a. | License | MIT | +| 0b. | Documentation | Documentation hosted on a separate webpage. | +| 0c. | Testing | Integration testing. Specific tests for every linting detector based on code examples and snippets of smart contracts. | +| 0d. | Docker | Does not apply at this stage. | +| 0e. | Article | We will upload a report summary to our blog. | + **1.a** | Research and Development | Vulnerability examples. In addition to the [examples developed in Milestone 1 of ScoutCoinFabrik PoC](https://github.com/w3f/Grants-Program/blob/master/applications/ScoutCoinFabrik.md#milestone-1-proof-of-concept), we will develop more code examples and snippets of vulnerabilities, best practices, and enhancements related to smart contracts written in ink!. | + **1.b** | Research and Development | Further example versions of [vulnerabilities developed in Milestone 1 of ScoutCoinFabrik PoC](https://github.com/CoinFabrik/web3-grant/tree/main/vulnerabilities). This step is geared to provide a wider set of examples, therefore improving our ability to measure the precision of our prototype and any other ink! vulnerability detection tool. | + **2.a** | Development | Building a prototype that improves over the [development of Milestone 1 of ScoutCoinFabrik PoC](https://github.com/w3f/Grants-Program/blob/master/applications/ScoutCoinFabrik.md#milestone-1-proof-of-concept), detecting more classes of vulnerabilities and improving in precision on existing detectors. We are to build a prototype that can analyze Rust code to detect vulnerabilities in ink! smart contracts and possibly in pallets and other pieces of code. This builds over [this proof-of-concept tool](https://github.com/CoinFabrik/web3-grant) we've built and [delivered as part of a grant for the web3 foundation](https://github.com/w3f/Grant-Milestone-Delivery/blob/master/deliveries/ScoutCoinFabrik-1.md) by: <br> a) Moving from a proof-of-concept (PoC) tool to a robust tool that integrates with a popular IDE (VSCode), includes a CLI, etc, <br> b) We will improve on the precision of the detectors we included in the PoC reducing the rate of false positives, and <br> c) We will add more detectors in order to have a reasonable coverage of the relevant security vulnerabilities that happen in smart contracts. | + **2.b** | Development | Command line interface for the prototype. For this prototype, we want to develop a simple command line interface like the one used in other static analyzers from other blockchains (eg: [Slither](https://github.com/crytic/slither), [Rustle](https://github.com/blocksecteam/rustle)). <br> In particular, we will develop the possibility to run the prototype on smart contract files or directories. <br> The base command will be: `cargo scout file_name.rs` <br> We will also include options for running subsets of detectors and triggering errors for CI/CD workflows. | + **2.c** | Development | VSCode integration for the prototype. Our VSCode development will list security issues, highlight issues with squiggles and hover-over descriptions. We will seek compatibility of this development with other relevant ink! extensions such as [Ink! Analyzer](https://w3f.github.io/Grants-Program/applications/ink-analyzer#future-plans). | What do you mean by compatibility with other extensions? In applications/ScoutCoinFabrik_2.md <#1666 (comment)>: > +### Milestone 1: Prototype + +- **Estimated duration:** 5 weeks +- **FTE:** 5 +- **Costs:** 30,000 USD + +| Number | Deliverable | Specification | +| -----: | ----------- | ------------- | +| 0a. | License | MIT | +| 0b. | Documentation | Documentation hosted on a separate webpage. | +| 0c. | Testing | Integration testing. Specific tests for every linting detector based on code examples and snippets of smart contracts. | +| 0d. | Docker | Does not apply at this stage. | +| 0e. | Article | We will upload a report summary to our blog. | + **1.a** | Research and Development | Vulnerability examples. In addition to the [examples developed in Milestone 1 of ScoutCoinFabrik PoC](https://github.com/w3f/Grants-Program/blob/master/applications/ScoutCoinFabrik.md#milestone-1-proof-of-concept), we will develop more code examples and snippets of vulnerabilities, best practices, and enhancements related to smart contracts written in ink!. | + **1.b** | Research and Development | Further example versions of [vulnerabilities developed in Milestone 1 of ScoutCoinFabrik PoC](https://github.com/CoinFabrik/web3-grant/tree/main/vulnerabilities). This step is geared to provide a wider set of examples, therefore improving our ability to measure the precision of our prototype and any other ink! vulnerability detection tool. | + **2.a** | Development | Building a prototype that improves over the [development of Milestone 1 of ScoutCoinFabrik PoC](https://github.com/w3f/Grants-Program/blob/master/applications/ScoutCoinFabrik.md#milestone-1-proof-of-concept), detecting more classes of vulnerabilities and improving in precision on existing detectors. We are to build a prototype that can analyze Rust code to detect vulnerabilities in ink! smart contracts and possibly in pallets and other pieces of code. This builds over [this proof-of-concept tool](https://github.com/CoinFabrik/web3-grant) we've built and [delivered as part of a grant for the web3 foundation](https://github.com/w3f/Grant-Milestone-Delivery/blob/master/deliveries/ScoutCoinFabrik-1.md) by: <br> a) Moving from a proof-of-concept (PoC) tool to a robust tool that integrates with a popular IDE (VSCode), includes a CLI, etc, <br> b) We will improve on the precision of the detectors we included in the PoC reducing the rate of false positives, and <br> c) We will add more detectors in order to have a reasonable coverage of the relevant security vulnerabilities that happen in smart contracts. | What do you mean by prototype? I just see a list of detectors <https://github.com/CoinFabrik/web3-grant/blob/main/detectors/README.md> in the linked repo. — Reply to this email directly, view it on GitHub <#1666 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ARBOA7QG3PX6DVGKHTDKBALXFEJQ7ANCNFSM6AAAAAAWZLEN3Q>. You are receiving this because you were mentioned.

[semuelle]

Since this is called "Evaluation", could you add some kind of success criteria that we could measure the tool's success on? I would actually recommend building or deploying your own contracts to evaluate against. I believe there are collections of ink! vulnerabilities published in some places.

[valeriacaracciolo]

During this period we will select some popular ink! projects, accepting selections form the web3 foundation, and run the tool against these smart contracts. A security professional will then review the vulnerabilities found by the tool, one by one, to decide whether they are false or true positives. We will then report on the tool's precision estimate and analyze if changes can be made to improve precision.

[valeriacaracciolo]

Thank you for your question. Please let us know if you have further comments.

[github-actions]

Congratulations and welcome to the Web3 Foundation Grants Program! Please refer to our Milestone Delivery repository for instructions on how to submit milestones and invoices, our FAQ for frequently asked questions and the support section of our README for more ways to find answers to your questions.

Before you start, take a moment to read through our announcement guidelines for all communications related to the grant or make them known to the right person in your organisation. In particular, please don't announce the grant publicly before at least the first milestone of your project has been approved. At that point or shortly before, you can get in touch with us at [email protected] and we'll be happy to collaborate on an announcement about the work you’re doing.

Lastly, please remember to let us know in case you run into any delays or deviate from the deliverables in your application. You can either leave a comment here or directly request to amend your application via PR. We wish you luck with your project! 🚀