DINFRA Application

[rvalle]

Project Abstract

DINFRA, decentralized infrastructure, is substrate orchestrated infrastructure, a decentralized alternative to public clouds.

Grant level

• Level 1: Up to $10,000, 2 approvals
• Level 2: Up to $30,000, 3 approvals
• Level 3: Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

• The application template has been copied and aptly renamed (project_name.md).
• I have read the application guidelines.
• Payment details have been provided (bank details via email or BTC, Ethereum (USDC/DAI) or Polkadot/Kusama (USDT) address in the application).
• The software delivered for this grant will be released under an open-source license specified in the application.
• The initial PR contains only one commit (squash and force-push if needed).
• The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
• I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @_______:matrix.org (change the homeserver if you use a different one)

[CLAassistant]

All committers have signed the CLA.

[Noc2]

Thanks for the application. I have a couple of initial questions:

• Are you planning to release your own token? If so, it might make sense to apply for the SBP. If not, did you consider applying for treasury funding with this? With our grants program, we focus more and more on teams that are unfamiliar with our ecosystem and ask teams that are familiar with the treasury to apply there for funding.
• Could you add more technical details to the milestones? For example, the programming language and functionality of the Reactor
• We usually don’t fund the deployment-related deliveries because they are not useful for others, e.g., regarding “DINFRA Test Parachain” or “ACS Test Resources”
• In general, 75k for three months and 1 FTE seems really expensive to me.

[rvalle]

@Noc2

I am happy to follow your advise and/or incorporate your feedback, but I will share how I have thought this application through first:

• DINFRA's first phase, this project, is an MVP which is crucial for gaining community support. I have thought in the w3f because deliverables all well reviewed, funds are paid post execution, and I sense the community understands it as a quality stamp. The second phase, bringing DINFRA to production, will be and order of magnitude bigger in effort, that I plan to take to the treasury after successful completion of first phase with w3f.

• We believe we would benefit for SBP, and there will be a token created in next phase, however, we think of DINFRA as a common good parachain. I think it would be difficult have acceptance if DINFRA was an independent venture.

• With regards to pricing. The configuration of the team is atypical for an MVP. We have chosen 2 very senior profiles for this first phase, but at the same time, we made this first phase as short as possible. Apart of taking the right set of decisions from the start, it will be us who raise awareness and gain community support. This is not only for the Substrate community but also for OpenSource infrastructure community, thus I am bringing top talent in a effort to kick-start the DINFRA community with best possible chances. In any case we have priced the project as a whole and our estimation is more 4-6 months involvement before the next phase, despite initially targeting the engineering to be delivered in 3 months of effort.

• Infrastructure is necessary until the next phase starts because this is an MVP to be demoed and allow the target audience to experiment and decide if they want to join our effort for the next phase. Just like substrate needs a testing rely chain. Selling the DINFRA concept to infrastructure players without having something to play with would be impossible. We will need to record the VIDEO and let the teams play/develop with the deliverables, considering or even testing quickly how easy it would be for them to integrate their infrastructure with DINFRA. We will pitch them, help them, and hopefully in the next phase there will be more parties of the ecosystem involved.

• I am happy to add more details for the Reference Implementation of the Reactor, sure. Will also review the rest of deliverables.

I can see you also in Polkadot Direction channel, I will also ping you there.

[semuelle]

Wouldn't providers have their own signup/subscribe flow based on their jurisdiction and legal requirements? Also, I imagine it might be rather complicated to map all the possible complexities of payment plans.

[rvalle]

These are interesting points that we would address on subsequent phases. But we are certainly considering related functionality. For the MVP we will focus on end-to-end concept proving functionality.

If DINFRA can sing-up infrastructure consumer with one click away payment method already setup, this is tremendous value for Providers, just like an AppStore is for a mobile platform.

Down the road We have considered the possibility that Consumers and Providers can be matched for example by Regional Preferences, availability and budget.

For example consider a Contract of Type: Substrate Node (with a know preimage). A consumer could issue on-chain request for 4 instances each with regional preferences and max (fixed) budget. And the Network match to the best available providers matching the criteria.

Of course we could introduce consumption based variable pricing, but I think we can go a long way with fixed pricing and contract upgrades/downgrades, this simplifies a lot. Most providers in our community work with fixed contracts or tiers by usage.

I don't think DINFRA should get involved in legal requirements, but perhaps implement on-chain terms and conditions acceptance for Providers to become eligible, without understanding what the documents actually contain necessarily.

We have been thinking on things like Topology Oracles, that would put on chain regional and network characteristics of Providers (or their declared availability zones) and attestations that would put on chain service disruptions for implementation of penalties, or automated multiprovider migration of contracts... as similar advanced use cases.

There are Lots of possibilities once we start arranging deployments on chain. But it will be key to add features in the right order. 20% of functionality will surely provide 80% of the benefit....

In our opinion the MVP must be used to show the community that infrastructure can follow what the chain commands, that it is quite doable, feasible, and that it opens tones of possibilities.

In the second phase we should go for the 20% of functionality that delivers the 80% of value that our community actually needs and will use.

[Noc2]

Sorry for the late reply here. I have a few follow-up questions:

• Could you explain the following: “We believe we would benefit for SBP, and there will be a token created in next phase, however, we think of DINFRA as a common good parachain. I think it would be difficult have acceptance if DINFRA was an independent venture.”? Usually, common good parachains don’t have their own token.
• As I said before, I would at least remove the last milestone: “Integration Testing and Test Chain”, since it’s not something that we support. Also, the registration on Rococo is up to parity and not something that we can control
• The first milestone currently doesn’t have any polkadot/substrate-specific deliveries as part of the milestone. Could you add these?
• In general, my recommendation would be to significantly reduce the grant amount to a level 2 grant for an initial PoC (stamp of approval) and apply for the SBP program at the same time.

[rvalle]

@Noc2

I think I understand your point:

• We believe DINFRA needs a token from a functional point of view. We are planing features such as Hardware Tokenisation to facilitate CapEx for providers, or Tokenizing idle infrastructure to fund a treasury and help projects, etc. I understand your concern that a Token may generate rejection from the community following our discussion, but I really think that the only viable token would belong to the community, it would be created by the treasury on its conditions, and administer by the community the same way that our Treasuries work. I don't think DINFRA can survive on its own, I think it has to become a community tool. We have provided examples of projects that chose a different path. DINFRA will need community support/approval. Alternates solutions such as the IBP are generating concern for centralizing treasury resources, DINFRA would provide means for Treasuries to administer infrastructure in a Decentralized way. Of course non of this would work if the DINFRA team hijacks the token or offers it to investors, that's why we openly manifest we will not and DINFRA will be common good.

• We can move the public testing of DINFRA from this project to the first Treasury proposal. We initially planned to go to the treasury with Infrastructure providers support, for which a public test net is useful. In the worst case scenario we would have to introduce a public testing phase on treasury before launching the project for GA, which is doable.

• After removing the Public testing from this project the funding will be adjusted. I Will reduce accordingly but not to a point in which I start damaging project viability. Let me look into it.

• The deliverables form both phases are either Substrate or Substrate infrastructure, please note that they have been made generic as engineering "good practice". Yes, the Deployment descriptors could hold anything, but we know we are targeting Substrate infrastructure. I will however try to reflect your feedback in the development roadmap, an obvious way would be to provide Deployment descriptors for node deployments, for example.

I will come back to you with changes...

[rvalle]

@Noc2 @semuelle

See updated,

I have tried to address all the feedback expressed by @Noc2 in the application while trying not to compromise viability of the project.

[semuelle]

Thanks for the update. LGTM.

[rvalle]

@Noc2 @semuelle I am a bit lost in this process. What is the next step?
Are you waiting on any additional input?
I may have not used the github workflow features as expected, perhaps.

[Noc2]

Sorry for the delay here. I'm happy to go ahead with it and will share the application with the rest of the team.

[keeganquigley]

Thanks for the application @rvalle and for answering all questions. I think this is a good use case for common good, non-opinionated use for substrate and with your previous track record I am happy to approve as well.

[randombishop]

@rvalle Thank you for this application, very interesting use case!
I am happy to approve it.
Also have a question about the subscription/payment system. How is pricing managed and how does the framework ensure that the consumer has enough funds to pay for the services. Centralized providers typically hold credit cards, manage quotas, and there's an underlying trust that the consumer will be billed their exact consumption after the fact. Feels like a hard problem to solve in a trustless framework no?

[rvalle]

Thanks for your support @randombishop.

With regards to the use case, I agree with you it is interesting. Orchestrating multiple players using substrate for the goal of decentralization could be a killer use case for our community, in particular for industries that already enjoy certain degree of automation. Substrate could essentially offer alternatives to Uber or Airbnb, for example. Substrate could trigger a rebirth of the good and old cooperatives in a decentralized self governed fashion.

With regards to pricing (talking in general, not just this MVP), substrate would have the responsibility to award deployment contracts to providers. Pricing could be one of the factors for selection (i.e. max price allowed), among others such as geographical preferences (i.e. non EU) or network topology preferences. (i.e. banned networks AWS and Google Cloud), and even a list of preferred providers. Ultimately, It must be possible to use on-chain consensus in the award process so that participants understand the process is fair. It could also be possible to resort to other more generic parameters such as available capacity, reputation metrics or participation in governance. At the same time the weight of those metrics in the award process should be subject of governance.

Ensuring contracts are funded, in case of fix rate deployment contracts would work as follows:

• An infrastructure consumer issues a Deployment Descriptor for deployment, and reserves an arbitrary amount RA of her choice.
• The network awards the contract at block N with fix cost C. Cost per block can be therefore calculated, CB.
• Infrastructure provider declares the deployment was successful at block NB.
• From then on, without any need for further transactions the network knows how the reserved amount RA should be split between the provider and consumer as a function of the current block number, the cost per block CB, and the deployment block NB.
• The network also knows at which block number the deployment contract simply expires, when RA is projected to be fully spent.
• The consumer can be informed about the date in which the deployment expires via UI.
• The consumer is free to reserve more funds at will, and keep funding her contract, it is also capable to terminate the contract at will by removing unspent funds from RA.
• The provider would also be allowed to claim his part of RA at any time, in particular for long running deployment contracts, or after the contract has expired.

This would provide a very simple implementation of on-chain, variable-length, pay per use of services at fixed cost. And it can easily be complemented with minimum deployment period, minimum cancellation notice period, etc. It can be also complemented with more complex functionality such as witnesses (availability witness) and oracles (topology oracles), slashes, dispute handling, reputation, etc.

From fix pricing we could go to top-ups, as a way to limit price dynamics. For example it is already an industry practice to apply a surcharge for network traffic outside an "included" limit. We believe we could go a long with simple pricing strategies and almost avoid dynamic pricing altogether, perhaps the exception is elastic deployments where infrastructure would be scaled according to demand.

The MVP will implement the fix cost and award process in its most simple possible form, while leaving the door open to enhancements and contributions.

[randombishop]

@rvalle thanks for your additional notes, good luck with your implementation and looking forward to seeing the results.

imo, elastic pay-per-use based on actual after-the-fact consumption is one of the most powerful features of cloud computing, and is particularly handy to onboard new projects. It's way more convenient to start at zero and pay as it scales than commit to servers constantly running at fixed costs. Would highly recommend designing for elastic pay-per-use in your first version rather than later... Could be done in similar ways you're proposing already by adding a layer of escrow and some kind of "trust but verify" the billing.

Anyways, just my 2 cents...

[rvalle]

@randombishop Thanks for the feedback, we will raise priority to pay per use.

[rvalle]

@randombishop I have been thinking a bit more about this, and I believe you are right. In fact, re-setting a new price for a deployment while running is quite easy... this is consistent, for example with updating VM settings in a cloud UI. That is simple enough. In a fixed contract the consumer has to sign (explicit agreement) to pricing change. In a dynamic contract, the provider is granted capability to do it unilaterally.... without requiring consumer signature, thus opening the possibility to auto-scaling.

Furthermore, it would be possible to allow for dynamic changes within boundaries, and in future implementations witnesses could be involved in the auto scaling process. We definitely should be able to plan for this feature early.

[dsm-w3f]

Thank you for the application. The proposal seems to be very interesting and useful. The experience of the team also gives confidence that will be delivered. I'm happy to support it.

[github-actions]

Congratulations and welcome to the Web3 Foundation Grants Program! Please refer to our Milestone Delivery repository for instructions on how to submit milestones and invoices, our FAQ for frequently asked questions and the support section of our README for more ways to find answers to your questions.

Before you start, take a moment to read through our announcement guidelines for all communications related to the grant or make them known to the right person in your organisation. In particular, please don't announce the grant publicly before at least the first milestone of your project has been approved. At that point or shortly before, you can get in touch with us at [email protected] and we'll be happy to collaborate on an announcement about the work you’re doing.

Lastly, please remember to let us know in case you run into any delays or deviate from the deliverables in your application. You can either leave a comment here or directly request to amend your application via PR. We wish you luck with your project! 🚀

[rvalle]

@Noc2 @semuelle @keeganquigley @randombishop @dsm-w3f

Many thanks for your support and feedback!

[keeganquigley]

Hi @rvalle just checking in to see how milestone 1 is coming along?

[rvalle]

Hi @keeganquigley, making progress. Expecting to post soon.

Got delayed by external to the project events, and earlier than usual holidays in the team. Nothing inherent to the project itself. Should be fine going forward.

Thanks for following up.

[rvalle]

@keeganquigley polishing now, hoping to file before end of week.

[keeganquigley]

Thanks for the update @rvalle sounds good!

[rvalle]

Hi everybody, @keeganquigley ... we are making good progress.

We are targeting to deliver over the next 2 weeks.

Experiencing delays in time related to sub0, confusion over new SDK distribution, and our own learning curve. Effort invested in the project is in line with the planing.

We are enjoying the project and Substrate's potential and the results we are getting.

Will keep you posted.

[rvalle]

Hi Again, @keeganquigley , @Noc2... now starting to wrap up the delivery,

Yes, substrate/rust has a steep learning curve many mention, but we perceive the benefit as a game changer.

Will keep you posted.