Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic contract metadata proposal #3

Closed
wants to merge 10 commits into from
118 changes: 118 additions & 0 deletions text/0000-contract-metadata.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
- Proposal Name: `contract_metadata`
MaksymZavershynskyi marked this conversation as resolved.
Show resolved Hide resolved
- Start Date: (fill me in with today's date, YYYY-MM-DD)
- NEP PR: [nearprotocol/neps#0003](https://github.com/nearprotocol/NEPs/pull/3)

# Summary
[summary]: #summary

This NEP introduces contract metadata, which summarizes the content of a given contract in json format.
Contract metadata allows developers to easily list the methods in a contract and potentially display them to users to
provide transparency.

# Motivation
[motivation]: #motivation
Copy link

@janedegtiareva janedegtiareva Jul 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to also include app title in the metadata.
If we look at playstore app submission process, it requires devs to fill in some display info for the app so that it can be listed in various UI contexts. For us, this might mean displaying the name of the app in the wallet for authorization, and/or NEAR app listing. This data may need to be localized if we want to support multiple languages in the future. Google in fact provides easy integration with their translation service in case the developer is unable to translate in-house.

For more info on play store app submission process see https://themanifest.com/app-development/how-publish-app-google-play-step-step-guide -- see product details section.

Copy link

@DanielRX DanielRX Aug 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds very much to me as following on from https://github.com/ethereum/EIPs/blob/master/EIPS/eip-712.md With the idea of being able to add domain specific values (like app title) in a way that humans can verify it. Would it make sense to include all of this on contract, or out of band / on a separate part of the protocol?

EDIT: What would an exhaustive list of keys be that would be needed for every app, but not so large as to mean the majority of apps have "random_hex_string" as a placeholder to satisfy the requirements?


Currently there is no direct way for a developer to programmatically list the methods of a contract that is deployed on chain
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically, this is not correct. If names were mangled host wouldn't be able to call the method from Wasm. The argument names might be mangled though.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the names stored in a table or are they in a header file, or something else? If the names and types and offsets of all functions are programmatically available already, the metadata wouldn't need any of that?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I leave this research to @bowenwang1996 . I just know for a fact that names are not mangled, because host calls functions in Wasm by their names.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the names are stored anywhere. Only the function names are available. Also because wasm only has i32, i64, f32, and f64, all the type information is erased.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should figure out it for a fact, for instance, find it in the Wasm spec, or ask it from someone developing Wasm or Wasm-related infra (you can ask Max).

Also because wasm only has i32, i64, f32, and f64, all the type information is erased.

I don't see how this:

Also because wasm only has i32, i64, f32, and f64

implies this:

all the type information is erased.

I suggest we do not do own our deduction, and either find it in the Wasm spec or talk to the Wasm team.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate on this link?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, did you look into how Wasm preserves the names of the functions? I am pretty sure they are not mangled, otherwise we would not be able to call functions by name from Wasmer.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link says the parameter of a function takes valtype, which is one of i32, i64, f32, f64.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I see now that argument names are not preserved. Thank you. Could you do now the research on how function names are handled in Wasm?

because the contract code stored on chain is the compiled wasm code, where the methods name and parameters are already mangled.
As a result, if developers want to display the methods of a contract to an user of the app to provide transparency,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed with @bowenwang1996 before I think the only real application of the contract metadata is input validation. The case when front-end is somehow implemented to not have hardcoded dependencies on method or argument names has several issues:

  • As discussed, it is very unlikely that front-end developer will somehow use a contract method without using the name of the method anywhere in their code (because they at least need to identify it) so if the contract method name changes they would have to manually modify the code anyway;
  • It opens a huge security concern and such usage should be in general discouraged. We need to have a section in this NEP and documentation explaining the following example: Suppose someone malicious implements a contract with method burn(gas_in_thousands: u64) and lets other developers to use it in their front-ends. Then malicious contract developer changes the signature of the function to be burn(gas: u64) without announcing to anyone. If front-end did not break because it has automatically rebound to the the argument name the users now are burning 1000 more gas without having any notification. The core problem with metadata is that it does not describe semantics, it describes syntax

Copy link

@lexfrl lexfrl Nov 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed with @bowenwang1996 before I think the only real application of the contract metadata is input validation.

As well as it allows to generate a wrapper which would automatically encode/decode ABI (which could be considered as an input validation too).

especially in the case of financial apps, they have no choice but to hardcode them in the front end, which is suboptimal in many ways.
Furthermore, if developers want to get the list of methods in some contract for some downstream task like data analysis,
they have no way of doing so. Contract metadata aims to solve the aforementioned problems by providing a convenient way
for developers to list the methods of contracts.

# Guide-level explanation
MaksymZavershynskyi marked this conversation as resolved.
Show resolved Hide resolved
[guide-level-explanation]: #guide-level-explanation

For developers, there will be two main changes:
- Instead of annotating view and change methods in the `initContract` function in `main.js`,
they will instead annotate the methods of a contract by decorators.
More specifically, every method is by default a change method, unless annotated by `@view_method`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest that all our methods are callable as near call and near view independently on whether they change the state or not. We should remove the current constraint the prohibits calling near view on methods that mutate the state. There are not safety concerns to have this constraint. @evgenykuzyakov @ilblackdragon

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not about safety, it's about developer experience. If I call the function and it didn't purist state change silently I won't know WTF is going on.

Copy link
Contributor

@MaksymZavershynskyi MaksymZavershynskyi Jul 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you are calling near view then you know that it has no permanent effect on the state, because you read near view --help before calling it :)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On this point, does a view contract function have the restriction that it can't call a non-view function?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Right now if you call a state-changing function within a view function, the execution will fail.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the proposed change by max, that will not error but the state will not be changed instead.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two should be consistent though? Either they both error or neither error?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think there is some advantage of being explicit. The annotation serves as documentation for developers who might interact with the contract and with metadata, the list of view methods can be easily extracted and displayed, which is better, in my opinion, than looking at the source code and tries to understand for each function, whether it changes the state or not.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From today's hackathon, the distinction between view and change methods should be either strictly enforced (and extensively documented), or their functionality should be merged into a single kind. Otherwise, it gets really confusing and error-prone.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay it seems like the distinction is confusing. I think we should eliminate the difference and allow dry run of change methods. One thing that is not clear to me is how we should support cross contract view calls. It seems that if you pass 0 gas, the instantiation of wasm module for the call on the second contract will fail.

- Every contract will have a `metadata` method that returns a json that serializes the contract methods. For each method,
the json serialization is of the form `{"parameters": {<param1>: <type1>, .. }, "returnType": <return_type>}`.
The overall serialization is of the form `{"view_methods": {<method_name>: <method_metadata>, .. }, "change_methods": {<method_name>: <method_metadata>, .. }}`.

As an concrete example, suppose we have a contract that maintains a counter on chain:
bowenwang1996 marked this conversation as resolved.
Show resolved Hide resolved

```typescript
import { context, storage, near } from "./near";

export function incrementCounter(): void {
let newCounter = storage.get<i32>("counter") + 1;
storage.set<i32>("counter", newCounter)
near.log("Counter is now: " + newCounter.toString());
}

export function decrementCounter(): void {
let newCounter = storage.get<i32>("counter") - 1;
storage.set<i32>("counter", newCounter)
near.log("Counter is now: " + newCounter.toString());
}

@view_method
export function getCounter(): i32 {
return storage.get<i32>("counter");
}
```

This contract has two change methods, `incrementCounter` and `decrementCounter`, as well as one view method, `getCounter`.
In this case, the metadata we want looks like
```json
bowenwang1996 marked this conversation as resolved.
Show resolved Hide resolved
{
"view_methods":
bowenwang1996 marked this conversation as resolved.
Show resolved Hide resolved
{
"getCounter": {
"parameters": [],
"returnType": "i32"
}
},
"change_methods":
bowenwang1996 marked this conversation as resolved.
Show resolved Hide resolved
{
"incrementCounter": {
bowenwang1996 marked this conversation as resolved.
Show resolved Hide resolved
"parameters": [],
"returnType": "void"
},
"decrementCounter": {
"parameters": [],
"returnType": "void"
}
}
}
```
and the generated `metadata` method looks like:
```typescript
export function metadata(): string {
return "{\"view_methods\": {\"getCounter\": {\"parameters\": {}, \"returnType\": \"i32\"},\"change_methods\": {\"incrementCounter\": {\"parameters\": {}, \"returnType\": \"void\"}}, {\"decrementCounter\": {\"parameters\": {}, \"returnType\": \"void\"}}}"
bowenwang1996 marked this conversation as resolved.
Show resolved Hide resolved
}
```

# Reference-level explanation
[reference-level-explanation]: #reference-level-explanation

To implement this NEP, we just need to modify the binding generation to generate a method called `metadata` that returns
json serialization of contract metadata described in the previous section. This involves walking through the exported methods
in `main.ts`, get the metadata of each method, and serialize them in json. Metadata of a given method, including decorators,
are easily extractable from the assemblyscript IR and serialized into json format.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In Rust we would need to augment the procedural macro to inject the metadata method.

# Drawbacks
[drawbacks]: #drawbacks

The main drawback of this NEP is that it involves generating one more method during the binding generation phase, which
will in turn increase the size of each contract.

# Rationale and alternatives
[rationale-and-alternatives]: #rationale-and-alternatives

It is unclear to me what the alternative is.

# Unresolved questions
[unresolved-questions]: #unresolved-questions

* What other information, besides those mentioned in [guide-level-explanation], should we include in the metadata?
* Most of this NEP is concerned with contract metadata for assemblyscript, for the rust API, it is not yet unclear what
needs to be done given that it is not yet stabilized.

# Future possibilities
[future-possibilities]: #future-possibilities

Under the framework proposed in this NEP, it is also not difficult to add annotations to methods in natural language.
We can also add contract-level annotation as part of the contract metadata json.