Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SmtSpec for SMT proofs #57

Merged
merged 7 commits into from
Feb 1, 2022
Merged

Add SmtSpec for SMT proofs #57

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
df4bb4f
fix empty branch check
roysc Feb 1, 2022
c60c048
add SmtSpec
roysc Nov 22, 2021
05d4f03
add testdata/smt
roysc Dec 12, 2021
c02d029
add smt to vector tests
roysc Dec 13, 2021
7a38155
rust add smt spec
roysc Dec 15, 2021
ac18e04
rust port smt vector tests
roysc Dec 15, 2021
01524ae
js smt spec
roysc Dec 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 38 additions & 12 deletions go/proof.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,26 @@ var TendermintSpec = &ProofSpec{
},
}

// SmtSpec constrains the format for SMT proofs (as implemented by github.com/celestiaorg/smt)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to have a test case (even json file, table test... something generated by a real SMT that we can test against).

Let's verify this works with your SMT proofs.

Copy link
Contributor Author

@roysc roysc Nov 24, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Originally this was in the cosmos-sdk repo, but I moved it here to keep a single source of truth.

Test cases are here, in the code for the sdk PR. I don't have one with a failed verification yet, but I will add that. Should I add any tests within this repo?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add some basic tests in this repo.
Happy to have it here, just tested here as well

var SmtSpec = &ProofSpec{
LeafSpec: &LeafOp{
Hash: HashOp_SHA256,
PrehashKey: HashOp_NO_HASH,
PrehashValue: HashOp_SHA256,
Length: LengthOp_NO_PREFIX,
Prefix: []byte{0},
},
InnerSpec: &InnerSpec{
ChildOrder: []int32{0, 1},
ChildSize: 32,
MinPrefixLength: 1,
MaxPrefixLength: 1,
EmptyChild: make([]byte, 32),
Hash: HashOp_SHA256,
},
MaxDepth: 256,
}

// Calculate determines the root hash that matches a given Commitment proof
// by type switching and calculating root based on proof type
// NOTE: Calculate will return the first calculated root in the proof,
Expand Down Expand Up @@ -216,7 +236,7 @@ func IsLeftMost(spec *InnerSpec, path []*InnerOp) bool {

// ensure every step has a prefix and suffix defined to be leftmost, unless it is a placeholder node
for _, step := range path {
if !hasPadding(step, minPrefix, maxPrefix, suffix) && !leftBranchesAreEmpty(spec, step, 0) {
if !hasPadding(step, minPrefix, maxPrefix, suffix) && !leftBranchesAreEmpty(spec, step) {
return false
}
}
Expand All @@ -230,7 +250,7 @@ func IsRightMost(spec *InnerSpec, path []*InnerOp) bool {

// ensure every step has a prefix and suffix defined to be rightmost, unless it is a placeholder node
for _, step := range path {
if !hasPadding(step, minPrefix, maxPrefix, suffix) && !rightBranchesAreEmpty(spec, step, int32(last)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good to remove

if !hasPadding(step, minPrefix, maxPrefix, suffix) && !rightBranchesAreEmpty(spec, step) {
return false
}
}
Expand Down Expand Up @@ -310,12 +330,15 @@ func getPadding(spec *InnerSpec, branch int32) (minPrefix, maxPrefix, suffix int
return
}

// leftBranchesAreEmpty returns true if the padding bytes correspond to all empty children
// on the left side of this branch, ie. it's a valid placeholder on a leftmost path
func leftBranchesAreEmpty(spec *InnerSpec, op *InnerOp, branch int32) bool {
idx := getPosition(spec.ChildOrder, branch)
// leftBranchesAreEmpty returns true if the padding bytes correspond to all empty siblings
// on the left side of a branch, ie. it's a valid placeholder on a leftmost path
func leftBranchesAreEmpty(spec *InnerSpec, op *InnerOp) bool {
idx, err := orderFromPadding(spec, op)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good improvement

if err != nil {
return false
}
// count branches to left of this
leftBranches := idx
leftBranches := int(idx)
if leftBranches == 0 {
return false
}
Expand All @@ -333,12 +356,15 @@ func leftBranchesAreEmpty(spec *InnerSpec, op *InnerOp, branch int32) bool {
return true
}

// rightBranchesAreEmpty returns true if the padding bytes correspond to all empty children
// on the right side of this branch, ie. it's a valid placeholder on a rightmost path
func rightBranchesAreEmpty(spec *InnerSpec, op *InnerOp, branch int32) bool {
idx := getPosition(spec.ChildOrder, branch)
// rightBranchesAreEmpty returns true if the padding bytes correspond to all empty siblings
// on the right side of a branch, ie. it's a valid placeholder on a rightmost path
func rightBranchesAreEmpty(spec *InnerSpec, op *InnerOp) bool {
idx, err := orderFromPadding(spec, op)
if err != nil {
return false
}
// count branches to right of this one
rightBranches := len(spec.ChildOrder) - 1 - idx
rightBranches := len(spec.ChildOrder) - 1 - int(idx)
if rightBranches == 0 {
return false
}
Expand Down
8 changes: 2 additions & 6 deletions go/proof_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,14 +63,10 @@ func TestEmptyBranch(t *testing.T) {
if err := tc.Op.CheckAgainstSpec(tc.Spec); err != nil {
t.Errorf("Invalid InnerOp: %v", err)
}
order, err := orderFromPadding(tc.Spec.InnerSpec, tc.Op)
if err != nil {
t.Errorf("Cannot get orderFromPadding: %v", err)
}
if leftBranchesAreEmpty(tc.Spec.InnerSpec, tc.Op, order) != tc.IsLeft {
if leftBranchesAreEmpty(tc.Spec.InnerSpec, tc.Op) != tc.IsLeft {
t.Errorf("Expected leftBranchesAreEmpty to be %t but it wasn't", tc.IsLeft)
}
if rightBranchesAreEmpty(tc.Spec.InnerSpec, tc.Op, order) != tc.IsRight {
if rightBranchesAreEmpty(tc.Spec.InnerSpec, tc.Op) != tc.IsRight {
t.Errorf("Expected rightBranchesAreEmpty to be %t but it wasn't", tc.IsRight)
}
})
Expand Down
34 changes: 34 additions & 0 deletions go/vectors_data_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ type TestVectorsStruct struct {
func VectorsTestData() []TestVectorsStruct {
iavl := filepath.Join("..", "testdata", "iavl")
tendermint := filepath.Join("..", "testdata", "tendermint")
smt := filepath.Join("..", "testdata", "smt")
cases := []TestVectorsStruct{
{Dir: iavl, Filename: "exist_left.json", Spec: IavlSpec},
{Dir: iavl, Filename: "exist_right.json", Spec: IavlSpec},
Expand All @@ -45,6 +46,12 @@ func VectorsTestData() []TestVectorsStruct {
{Dir: tendermint, Filename: "nonexist_left.json", Spec: TendermintSpec},
{Dir: tendermint, Filename: "nonexist_right.json", Spec: TendermintSpec},
{Dir: tendermint, Filename: "nonexist_middle.json", Spec: TendermintSpec},
{Dir: smt, Filename: "exist_left.json", Spec: SmtSpec},
{Dir: smt, Filename: "exist_right.json", Spec: SmtSpec},
{Dir: smt, Filename: "exist_middle.json", Spec: SmtSpec},
{Dir: smt, Filename: "nonexist_left.json", Spec: SmtSpec},
{Dir: smt, Filename: "nonexist_right.json", Spec: SmtSpec},
{Dir: smt, Filename: "nonexist_middle.json", Spec: SmtSpec},
}
return cases
}
Expand All @@ -69,6 +76,7 @@ type BatchVectorData struct {
func BatchVectorsTestData(t *testing.T) map[string]BatchVectorData {
iavl := filepath.Join("..", "testdata", "iavl")
tendermint := filepath.Join("..", "testdata", "tendermint")
smt := filepath.Join("..", "testdata", "smt")
// Note that each item has a different commitment root,
// so maybe not ideal (cannot check multiple entries)
batchIAVL, refsIAVL := buildBatch(t, iavl, []string{
Expand All @@ -87,10 +95,22 @@ func BatchVectorsTestData(t *testing.T) map[string]BatchVectorData {
"nonexist_right.json",
"nonexist_middle.json",
})
batchSMT, refsSMT := buildBatch(t, smt, []string{
"exist_left.json",
"exist_right.json",
"exist_middle.json",
"nonexist_left.json",
"nonexist_right.json",
"nonexist_middle.json",
})

batchTMExist, refsTMExist := loadBatch(t, tendermint, "batch_exist.json")
batchTMNonexist, refsTMNonexist := loadBatch(t, tendermint, "batch_nonexist.json")
batchIAVLExist, refsIAVLExist := loadBatch(t, iavl, "batch_exist.json")
batchIAVLNonexist, refsIAVLNonexist := loadBatch(t, iavl, "batch_nonexist.json")
batchSMTexist, refsSMTexist := loadBatch(t, smt, "batch_exist.json")
batchSMTnonexist, refsSMTnonexist := loadBatch(t, smt, "batch_nonexist.json")

return map[string]BatchVectorData{
"iavl 0": {Spec: IavlSpec, Proof: batchIAVL, Ref: refsIAVL[0]},
"iavl 1": {Spec: IavlSpec, Proof: batchIAVL, Ref: refsIAVL[1]},
Expand All @@ -114,18 +134,32 @@ func BatchVectorsTestData(t *testing.T) map[string]BatchVectorData {
"tm invalid 2": {Spec: TendermintSpec, Proof: refsTML, Ref: refsIAVL[0], Invalid: true},
"tm batch exist": {Spec: TendermintSpec, Proof: batchTMExist, Ref: refsTMExist[10]},
"tm batch nonexist": {Spec: TendermintSpec, Proof: batchTMNonexist, Ref: refsTMNonexist[3]},
"smt 0": {Spec: SmtSpec, Proof: batchSMT, Ref: refsSMT[0]},
"smt 1": {Spec: SmtSpec, Proof: batchSMT, Ref: refsSMT[1]},
"smt 2": {Spec: SmtSpec, Proof: batchSMT, Ref: refsSMT[2]},
"smt 3": {Spec: SmtSpec, Proof: batchSMT, Ref: refsSMT[3]},
"smt 4": {Spec: SmtSpec, Proof: batchSMT, Ref: refsSMT[4]},
"smt 5": {Spec: SmtSpec, Proof: batchSMT, Ref: refsSMT[5]},
// Note this spec only differs for non-existence proofs
"smt invalid 1": {Spec: IavlSpec, Proof: batchSMT, Ref: refsSMT[4], Invalid: true},
"smt invalid 2": {Spec: SmtSpec, Proof: batchSMT, Ref: refsIAVL[0], Invalid: true},
"smt batch exist": {Spec: SmtSpec, Proof: batchSMTexist, Ref: refsSMTexist[10]},
"smt batch nonexist": {Spec: SmtSpec, Proof: batchSMTnonexist, Ref: refsSMTnonexist[3]},
}
}

func DecompressBatchVectorsTestData(t *testing.T) map[string]*CommitmentProof {
iavl := filepath.Join("..", "testdata", "iavl")
tendermint := filepath.Join("..", "testdata", "tendermint")
smt := filepath.Join("..", "testdata", "smt")
// note that these batches are already compressed
batchIAVL, _ := loadBatch(t, iavl, "batch_exist.json")
batchTM, _ := loadBatch(t, tendermint, "batch_nonexist.json")
batchSMT, _ := loadBatch(t, smt, "batch_nonexist.json")
return map[string]*CommitmentProof{
"iavl": batchIAVL,
"tendermint": batchTM,
"smt": batchSMT,
}
}

Expand Down
19 changes: 19 additions & 0 deletions js/src/proofs.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,25 @@ export const tendermintSpec: ics23.IProofSpec = {
},
};

export const smtSpec: ics23.IProofSpec = {
leafSpec: {
hash: ics23.HashOp.SHA256,
prehashKey: ics23.HashOp.NO_HASH,
prehashValue: ics23.HashOp.SHA256,
length: ics23.LengthOp.NO_PREFIX,
prefix: Uint8Array.from([0]),
},
innerSpec: {
childOrder: [0, 1],
childSize: 32,
minPrefixLength: 1,
maxPrefixLength: 1,
emptyChild: new Uint8Array(32),
hash: ics23.HashOp.SHA256
},
maxDepth: 256,
};

export type CommitmentRoot = Uint8Array;

// verifyExistence will throw an error if the proof doesn't link key, value -> root
Expand Down
128 changes: 128 additions & 0 deletions rust/src/api.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,30 @@ pub fn tendermint_spec() -> ics23::ProofSpec {
}
}

pub fn smt_spec() -> ics23::ProofSpec {
let leaf = ics23::LeafOp {
hash: ics23::HashOp::Sha256.into(),
prehash_key: 0,
prehash_value: ics23::HashOp::Sha256.into(),
length: 0,
prefix: vec![0_u8],
};
let inner = ics23::InnerSpec {
child_order: vec![0, 1],
min_prefix_length: 1,
max_prefix_length: 1,
child_size: 32,
empty_child: vec![0; 32],
hash: ics23::HashOp::Sha256.into(),
};
ics23::ProofSpec {
leaf_spec: Some(leaf),
inner_spec: Some(inner),
min_depth: 0,
max_depth: 0,
}
}

#[cfg(feature = "std")]
#[cfg(test)]
mod tests {
Expand Down Expand Up @@ -359,6 +383,48 @@ mod tests {
verify_test_vector("../testdata/tendermint/nonexist_middle.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_smt_left() -> Result<()> {
let spec = smt_spec();
verify_test_vector("../testdata/smt/exist_left.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_smt_right() -> Result<()> {
let spec = smt_spec();
verify_test_vector("../testdata/smt/exist_right.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_smt_middle() -> Result<()> {
let spec = smt_spec();
verify_test_vector("../testdata/smt/exist_middle.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_smt_left_non() -> Result<()> {
let spec = smt_spec();
verify_test_vector("../testdata/smt/nonexist_left.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_smt_right_non() -> Result<()> {
let spec = smt_spec();
verify_test_vector("../testdata/smt/nonexist_right.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_smt_middle_non() -> Result<()> {
let spec = smt_spec();
verify_test_vector("../testdata/smt/nonexist_middle.json", &spec)
}

#[cfg(feature = "std")]
fn load_batch(files: &[&str]) -> Result<(ics23::CommitmentProof, Vec<RefData>)> {
let mut entries = Vec::new();
Expand Down Expand Up @@ -505,4 +571,66 @@ mod tests {
])?;
verify_batch(&spec, &proof, &data[5])
}

#[test]
#[cfg(feature = "std")]
fn test_vector_smt_batch_exist() -> Result<()> {
let spec = smt_spec();
let (proof, data) = load_batch(&[
"../testdata/smt/exist_left.json",
"../testdata/smt/exist_right.json",
"../testdata/smt/exist_middle.json",
"../testdata/smt/nonexist_left.json",
"../testdata/smt/nonexist_right.json",
"../testdata/smt/nonexist_middle.json",
])?;
verify_batch(&spec, &proof, &data[0])
}

#[test]
#[cfg(feature = "std")]
fn compressed_smt_batch_exist() -> Result<()> {
let spec = smt_spec();
let (proof, data) = load_batch(&[
"../testdata/smt/exist_left.json",
"../testdata/smt/exist_right.json",
"../testdata/smt/exist_middle.json",
"../testdata/smt/nonexist_left.json",
"../testdata/smt/nonexist_right.json",
"../testdata/smt/nonexist_middle.json",
])?;
let comp = compress(&proof)?;
verify_batch(&spec, &comp, &data[0])
}

#[test]
#[cfg(feature = "std")]
fn test_vector_smt_batch_nonexist() -> Result<()> {
let spec = smt_spec();
let (proof, data) = load_batch(&[
"../testdata/smt/exist_left.json",
"../testdata/smt/exist_right.json",
"../testdata/smt/exist_middle.json",
"../testdata/smt/nonexist_left.json",
"../testdata/smt/nonexist_right.json",
"../testdata/smt/nonexist_middle.json",
])?;
verify_batch(&spec, &proof, &data[4])
}

#[test]
#[cfg(feature = "std")]
fn compressed_smt_batch_nonexist() -> Result<()> {
let spec = smt_spec();
let (proof, data) = load_batch(&[
"../testdata/smt/exist_left.json",
"../testdata/smt/exist_right.json",
"../testdata/smt/exist_middle.json",
"../testdata/smt/nonexist_left.json",
"../testdata/smt/nonexist_right.json",
"../testdata/smt/nonexist_middle.json",
])?;
let comp = compress(&proof)?;
verify_batch(&spec, &comp, &data[4])
}
}
2 changes: 1 addition & 1 deletion rust/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ mod verify;

pub use crate::ics23::*;
pub use api::{
iavl_spec, tendermint_spec, verify_batch_membership, verify_batch_non_membership,
iavl_spec, smt_spec, tendermint_spec, verify_batch_membership, verify_batch_non_membership,
verify_membership, verify_non_membership,
};
pub use compress::{compress, decompress, is_compressed};
Expand Down
Loading