investigate module simulations

[LaurentMontBlanc]

No description provided.

[jmastr]

the docs can be found here: https://docs.cosmos.network/v0.47/build/building-modules/simulator

[jmastr]

in my opinion PR #490 tries to implement: https://docs.cosmos.network/v0.47/build/building-modules/depinject, which is not about module simulation

[jmastr]

fuzzy testing was introduced on Jun 17th, 2024: cosmos/cosmos-sdk#20490

we currently don't have this patch

[jmastr]

the commit above adds two Makefiles to be able to execute make test-sim-import-export. one thing i noticed is on a very early commit like 7810d08 the result looks like this:

...
exporting app state...
exporting simulation params...

LevelDB Stats
Compactions
 Level |   Tables   |    Size(MB)   |    Time(sec)  |    Read(MB)   |   Write(MB)
-------+------------+---------------+---------------+---------------+---------------
   0   |          0 |       0.00000 |       0.19288 |       0.00000 |      18.94249
   1   |         10 |      18.37142 |       1.18400 |      72.93294 |      72.36187
-------+------------+---------------+---------------+---------------+---------------
 Total |         10 |      18.37142 |       1.37689 |      72.93294 |      91.30436

LevelDB cached block size 8380916
exporting genesis...
importing genesis...
comparing stores...
compared 0 different key/value pairs between KVStoreKey{0xc0006bba10, acc} and KVStoreKey{0xc008d6bbc0, acc}
compared 0 different key/value pairs between KVStoreKey{0xc0006bba40, staking} and KVStoreKey{0xc008d6bbf0, staking}
compared 0 different key/value pairs between KVStoreKey{0xc0006bba80, slashing} and KVStoreKey{0xc008d6bc30, slashing}
compared 0 different key/value pairs between KVStoreKey{0xc0006bba60, mint} and KVStoreKey{0xc008d6bc10, mint}
compared 0 different key/value pairs between KVStoreKey{0xc0006bba70, distribution} and KVStoreKey{0xc008d6bc20, distribution}
compared 0 different key/value pairs between KVStoreKey{0xc0006bba30, bank} and KVStoreKey{0xc008d6bbe0, bank}
compared 0 different key/value pairs between KVStoreKey{0xc0006bbaa0, params} and KVStoreKey{0xc008d6bc50, params}
compared 0 different key/value pairs between KVStoreKey{0xc0006bba90, gov} and KVStoreKey{0xc008d6bc40, gov}
compared 0 different key/value pairs between KVStoreKey{0xc0006bbae0, evidence} and KVStoreKey{0xc008d6bc90, evidence}
compared 0 different key/value pairs between KVStoreKey{0xc0006bbb10, capability} and KVStoreKey{0xc008d6bcc0, capability}
compared 0 different key/value pairs between KVStoreKey{0xc0006bba20, authz} and KVStoreKey{0xc008d6bbd0, authz}
--- PASS: TestAppImportExport (10.77s)
PASS
ok      planetmint-go/app       10.843s

on tag v0.5.5:

...
exporting app state...
exporting simulation params...

LevelDB Stats
Compactions
 Level |   Tables   |    Size(MB)   |    Time(sec)  |    Read(MB)   |   Write(MB)
-------+------------+---------------+---------------+---------------+---------------
   0   |          0 |       0.00000 |       0.27573 |       0.00000 |      24.26920
   1   |         12 |      23.50313 |       1.57588 |     117.43797 |     116.67189
-------+------------+---------------+---------------+---------------+---------------
 Total |         12 |      23.50313 |       1.85161 |     117.43797 |     140.94109

LevelDB cached block size 8387444
I[2024-12-09|13:27:52.560] exporting genesis...

I[2024-12-09|13:27:52.865] importing genesis...

I[2024-12-09|13:27:53.987] comparing stores...

I[2024-12-09|13:27:54.015] compared %d different key/value pairs between %s and %s
 0="KVStoreKey{0xc000c00dc0, acc}" KVStoreKey{0xc007b428c0,acc}=(MISSING)
I[2024-12-09|13:27:54.022] compared %d different key/value pairs between %s and %s
 0="KVStoreKey{0xc000c00df0, staking}" KVStoreKey{0xc007b428f0,staking}=(MISSING)
I[2024-12-09|13:27:54.025] compared %d different key/value pairs between %s and %s
 0="KVStoreKey{0xc000c011b0, slashing}" KVStoreKey{0xc007b42930,slashing}=(MISSING)
I[2024-12-09|13:27:54.025] compared %d different key/value pairs between %s and %s
 0="KVStoreKey{0xc000c00e10, mint}" KVStoreKey{0xc007b42910,mint}=(MISSING)
I[2024-12-09|13:27:54.028] compared %d different key/value pairs between %s and %s
 0="KVStoreKey{0xc000c011a0, distribution}" KVStoreKey{0xc007b42920,distribution}=(MISSING)
I[2024-12-09|13:27:54.037] compared %d different key/value pairs between %s and %s
 0="KVStoreKey{0xc000c00de0, bank}" KVStoreKey{0xc007b428e0,bank}=(MISSING)
I[2024-12-09|13:27:54.037] compared %d different key/value pairs between %s and %s
 0="KVStoreKey{0xc000c011d0, params}" KVStoreKey{0xc007b42950,params}=(MISSING)
I[2024-12-09|13:27:54.043] compared %d different key/value pairs between %s and %s
 0="KVStoreKey{0xc000c011c0, gov}" KVStoreKey{0xc007b42940,gov}=(MISSING)
I[2024-12-09|13:27:54.043] compared %d different key/value pairs between %s and %s
 0="KVStoreKey{0xc000c01210, evidence}" KVStoreKey{0xc007b42990,evidence}=(MISSING)
I[2024-12-09|13:27:54.043] compared %d different key/value pairs between %s and %s
 0="KVStoreKey{0xc000c01240, capability}" KVStoreKey{0xc007b429c0,capability}=(MISSING)
I[2024-12-09|13:27:54.055] compared %d different key/value pairs between %s and %s
 0="KVStoreKey{0xc000c00dd0, authz}" KVStoreKey{0xc007b428d0,authz}=(MISSING)
--- PASS: TestAppImportExport (18.58s)
PASS
ok      github.com/planetmint/planetmint-go/app 18.662s

and on a very recent tag like v0.12.12 it looks like this:

...
exit status 1
[W3] Seed 7: FAILED
To reproduce run: go test . -run TestAppImportExport -Enabled=true -NumBlocks=50 -Genesis= -Verbose=true -Commit=true -Seed=7 -Period=5 -ExportParamsPath /tmp/sim-logs-1048919952/sim_params-7.json -ExportStatePath /tmp/sim-logs-1048919952/sim_state-7.json -v -timeout 24h
ERROR OUTPUT

exit status 1
panic: halting simulations

goroutine 38 [running]:
main.worker(0x3, 0xc0002207e0, 0xc000114000)
        /go/pkg/mod/github.com/cosmos/tools/cmd/[email protected]/main.go:216 +0x825
main.main.func2(0x0?)
        /go/pkg/mod/github.com/cosmos/tools/cmd/[email protected]/main.go:160 +0x4b
created by main.main in goroutine 1
        /go/pkg/mod/github.com/cosmos/tools/cmd/[email protected]/main.go:158 +0xe3b
make: *** [Makefile:285: test-sim-import-export] Error 2

this is just an observation and it needs further digging. just wanted to share this finding.

[jmastr]

the commit breaking the test-sim-import-export is:

a38fe78 187 implement rddl claim (#298)