investigate: getState related all regenFns's metric show empty data plus getPreState regenFn job metrics

[g11tech]

[g11tech]

on a cursory glance, seems like queued getState seems to be never called (which is our entrypoint for starting any regen metric).
however will there is a getState in api/impl/beacon/state/index.ts that further calls resolveStateId:

resolveStateId either picks from stateCache, or stateArchive or constructs from stateArchive.
seems like getState is not being called from outside.

[g11tech]

seems like getPreState computation is never getting queued for computation and is getting resolved in cache only.
it does get called from processBlocksInEpoch but gets resolved using cache's only in regen/queued.ts and never further ahead.

here is an effort to look up any observation across last 30 days in a prater running node"

[g11tech]

@dapplion should these metrics panels be removed from regenFn Stats of the metrics dashboard? or should I aggregate all these metrics in a single widget (multiple targets), someday if something shows up we can dig deeper
atleast we can remove the crowding from regenFn Stats

[dapplion]

@dapplion should these metrics panels be removed from regenFn Stats of the metrics dashboard? or should I aggregate all these metrics in a single widget (multiple targets), someday if something shows up we can dig deeper
atleast we can remove the crowding from regenFn Stats

Sounds good do a PR with screen captures and let's see how it looks. If you can make all panels by 2 per row format would be great, there are still 2 full row width panels.

[g11tech]

get preState graphs are now showing up when the node is experiencing the sync failure, which means these graphs can be retained,

will collapse only getState graphs (apart from also resizing the full width graphs into half width)

[dapplion]

@g11tech Why is the cache hit ratio negative sometimes?

[g11tech]

because its an approximation as we skipped counting the queued but are deriving it from queued processing. so assume at sec 1 one gets 10 requests, which gets queued, on second 2 queued length processes those 10, so delta(total) - delta(queued processed) creates these artifacts.

the better solution will be to count queued in the same function scope as where we count total so that there is no drift when we do delta(total) - delta (queued). let me know if you would want me to make this change?