Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add execution per shard stats #3179

Merged
merged 3 commits into from
Apr 9, 2020
Merged

Add execution per shard stats #3179

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7826863
add distribution stats on executions per shard
andrewjdawson2016 Apr 8, 2020
708a8d0
fix NPE
andrewjdawson2016 Apr 8, 2020
e4ec9d2
Merge branch 'master' into addExecutionPerShardStats
meiliang86 Apr 9, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 26 additions & 9 deletions tools/cli/adminDBScanCommand.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,15 +141,16 @@ type (
// ProgressReport contains metadata about the scan for all shards which have been finished
// This is periodically printed to stdout
ProgressReport struct {
NumberOfShardsFinished int
TotalExecutionsCount int64
CorruptedExecutionsCount int64
ExecutionCheckFailureCount int64
NumberOfShardScanFailures int64
PercentageCorrupted float64
PercentageCheckFailure float64
Rates Rates
CorruptionTypeBreakdown CorruptionTypeBreakdown
NumberOfShardsFinished int
TotalExecutionsCount int64
CorruptedExecutionsCount int64
ExecutionCheckFailureCount int64
NumberOfShardScanFailures int64
PercentageCorrupted float64
PercentageCheckFailure float64
Rates Rates
CorruptionTypeBreakdown CorruptionTypeBreakdown
ShardExecutionCountsDistribution ShardExecutionCountsDistribution
}

// CorruptionTypeBreakdown breaks down counts and percentages of corruption types
Expand All @@ -170,6 +171,13 @@ type (
ShardsPerHour float64
ExecutionsPerHour float64
}

// ShardExecutionCountsDistribution breaks down stats on the distribution of executions per shard
ShardExecutionCountsDistribution struct {
MinExecutions *int64
MaxExecutions *int64
AverageExecutions int64
}
)

// AdminDBScan is used to scan over all executions in database and detect corruptions
Expand Down Expand Up @@ -739,6 +747,15 @@ func includeShardInProgressReport(report *ShardScanReport, progressReport *Progr
progressReport.CorruptionTypeBreakdown.TotalHistoryMissing += report.Scanned.CorruptionTypeBreakdown.TotalHistoryMissing
progressReport.CorruptionTypeBreakdown.TotalOpenExecutionInvalidCurrentExecution += report.Scanned.CorruptionTypeBreakdown.TotalOpenExecutionInvalidCurrentExecution
progressReport.CorruptionTypeBreakdown.TotalInvalidFirstEvent += report.Scanned.CorruptionTypeBreakdown.TotalInvalidFirstEvent
if progressReport.ShardExecutionCountsDistribution.MinExecutions == nil ||
*progressReport.ShardExecutionCountsDistribution.MinExecutions > report.Scanned.TotalExecutionsCount {
progressReport.ShardExecutionCountsDistribution.MinExecutions = &report.Scanned.TotalExecutionsCount
}
if progressReport.ShardExecutionCountsDistribution.MaxExecutions == nil ||
*progressReport.ShardExecutionCountsDistribution.MaxExecutions < report.Scanned.TotalExecutionsCount {
progressReport.ShardExecutionCountsDistribution.MaxExecutions = &report.Scanned.TotalExecutionsCount
}
progressReport.ShardExecutionCountsDistribution.AverageExecutions = progressReport.TotalExecutionsCount / int64(progressReport.NumberOfShardsFinished)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: AverageExecutions can be calculated only once outside of the per shard loop?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We emit this periodically though. So this is saying "based on the shards we have handled so far" this is the average.

}

if progressReport.TotalExecutionsCount > 0 {
Expand Down