[to #421] add a python script to analyze slow log stats

[zz-jason]

Signed-off-by: Jian Zhang [email protected]

What problem does this PR solve?

Issue Number: close [to #421]

Problem Description: add a python script to analyze slow log and print the stats

What is changed and how it works?

➜ ./scripts/rstats.py --help
usage: rstats.py [-h] [--order ORDER] slowlog

rstats: A TiKV Java Client Request Stats Analyzer

positional arguments:
  slowlog        slow log file

optional arguments:
  -h, --help     show this help message and exit
  --order ORDER  order the output, default: total. accepted value: total,
                 grpc, backoff

an output example:

➜ ./scripts/rstats.py ~/slow.log
Request      Start          Total Lat(ms)  Total gRPC Lat(ms)   Total Backoff Lat(ms)
putIfAbsent  22:50:36.364   868            751                  116
putIfAbsent  22:50:36.945   625            571                  46
putIfAbsent  05:23:10.665   23             0                    21
get          05:23:05.893   11             11                   0

Code changes

• No code, only scripts

Check List for Tests

This PR has been tested by the at least one of the following methods:

• No code

Side effects

• NO side effects

Related changes

• Need to cherry-pick to the release branch

[zz-jason]

/run-all-tests

[peng1999]

This script targets to the old slowlog format, maybe should merged into 3.1 branch?

[marsishandsome]

LGTM

[zz-jason]

This script targets to the old slowlog format, maybe should merged into 3.1 branch?

yes, of course. I added the 3.1 cherry-pick label.

[codecov]

Codecov Report

Merging #527 (794fcaa) into master (d24b8e9) will decrease coverage by 0.31%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #527      +/-   ##
============================================
- Coverage     32.18%   31.86%   -0.32%     
+ Complexity     1320     1311       -9     
============================================
  Files           278      278              
  Lines         17344    17344              
  Branches       1975     1975              
============================================
- Hits           5582     5527      -55     
- Misses        11137    11198      +61     
+ Partials        625      619       -6     

Impacted Files	Coverage Δ
...rc/main/java/io/grpc/netty/NettyClientHandler.java	56.89% <0.00%> (-7.33%)	⬇️
...va/org/tikv/common/region/StoreHealthyChecker.java	57.89% <0.00%> (-5.27%)	⬇️
...ty/handler/codec/http2/Http2ConnectionHandler.java	48.65% <0.00%> (-4.89%)	⬇️
src/main/java/io/grpc/internal/ClientCallImpl.java	56.34% <0.00%> (-0.49%)	⬇️
...java/org/tikv/common/region/RegionStoreClient.java	47.85% <0.00%> (-0.31%)	⬇️
.../main/java/org/tikv/common/region/RegionCache.java	50.92% <0.00%> (+1.85%)	⬆️
src/main/java/org/tikv/common/region/TiStore.java	69.23% <0.00%> (+12.82%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d24b8e9...794fcaa. Read the comment docs.

[iosmanthus]

What if there is a line containing corrupted JSON?

[zz-jason]

the script assumes that the slow log file is generated by tikv applications without manual modifications. If it's not a JSON, it means the file is mistakenly modified by someone, the script just not working since it breaks the assumption.

[iosmanthus]

LGTM

[ti-srebot]

cherry pick to release-3.1 in PR #534