Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reproduce DoS by hash map collisions for uJson #325

Closed
wants to merge 3 commits into from

Conversation

plokhotnyuk
Copy link
Owner

@plokhotnyuk plokhotnyuk commented Jun 28, 2019

See https://github.com/lihaoyi/upickle/issues/273

EDIT: @lihaoyi removed the ticked above including all suggestions from the Scala community about how to fix it... Bellow is an original report with code and instructions to reproduce this security flaw...

Sub-quadratic decreasing of throughput when number of JSON object fields (with keys that have the same hash code) is increasing

On contemporary CPUs parsing of such JSON object (with a sequence of 100000 fields like below that is ~1.6Mb) can took ~200 seconds:

{
"!!sjyehe":null,
"!!sjyeiF":null,
"!!sjyej'":null,
"!!sjyfIe":null,
"!!sjyfJF":null,
...
}

Below are results of the benchmark where size is a number of such fields:

[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark                   (size)   Mode  Cnt        Score   Error  Units
[info] ExtractFieldsReading.uJson       1  thrpt       1094798.399          ops/s
[info] ExtractFieldsReading.uJson      10  thrpt        226871.642          ops/s
[info] ExtractFieldsReading.uJson     100  thrpt         10772.525          ops/s
[info] ExtractFieldsReading.uJson    1000  thrpt           212.781          ops/s
[info] ExtractFieldsReading.uJson   10000  thrpt             1.627          ops/s
[info] ExtractFieldsReading.uJson  100000  thrpt             0.005          ops/s

To run that benchmarks on your JDK:

  1. Install latest version of sbt and/or ensure that it already installed properly:
sbt about
  1. Clone jsoniter-scala repo:
git clone https://github.com/plokhotnyuk/jsoniter-scala.git
  1. Enter to the cloned directory and checkout for the specific branch:
cd jsoniter-scala
git checkout uJson-DoS-by-hashmap-collisions
  1. Run benchmarks using a path parameter to your JDK:
sbt -no-colors 'jsoniter-scala-benchmark/jmh:run -jvm /usr/lib/jvm/jdk-11/bin/java -wi 1 -i 1 ExtractFields.*uJson'

@codecov
Copy link

codecov bot commented Jun 28, 2019

Codecov Report

Merging #325 into master will decrease coverage by 0.03%.
The diff coverage is 0%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #325      +/-   ##
==========================================
- Coverage     100%   99.96%   -0.04%     
==========================================
  Files         181      181              
  Lines        5297     5299       +2     
  Branches      501      528      +27     
==========================================
  Hits         5297     5297              
- Misses          0        2       +2
Impacted Files Coverage Δ
...soniter_scala/benchmark/ExtractFieldsReading.scala 88.23% <0%> (-11.77%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f39b98e...b6cbb54. Read the comment docs.

@codecov
Copy link

codecov bot commented Jun 28, 2019

Codecov Report

Merging #325 into master will decrease coverage by 0.03%.
The diff coverage is 0.00%.

Impacted file tree graph

@@             Coverage Diff             @@
##            master     #325      +/-   ##
===========================================
- Coverage   100.00%   99.96%   -0.04%     
===========================================
  Files          180      180              
  Lines         5402     5404       +2     
  Branches       556      551       -5     
===========================================
  Hits          5402     5402              
- Misses           0        2       +2     
Impacted Files Coverage Δ
...soniter_scala/benchmark/ExtractFieldsReading.scala 87.50% <0.00%> (-12.50%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0e2ed85...2b7a003. Read the comment docs.

@plokhotnyuk
Copy link
Owner Author

It is reproducible with uJson 0.9.6 still.

Here is a screenshot with the flame graph for the affected stack:
image

@plokhotnyuk plokhotnyuk force-pushed the master branch 3 times, most recently from 6e0e397 to f0ebbe1 Compare January 3, 2020 16:33
@plokhotnyuk plokhotnyuk force-pushed the master branch 5 times, most recently from b408876 to 2aa234e Compare March 30, 2020 06:07
@plokhotnyuk plokhotnyuk force-pushed the master branch 6 times, most recently from 8a28c79 to b1cec24 Compare March 28, 2022 13:49
@plokhotnyuk plokhotnyuk force-pushed the master branch 6 times, most recently from e2f7a7a to 89920a9 Compare April 17, 2022 11:52
@plokhotnyuk plokhotnyuk force-pushed the master branch 4 times, most recently from 5dc9b0f to 51dcd60 Compare April 26, 2022 10:51
@plokhotnyuk plokhotnyuk force-pushed the master branch 5 times, most recently from 74b71bf to b898694 Compare May 16, 2022 11:12
@plokhotnyuk plokhotnyuk force-pushed the master branch 6 times, most recently from ad1ac7c to 53fd6b7 Compare May 21, 2022 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant