Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

p-median Julia benchmarks #1

Merged
merged 3 commits into from
Jul 25, 2020
Merged

p-median Julia benchmarks #1

merged 3 commits into from
Jul 25, 2020

Conversation

mlubin
Copy link
Member

@mlubin mlubin commented Jul 18, 2020

This is rough code to benchmark solving the p-median (facility location) problem using SCS and GLPK through both MOI and the direct APIs.
It needs to be cleaned up before calling it official, but this at least provides the performance numbers:

Second run of run_benchmark(num_facilities=5, num_customers=20, num_locations=10, time_limit_sec=Inf, max_iters=10000):

 ──────────────────────────────────────────────────────────────────────
                               Time                   Allocations      
                       ──────────────────────   ───────────────────────
   Tot / % measured:        284ms / 100%            2.20MiB / 100%     

 Section       ncalls     time   %tot     avg     alloc   %tot      avg
 ──────────────────────────────────────────────────────────────────────
 SCS MOI            1    147ms  52.0%   147ms   1.62MiB  73.6%  1.62MiB
   solve            1    145ms  51.3%   145ms   0.99MiB  45.3%  0.99MiB
   generate         1   1.79ms  0.63%  1.79ms    607KiB  27.0%   607KiB
 SCS direct         1    132ms  46.7%   132ms    133KiB  5.89%   133KiB
   solve            1    132ms  46.6%   132ms   25.6KiB  1.14%  25.6KiB
   generate         1    290μs  0.10%   290μs    105KiB  4.68%   105KiB
 GLPK MOI           1   2.38ms  0.84%  2.38ms    425KiB  18.9%   425KiB
   solve            1   1.56ms  0.55%  1.56ms     0.00B  0.00%    0.00B
   generate         1    568μs  0.20%   568μs    395KiB  17.6%   395KiB
 GLPK direct        1   1.40ms  0.49%  1.40ms   36.2KiB  1.61%  36.2KiB
   solve            1   1.32ms  0.47%  1.32ms     0.00B  0.00%    0.00B
   generate         1   62.1μs  0.02%  62.1μs   34.2KiB  1.52%  34.2KiB
 ──────────────────────────────────────────────────────────────────────

Overhead of MOI for SCS is 147/132 = 1.11 (very good) and for GLPK 2.38/1.4 = 1.7 (also pretty good).

run_benchmark(num_facilities=10, num_customers=2000, num_locations=1000, time_limit_sec=5, max_iters=10) (Note: solve hits the time limit/iteration limit)

 ──────────────────────────────────────────────────────────────────────
                               Time                   Allocations      
                       ──────────────────────   ───────────────────────
   Tot / % measured:         122s / 100%            17.1GiB / 100%     

 Section       ncalls     time   %tot     avg     alloc   %tot      avg
 ──────────────────────────────────────────────────────────────────────
 SCS MOI            1    70.8s  57.9%   70.8s   12.6GiB  73.3%  12.6GiB
   solve            1    56.5s  46.2%   56.5s   7.75GiB  45.2%  7.75GiB
   generate         1    14.4s  11.8%   14.4s   4.82GiB  28.1%  4.82GiB
 GLPK MOI           1    24.1s  19.7%   24.1s   3.26GiB  19.0%  3.26GiB
   generate         1    16.9s  13.8%   16.9s   3.24GiB  18.9%  3.24GiB
   solve            1    7.08s  5.79%   7.08s   15.3MiB  0.09%  15.3MiB
 SCS direct         1    19.4s  15.9%   19.4s   1.17GiB  6.81%  1.17GiB
   solve            1    17.6s  14.4%   17.6s    214MiB  1.22%   214MiB
   generate         1    1.83s  1.50%   1.83s   0.96GiB  5.60%  0.96GiB
 GLPK direct        1    7.93s  6.49%   7.93s    139MiB  0.79%   139MiB
   solve            1    6.65s  5.44%   6.65s     0.00B  0.00%    0.00B
   generate         1    1.23s  1.01%   1.23s    139MiB  0.79%   139MiB
 ──────────────────────────────────────────────────────────────────────

Overhead of MOI for SCS is 70.8/19.4 = 3.6 (not great) and for GLPK 24.1/7.93 = 3.03 (not great).
These ratios are a bit fishy because they depend on how early the solve stops, but for SCS there's still apparently 50 seconds of extra overhead in the MOI version compared with the direct version.

This is using master of both SCS and GLPK. I haven't tried jump-dev/GLPK.jl#143.

pmedian.jl Outdated Show resolved Hide resolved
@mlubin mlubin mentioned this pull request Jul 22, 2020
@mlubin mlubin merged commit d47b968 into master Jul 25, 2020
@mlubin mlubin deleted the pmedian branch July 25, 2020 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants