v0.0.19 - Stability improvements #116

cirospaciari · 2023-03-19T17:24:51Z

cirospaciari
Mar 19, 2023
Maintainer

What is changed in socketify:

Fix memory leaks on WSGI and ASGI
Fix segfault on broken headers

Work in progress for v0.1.0:

Native optimizations Add native optimizations #52
Cache tools LRU Cache and Timed Cache avoiding GIL a in memory cache solution #60
Hot reloading Add hot reloading on CLI Tools #61

What's Changed

WSGI Content-Length and Transfer-Encoding by @cirospaciari in WSGI Content-Length and Transfer-Encoding #75

Full Changelog: v0.0.18...v0.0.19

This discussion was created from the release v0.0.19 - Stability improvements.

remittor · 2023-03-21T13:02:46Z

remittor
Mar 21, 2023

I noticed that it is now slower (Debian 11 64-bit, Python 3.9)

Socketify 0.0.19

$ python3 server.py -g si

$ wrk -t1 -c1 -d15 -H 'Connection: keep-alive' http://127.0.0.1:5000/plaintext
Running 15s test @ http://127.0.0.1:5000/plaintext
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.15ms   49.67us   2.99ms   96.95%
    Req/Sec     0.87k     9.53     0.89k    78.00%
  13028 requests in 15.01s, 2.17MB read
Requests/sec:    867.96
Transfer/sec:    148.33KB

$ wrk -t1 -c100 -d15 -H 'Connection: keep-alive' http://127.0.0.1:5000/plaintext
Running 15s test @ http://127.0.0.1:5000/plaintext
  1 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.65ms  173.36us   8.15ms   85.97%
    Req/Sec    21.62k   435.63    22.39k    70.00%
  322662 requests in 15.00s, 53.85MB read
Requests/sec:  21508.41
Transfer/sec:      3.59MB

$ wrk -t1 -c100 -d15 -H 'Connection: keep-alive' http://127.0.0.1:5000/plaintext -s pipeline.lua -- 16
Running 15s test @ http://127.0.0.1:5000/plaintext
  1 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    17.92ms    8.95ms  54.46ms   57.06%
    Req/Sec    49.15k   735.45    50.58k    67.33%
  733488 requests in 15.00s, 122.41MB read
Requests/sec:  48891.50
Transfer/sec:      8.16MB

FastWSGI (branch 2023-03-21 with pipelining)

$ python3 server.py -g si -v 2

$ wrk -t1 -c1 -d15 -H 'Connection: keep-alive' http://127.0.0.1:5000/plaintext
Running 15s test @ http://127.0.0.1:5000/plaintext
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    47.54us   35.43us   2.19ms   99.20%
    Req/Sec    21.24k   743.17    22.40k    79.47%
  319085 requests in 15.10s, 47.78MB read
Requests/sec:  21131.14
Transfer/sec:      3.16MB

$ wrk -t1 -c100 -d15 -H 'Connection: keep-alive' http://127.0.0.1:5000/plaintext
Running 15s test @ http://127.0.0.1:5000/plaintext
  1 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.18ms  102.19us   3.42ms   89.34%
    Req/Sec    84.64k     3.49k   89.43k    74.67%
  1263457 requests in 15.00s, 189.17MB read
Requests/sec:  84224.77
Transfer/sec:     12.61MB

$ wrk -t1 -c100 -d15 -H 'Connection: keep-alive' http://127.0.0.1:5000/plaintext -s pipeline.lua -- 16
Running 15s test @ http://127.0.0.1:5000/plaintext
  1 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    18.62ms   13.52ms  49.97ms   54.09%
    Req/Sec   123.79k    19.99k  157.63k    68.67%
  1847720 requests in 15.00s, 276.65MB read
Requests/sec: 123160.49
Transfer/sec:     18.44MB

server.py script: https://gist.github.com/remittor/c9411e62b5ea4776200bee288a331016

0 replies

remittor · 2023-03-21T13:10:31Z

remittor
Mar 21, 2023

Previous version: #111

$ wrk -t1 -c1 -d15 http://127.0.0.1:5000
Running 15s test @ http://127.0.0.1:5000
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    54.19us   58.65us   4.83ms   99.43%
    Req/Sec    18.85k   763.04    19.83k    84.00%
  281228 requests in 15.00s, 42.11MB read
Requests/sec:  18748.14
Transfer/sec:      2.81MB

2 replies

cirospaciari Mar 21, 2023
Maintainer Author

In every version, I add corrections and in the next iteration, I will do some optimizations, now the WSGI is more correct but slower. I normally wait for TFB results to see how every web framework is affected. After v1.0.0 I will not release in this TIC/TOC way, but for now, performance will go up and down, last version has a pretty big performance hit, the next step is to do ASGI correction, and after this convert my current CFFI solution to HPy so performance will improve a lot more.
In this version I relaxed the event loop (maybe too much) probably I will revert that.

socketify aims to be used without WSGI or ASGI, ASGI and WSGI are only to help the community with existing projects.

fastWSGI and meinheld as native extensions will have a great advantage over socketify WSGI on CPython. WSGI is not my primary focus and my primary target is PyPy, not CPython.

My target for WSGI is to be on par with meinheld (meinheld is faster on CPython but we can be faster on PyPy), ASGI target is to be on par with uvicorn (today we are faster in most cases on CPython, and even faster on PyPy)

I don't think socketify will beat fastWSGI on CPython or even PyPy at this stage, is too soon and too young. But for sure I will love to see it taking blows with fastWSGI and I hope fastWSGI gets even faster and faster.

Here are some tests against the last release of fastWSGI with is not really fair because pipelining is kinda broke, I will wait until 0.0.9 to compare again.

My configs:

Python 3.9.16 (7.3.11+dfsg-2, Feb 06 2023, 16:52:03) [PyPy 7.3.11 with GCC 12.2.0]
Python 3.11.2 (CPython)

OS: Debian GNU/Linux bookworm/sid x86_64
Kernel: 6.1.0-6-amd64
CPU: Intel i7-7700HQ (8) @ 3.800GHz

meinheld==meinheld-1.0.2 [git+https://github.com/idot/meinheld.git@2bfe452d6608c92688d92337c87b1dd6448f4ccb]
fastWSGI==0.0.8
socketify==0.0.18 using WSGI

My local results today are with 1 run for a warm-up and after this 3 runs avg, with and without multiprocessing/fork:

For 8 thread and 8 process:

wrk -H "Host: tfb-server" -H "Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,/;q=0.7"  --latency -d 5 -c 4096 --timeout 8 -t 1 http://localhost:8000 -s pipeline.lua -- 16

For 1 thread and 1 process:

wrk -H "Host: tfb-server" -H "Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,/;q=0.7"  --latency -d 5 -c 4096 --timeout 8 -t 8 http://localhost:8000 -s pipeline.lua -- 16

For results without pipelining, I just removing the -s pipeline.lua --16 script.

def app_hello(environ, start_response):
    start_response('200 OK', [('Content-Type', 'text/plain'), ('Content-Length', '13')])
    return [ b'Hello, World!']

fastWSGI without pipelining

Running 5s test @ http://localhost:8000
  1 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.59ms    1.09ms  23.35ms   92.83%
    Req/Sec    81.19k     3.40k   86.87k    67.35%
  Latency Distribution
     50%   12.35ms
     75%   12.77ms
     90%   13.32ms
     99%   17.64ms
  395452 requests in 5.00s, 52.42MB read
  Socket errors: connect 3076, read 0, write 0, timeout 0
Requests/sec:  79055.96
Transfer/sec:     10.48MB

fastWSGI with pipelining

Running 5s test @ http://localhost:8000
  1 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     0.00us    0.00us   0.00us    -nan%
    Req/Sec    14.54k     3.59k   19.81k    50.00%
  Latency Distribution
     50%    0.00us
     75%    0.00us
     90%    0.00us
     99%    0.00us
  72420 requests in 5.05s, 7.11MB read
  Socket errors: connect 3076, read 0, write 0, timeout 0
  Non-2xx or 3xx responses: 72420
Requests/sec:  14339.69
Transfer/sec:      1.41MB

fastWSGI without pipelining

Running 5s test @ http://localhost:8000
  8 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.86ms    4.31ms  45.26ms   86.57%
    Req/Sec    30.51k     8.55k   62.50k    81.98%
  Latency Distribution
     50%    2.09ms
     75%    4.89ms
     90%   10.06ms
     99%   19.68ms
  1210934 requests in 5.09s, 160.52MB read
  Socket errors: connect 3083, read 0, write 0, timeout 0
Requests/sec: 237967.73
Transfer/sec:     31.55MB

fastWSGI with pipelining

Running 5s test @ http://localhost:8000
  8 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     0.00us    0.00us   0.00us    -nan%
    Req/Sec     6.51k     3.03k   12.41k    69.41%
  Latency Distribution
     50%    0.00us
     75%    0.00us
     90%    0.00us
     99%    0.00us
  259103 requests in 5.09s, 25.45MB read
  Socket errors: connect 3083, read 0, write 0, timeout 0
  Non-2xx or 3xx responses: 259103
Requests/sec:  50929.61
Transfer/sec:      5.00MB

socketify without pipelining

Running 5s test @ http://localhost:8000
  1 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    26.44ms    2.08ms  52.03ms   97.41%
    Req/Sec    38.63k     1.39k   40.10k    90.00%
  Latency Distribution
     50%   26.24ms
     75%   26.56ms
     90%   26.94ms
     99%   30.31ms
  192126 requests in 5.08s, 25.47MB read
  Socket errors: connect 3076, read 0, write 0, timeout 0
Requests/sec:  37784.67
Transfer/sec:      5.01MB

socketify with pipelining

Running 5s test @ http://localhost:8000
  1 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   183.24ms   71.19ms 371.11ms   60.80%
    Req/Sec    61.80k     9.64k   73.21k    64.00%
  Latency Distribution
     50%  170.50ms
     75%  236.04ms
     90%  279.27ms
     99%  356.50ms
  307440 requests in 5.09s, 40.75MB read
  Socket errors: connect 3076, read 0, write 0, timeout 0
Requests/sec:  60380.13
Transfer/sec:      8.00MB

socketify without pipelining

Running 5s test @ http://localhost:8000
  8 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     9.08ms    3.12ms  40.60ms   77.28%
    Req/Sec    13.89k     4.49k   26.18k    70.75%
  Latency Distribution
     50%    8.43ms
     75%   10.48ms
     90%   13.04ms
     99%   19.28ms
  553377 requests in 5.07s, 73.36MB read
  Socket errors: connect 3083, read 0, write 0, timeout 0
Requests/sec: 109246.72
Transfer/sec:     14.48MB

socketify with pipelining

Running 5s test @ http://localhost:8000
  8 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    43.04ms   15.97ms 137.87ms   59.90%
    Req/Sec    33.03k    14.87k   65.60k    63.50%
  Latency Distribution
     50%   42.45ms
     75%   55.88ms
     90%   64.18ms
     99%   74.99ms
  1314880 requests in 5.05s, 174.30MB read
  Socket errors: connect 3083, read 0, write 0, timeout 0
Requests/sec: 260239.74
Transfer/sec:     34.50MB

meinheld without pipelining

Running 5s test @ http://localhost:8000
  1 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    33.57ms   90.62ms   1.55s    98.07%
    Req/Sec    39.18k     1.73k   40.86k    94.00%
  Latency Distribution
     50%   23.91ms
     75%   24.65ms
     90%   25.12ms
     99%  498.79ms
  194676 requests in 5.10s, 30.26MB read
  Socket errors: connect 3076, read 0, write 0, timeout 0
Requests/sec:  38207.78
Transfer/sec:      5.94MB

meinheld with pipelining

Running 5s test @ http://localhost:8000
  1 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   643.41ms  987.29ms   4.98s    84.90%
    Req/Sec    50.07k     6.04k   57.52k    76.00%
  Latency Distribution
     50%  132.38ms
     75%  786.33ms
     90%    2.22s 
     99%    4.13s 
  248821 requests in 5.08s, 38.68MB read
  Socket errors: connect 3076, read 0, write 0, timeout 0
Requests/sec:  48944.04
Transfer/sec:      7.61MB

meinheld without pipelining

Running 5s test @ http://localhost:8000
  8 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.09ms    2.79ms  52.81ms   75.46%
    Req/Sec    15.69k     4.72k   25.21k    59.75%
  Latency Distribution
     50%    7.84ms
     75%    9.44ms
     90%   11.33ms
     99%   17.31ms
  624641 requests in 5.03s, 97.10MB read
  Socket errors: connect 3083, read 0, write 0, timeout 0
Requests/sec: 124131.65
Transfer/sec:     19.30MB

meinheld with pipelining

Running 5s test @ http://localhost:8000
  8 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    73.09ms   37.90ms 429.58ms   76.06%
    Req/Sec    21.08k     7.65k   42.15k    62.16%
  Latency Distribution
     50%   67.48ms
     75%   90.72ms
     90%  115.31ms
     99%  206.57ms
  837995 requests in 5.05s, 130.27MB read
  Socket errors: connect 3083, read 0, write 0, timeout 0
Requests/sec: 165778.72
Transfer/sec:     25.77MB

socketify PyPy without pipelining

Running 5s test @ http://localhost:8000
  1 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    15.62ms    1.76ms  33.31ms   92.96%
    Req/Sec    65.41k     4.46k   71.05k    80.00%
  Latency Distribution
     50%   15.27ms
     75%   15.80ms
     90%   16.74ms
     99%   24.07ms
  325183 requests in 5.10s, 43.11MB read
  Socket errors: connect 3076, read 0, write 0, timeout 0
Requests/sec:  63756.52
Transfer/sec:      8.45MB

socketify PyPy with pipelining

Running 5s test @ http://localhost:8000
  1 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    15.38ms    3.79ms  61.07ms   85.54%
    Req/Sec   137.59k    21.08k  170.83k    75.51%
  Latency Distribution
     50%   14.08ms
     75%   16.37ms
     90%   20.73ms
     99%   23.88ms
  670368 requests in 5.02s, 88.86MB read
  Socket errors: connect 3076, read 0, write 0, timeout 0
Requests/sec: 133496.34
Transfer/sec:     17.70MB

socketify PyPy without pipelining

Running 5s test @ http://localhost:8000
  8 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.63ms    1.39ms  25.53ms   77.30%
    Req/Sec    26.37k     8.14k   50.43k    74.75%
  Latency Distribution
     50%    4.40ms
     75%    5.26ms
     90%    6.30ms
     99%    8.91ms
  1051639 requests in 5.05s, 139.41MB read
  Socket errors: connect 3083, read 0, write 0, timeout 0
Requests/sec: 208427.44
Transfer/sec:     27.63MB

socketify PyPy with pipelining

Running 5s test @ http://localhost:8000
  8 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    19.33ms    8.38ms  80.59ms   67.65%
    Req/Sec    76.03k    35.32k  152.00k    56.50%
  Latency Distribution
     50%   18.71ms
     75%   24.51ms
     90%   30.26ms
     99%   43.84ms
  3025680 requests in 5.07s, 401.09MB read
  Socket errors: connect 3083, read 0, write 0, timeout 0
Requests/sec: 596284.98
Transfer/sec:     79.04MB

cirospaciari Mar 21, 2023
Maintainer Author

About the performance drop:
Loop relax:

Running 15s test @ http://127.0.0.1:8000
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.44ms  284.61us  10.05ms   89.04%
    Req/Sec   699.81     28.39   770.00     82.00%
  10464 requests in 15.03s, 1.39MB read
Requests/sec:    696.39
Transfer/sec:     94.53KB

Without relax the loop:

Running 15s test @ http://127.0.0.1:8000
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    37.78us   10.20us   1.27ms   95.60%
    Req/Sec    25.64k     0.89k   26.47k    97.35%
  385192 requests in 15.10s, 51.06MB read
Requests/sec:  25509.39
Transfer/sec:      3.38MB

Removed that on v0.0.20 for now

remittor · 2023-03-21T17:34:43Z

remittor
Mar 21, 2023

the next step is to do ASGI correction, and after this convert my current CFFI solution to HPy so performance will improve a lot more.

Have you tried using Nuitka?

1 reply

cirospaciari Mar 21, 2023
Maintainer Author

Never used Nuitka before, I will take a look

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v0.0.19 - Stability improvements #116

{{title}}

Replies: 3 comments 3 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

v0.0.19 - Stability improvements #116

cirospaciari Mar 19, 2023 Maintainer

What is changed in socketify:

Work in progress for v0.1.0:

What's Changed

Replies: 3 comments · 3 replies

remittor Mar 21, 2023

Socketify 0.0.19

FastWSGI (branch 2023-03-21 with pipelining)

remittor Mar 21, 2023

cirospaciari Mar 21, 2023 Maintainer Author

cirospaciari Mar 21, 2023 Maintainer Author

remittor Mar 21, 2023

cirospaciari Mar 21, 2023 Maintainer Author

cirospaciari
Mar 19, 2023
Maintainer

Replies: 3 comments 3 replies

remittor
Mar 21, 2023

remittor
Mar 21, 2023

cirospaciari Mar 21, 2023
Maintainer Author

cirospaciari Mar 21, 2023
Maintainer Author

remittor
Mar 21, 2023

cirospaciari Mar 21, 2023
Maintainer Author