Add vnswrr algorithm for upstream module.

[wangfakang]

Test Result

$TEST_NGINX_BINARY=/home/fakang.wfk/work/github/tengine/objs/nginx  prove -v -I ./tests/nginx-tests/nginx-tests/lib/    ./tests/nginx-tests/tengine-tests/ngx_http_upstream_vnswrr.t 
./tests/nginx-tests/tengine-tests/ngx_http_upstream_vnswrr.t .. 
1..9
ok 1 - vnswrr 8081
ok 2 - vnswrr 8082
ok 3 - peer down
ok 4 - weight 1
ok 5 - weight 2
ok 6 - vnswrr zone
ok 7 - vnswrr backup
ok 8 - no alerts
ok 9 - no sanitizer errors
ok
All tests successful.
Files=1, Tests=9,  0 wallclock secs ( 0.02 usr  0.00 sys +  0.08 cusr  0.01 csys =  0.11 CPU)
Result: PASS

Introduction

The VNSWRR module is an efficient load balancing algorithm that is smooth, decentralized, and high-performance compared to Nginx's official SWRR algorithm.

Example

http {

    upstream backend {
        vnswrr; # enable VNSWRR load balancing algorithm.
        127.0.0.1 port=81;
        127.0.0.1 port=82 weight=2;
        127.0.0.1 port=83;
        127.0.0.1 port=84 backup;
        127.0.0.1 port=85 down;
    }
    
    server {
        server_name localhost;
        
        location / {
            proxy_pass http://backend;
        }
    }
}

Installation

Build Tengine with this module from source:

./configure --add-module=./modules/ngx_http_upstream_vnswrr_module/
make
make install

Directive

• vnswrr

Syntax: vnswrr
Default: none
Context: upstream

Enable VNSWRR load balancing algorithm.

Performance

In the same pressure environment, the CPU consumption of VNSWRR algorithm accounts for 0.27% ( ngx_http_upstream_get_vnswrr).
Compared with VNSWRR algorithm, the CPU consumption of SWRR (ngx_http_upstream_get_peer 39%) is an order of magnitude higher than VNSWRR.

• Environment

CPU： Intel(R) Xeon(R) CPU E5-2682 v4 @ 2.50GHz

Tools：./wrk -t25  -d5m -c500  'http://ip/t2000'

Tengine：worker_processes 2; upstream endpoint {server0; ... server1999;}

Request Trace: Client --(keepalive)--> Tengine(proxy_pass) --(no keepalive)--> Endpoint

In the above environment, the QPS of VNSWRR increases by 60% compared with SWRR algorithm.

Observing the changes of QPS and RT in the different back-end number scenarios.
Under SWRR algorithm, when the number of back-end for every 500 addition, then the QPS of Nginx decreases by about 10% and RT increases by about 1 ms. But under the VNSWRR algorithm, QPS and RT do not change much.

[spacewander]

peers->peer will be never empty, and we don't skip failed peer, is it possible to have a NULL value best?

[wangfakang]

Yeah, thx.
The check logical is not necessary.

[spacewander]

Maybe we don't need to % uvnscf->vnumber here? There is already a % uvnscf->vnumber in the for loop below.

[wangfakang]

This is necessary. Because the initial value of begin_number is ngx_random() % peers->number , But uvnscf->vnumber <= peers->number.