mORMot Open Source

Makhaon

Member

Registered: 2021-03-15

Posts: 1

Testing thousands connections

Hi! How do you test hundreds of thousands simultaneous connections?
Which soft can be used as client(s)? JMeter or someone else?

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Testing thousands connections

I doubt you would be able to do it on Windows.
Anyway, for so many connections, you should put a reverse proxy like nginx as frontend, then configure the mORMot server on Linux to use HTTP/1.0 requests over the unix daemon loopback (not TCP loopback), while nginx would maintain HTTP/1.1 kept alive connections.
Note that mORMot could handle around 100,000 requests per second so thousands connections need to be mostly steady (not calling at the same time).
For thousands of connections on production, consider putting something haproxy as frontend for load balancing, and distribute the connections to several servers.

Currently, our WebSockets server maintain a thread per client, so it is not scalable.
I am working on it, and the thread pooll will be used for mORMot 2 for WebSockets, as it currently does for HTTP/1.0 requests.

About benchmark and performance, mpv seems to prefer https://github.com/wg/wrk
See e.g. https://synopse.info/forum/viewtopic.ph … 391#p31391

Eugene Ilyin

Member

From: milky_way/orion_arm/sun/earth

Registered: 2016-03-27

Posts: 132

Website

Re: Testing thousands connections

My favorite HTTP benchmark performance testing tools used for mORMot:

Class A
Minimal resources consumption per worker, maximum parallel threads, precise measurements, fast start per instance

1. Oha (the best one from resource, quality, parallel measuments, reporting, UI )

2. bombardier

3. fasthttploader

Class B

1. gohttpbench

2. gobench

3. k6

Class C

1. autocannon

2. fortio

Class D

1. baton

2. cassowary

3. drill

4. go-wrk

The other are listed in awesome-http-benchmark

mpv

Member

From: Ukraine

Registered: 2012-03-24

Posts: 1,571

Website

Re: Testing thousands connections

Nice list!
I use an original wrk/wrk2, it allows creating very complex scenarious using lua scripts, so I can test complex API. But it's unix only.

Yesterday I tests API on the production environment of one of my customer and (on good hardware with 10g network between wrk and app server and Xeon CPU  on both, OS is OEL8, DB is Oracle19) got an awersome 250MB traffic between wrk and app server (with nginx as lb and mormot in http1 mode).

Last edited by mpv (2021-03-19 14:14:53)

Eugene Ilyin

Member

From: milky_way/orion_arm/sun/earth

Registered: 2016-03-27

Posts: 132

Website

Re: Testing thousands connections

@mpv, you can easily migrate to k6

It's army swiss knife on the modern complex load testing scenarios:
Asserts, Thresholds, HTTP/1.1, HTTP/2, WebSocket, gRPC, Cookies, Crypto, Custom metrics, Encodings, Environment variables, JSON, HTML forms, files.

Configuation-as-a-script and very pleasant to write scenarios.
Not sticked to Linux only.

Here is a "hello world" HTTP load test under Windows:

 
import http from "k6/http";

export default function() {
  let response = http.get("http://127.0.0.1:8080/");
};

k6.exe run -u 256 -i 1000000 script.js

mpv

Member

From: Ukraine

Registered: 2012-03-24

Posts: 1,571

Website

Re: Testing thousands connections

Eugene, thank you! 

K6 is really cool. Just tried  - it's cover all I need and much more. And scripting via JS is much easy to me compared to lua in wrk or XML in JMeater. K6 is my favorite testing tool for now for complex scenarios

For simple cases K6 is up to two times slower compared to wkr (simple POST with some variable parameters. server just reply JSON back)
K6             - 11 080 RPS
wrk (+lua) - 16 624 RPS

but for complex test, where server method do some stuff and test logic should be complex k6 shine.

Last edited by mpv (2021-03-20 18:39:01)

Eugene Ilyin

Member

From: milky_way/orion_arm/sun/earth

Registered: 2016-03-27

Posts: 132

Website

Re: Testing thousands connections

Yeap, k6 is NOT for the maximum performance testing.
k6 is for complex scenarios to check relative to previous runs performance degradation/increase.
That why for the speed tests it's in my "Class B"

As for the raw HTTP queries flow - the "Class A" exist (really try Oha, after 10-12 Oha runs you will love this console like UI and performance measurements. Oha is written in Rust, very stable, small, fast, and predictable - the same run sequences show the same performance results with a very low deviation).

Last edited by Eugene Ilyin (2021-03-20 18:59:27)