chore: benchmarks and readme update

This commit is contained in:
Arthur Khachaturov 2024-07-20 14:48:46 +03:00
parent fda3920a3f
commit 7092dc6a70
No known key found for this signature in database
GPG key ID: CAC2B7EB6DF45D55
9 changed files with 115 additions and 18 deletions

View file

@ -84,12 +84,61 @@ http::run "$@"
```
# ⚡ Benchmarking
To benchmark HTTB, use the Python module `Locust`. Configuration files are in the `benchmark` folder. Set up and run Locust with the following commands:
It's... bad 🥲
```sh
cd benchmark
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
locust
All benchmarks were conducted on an i7-1360P laptop with 16GB of RAM using the tool codesenberg/bombardier. The benchmark folder in the repository contains all the source files used for benchmarking, along with a run.sh script to automate the benchmarking process. Here are the detailed results:
### python3 -m http.server
```
Statistics Avg Stdev Max
Reqs/sec 1501.94 297.64 3249.71
Latency 84.75ms 458.09ms 15.47s
HTTP codes:
1xx - 0, 2xx - 45205, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 0.86MB/s
```
### Flask w/ guvicorn
```
Statistics Avg Stdev Max
Reqs/sec 23355.06 8909.87 36359.07
Latency 5.35ms 2.73ms 121.66ms
HTTP codes:
1xx - 0, 2xx - 700131, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 4.81MB/s
```
### HTTB
```
Statistics Avg Stdev Max
Reqs/sec 1287.59 226.72 2057.52
Latency 96.81ms 168.72ms 3.17s
HTTP codes:
1xx - 0, 2xx - 13004, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 304.04KB/s
```
### Node.js
```
Statistics Avg Stdev Max
Reqs/sec 91833.85 4907.33 97671.88
Latency 1.36ms 119.26us 17.27ms
HTTP codes:
1xx - 0, 2xx - 2754740, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 19.09MB/s
```
### Golang
```
Statistics Avg Stdev Max
Reqs/sec 304523.14 32166.94 453253.21
Latency 407.73us 182.95us 91.07ms
HTTP codes:
1xx - 0, 2xx - 9138757, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 52.29MB/s
```

View file

@ -1,5 +0,0 @@
autostart=1
headless=1
host=http://localhost:8081
users=1024
spawn-rate=128

View file

@ -1,5 +0,0 @@
from locust import HttpUser, task
class RootUser(HttpUser):
@task
def root(self):
self.client.get("/")

View file

@ -1 +1,2 @@
locust
flask
gunicorn

28
benchmark/run.sh Executable file
View file

@ -0,0 +1,28 @@
#!/bin/bash
set -e
set -m
install_gunicorn() {
python3 -m venv .venv
source .venv/bin/activate
pip install gunicorn
}
declare -a reqs=('node' 'python3' 'go')
declare -A run_cmds=([8082]='go run test.go' [8083]='node test.js' [8084]='gunicorn -w 8 test:app -b localhost:8084' [8000]='python3 -m http.server' [8081]='bash ./test.sh run')
for package in "${reqs[@]}"; do
command -v "${package}" >/dev/null || { echo "Error! ${package} is not installed on this system."; exit 1; }
done
command -v bombardier >/dev/null || go install github.com/codesenberg/bombardier@latest
command -v gunicorn >/dev/null || install_gunicorn >/dev/null
for port in "${!run_cmds[@]}"; do
echo "Running: ${run_cmds[$port]}"
${run_cmds[$port]} >/dev/null 2>&1 & pid="$!"
sleep 0.5
bombardier -d 30s http://127.0.0.1:"$port"
kill -TERM -- -"$pid"
done

6
benchmark/test.go Normal file
View file

@ -0,0 +1,6 @@
package main
import ( "net/http")
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { w.Write([]byte("{}")) });
http.ListenAndServe(":8082", nil);
}

7
benchmark/test.js Normal file
View file

@ -0,0 +1,7 @@
const http = require('http');
const server = http.createServer((req, res) => {
res.statusCode = 200;
res.setHeader('Content-Type', 'application/json');
res.end("{}");
});
server.listen(8083);

7
benchmark/test.py Normal file
View file

@ -0,0 +1,7 @@
from flask import Flask
app = Flask('app')
@app.get("/")
def root():
return "{}"

9
benchmark/test.sh Normal file
View file

@ -0,0 +1,9 @@
. ../src/http_server.sh
http::bind "127.0.0.1" "8081"
main() {
http::response 200
} && http::get main "/"
http::run "$@"