loadtesting: request rate for uploading charts and get index drops a lot with 0.16.0 release #711
Description
Version:
0.16.0 release
Problem
Using locust to have the loadtesting, the request rate for uploading charts and get index drops a lot.
Description
Previously
Previously chartmuseum release version v0.12.0 with pvc:
locust -f locustfile.py --no-web -L DEBUG -c 200 -r 50 -n 100000 --csv=result
We define posting new charts and get index in locustfile.py. The request rate for uploading charts could arrive 300 req/s and get index 30 req/s
Using 0.16.0
But when we use 0.16.0, the request rate drops to 5 req/s and 0.5 req/s and after the charts arrives 230 or so, the post requests begin to fail.
From the log, you can see the latency is very high, {"L":"INFO","T":"2023-08-11T10:48:11.323Z","M":"[12781] Request served","path":"/api/mychart/charts","comment":"","clientIP":"XXXXX","method":"POST","statusCode":201,"latency":"19m6.78816919s","reqID":"xxxxxxxxxxxx"}
The chartmuseum pod cpu is not high only 50%
Using go tool pprof to get some data:
