add drop-in json library adapter#364
Open
halfcrazy wants to merge 1 commit intoovn-kubernetes:mainfrom
Open
Conversation
halfcrazy
force-pushed
the
json-adapter
branch
from
3414a80 to
e8ff116
Compare
Collaborator
halfcrazy
force-pushed
the
json-adapter
branch
2 times, most recently
from
6a21b6a to
666932d
Compare
jcaamano
reviewed
Aug 5, 2025
go.mod
Outdated
Comment on lines
+11
to
+13
| github.com/goccy/go-json v0.10.5 | ||
| github.com/google/uuid v1.6.0 | ||
| github.com/json-iterator/go v1.1.12 |
Collaborator
There was a problem hiding this comment.
I am just wondering, given that the new deps are going to be end up downloaded by every one...
- What's our interest in keeping
github.com/json-iterator/gosincegithub.com/goccy/go-jsonis faster in every aspect? - What would be holding us off to switch to
github.com/goccy/go-jsonas default?
Contributor
Author
There was a problem hiding this comment.
I'm just picking up this MR, as I'm doing some refactoring and plan to drop json-iterator since this repo is no longer maintained.
If offering a default choice, I prefer switching to enconding/json/v2.
A brief benchmark can be found in https://github.com/go-json-experiment/jsonbench.
Contributor
Author
There was a problem hiding this comment.
Providing the choice to the user is learned from gin.
Collaborator
There was a problem hiding this comment.
Thanks for that link, it paints a clear picture of why we wouldn't go github.com/goccy/go-json as default purely based on correctness.
Contributor
Author
There was a problem hiding this comment.
Based on the benchmark result from https://github.com/go-json-experiment/jsonbench/blob/master/README.md, I'm going to use JSONv2(safe) as the default JSON serializer library and allow users to switch to Sonic JSON (unsafe but have best performance in general). What do you think？
SonicJSON is github.com/bytedance/sonic
JSONv2 is github.com/go-json-experiment/json
Collaborator
There was a problem hiding this comment.
JSONv2 is github.com/go-json-experiment/json
is that the same as encoding/json/v2 ? Shouldn't we wait until it is not considered experimental to default to it?
Contributor
Author
There was a problem hiding this comment.
Yes, it's on the way to go 1.25 golang/go#71845 in the next week.
edit: based on my test, jsonv2 shows poor performance. 😳
$ benchstat bench.out.std bench.out.jsonv2 bench.out.sonic bench.out.go_json
goos: linux
goarch: amd64
pkg: github.com/ovn-kubernetes/libovsdb/internal/json
cpu: Intel(R) Core(TM) i7-1065G7 CPU @ 1.30GHz
│ bench.out.std │ bench.out.jsonv2 │ bench.out.sonic │ bench.out.go_json │
│ sec/op │ sec/op vs base │ sec/op vs base │ sec/op vs base │
SetMarshalJSONOperations1-4 5.724µ ± ∞ ¹ 7.423µ ± ∞ ¹ ~ (p=0.100 n=3) ² 3.329µ ± ∞ ¹ ~ (p=0.100 n=3) ² 5.603µ ± ∞ ¹ ~ (p=1.000 n=3) ²
SetMarshalJSONOperations2-4 11.333µ ± ∞ ¹ 16.276µ ± ∞ ¹ ~ (p=0.100 n=3) ² 6.799µ ± ∞ ¹ ~ (p=0.100 n=3) ² 7.014µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONOperations3-4 16.215µ ± ∞ ¹ 18.335µ ± ∞ ¹ ~ (p=0.100 n=3) ² 9.745µ ± ∞ ¹ ~ (p=0.100 n=3) ² 10.732µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONOperations5-4 26.96µ ± ∞ ¹ 30.10µ ± ∞ ¹ ~ (p=0.100 n=3) ² 15.79µ ± ∞ ¹ ~ (p=0.100 n=3) ² 15.22µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONOperations8-4 43.23µ ± ∞ ¹ 48.35µ ± ∞ ¹ ~ (p=0.100 n=3) ² 26.02µ ± ∞ ¹ ~ (p=0.100 n=3) ² 24.52µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONTableUpdate1-4 1864.0n ± ∞ ¹ 1177.0n ± ∞ ¹ ~ (p=0.100 n=3) ² 599.5n ± ∞ ¹ ~ (p=0.100 n=3) ² 556.2n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONTableUpdate2-4 2021.0n ± ∞ ¹ 1285.0n ± ∞ ¹ ~ (p=0.100 n=3) ² 691.7n ± ∞ ¹ ~ (p=0.100 n=3) ² 574.2n ± ∞ ¹ ~ (p=0.100 n=3) ²
geomean 8.955µ 9.025µ +0.78% 4.503µ -49.71% 4.695µ -47.57%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
pkg: github.com/ovn-kubernetes/libovsdb/ovsdb
│ bench.out.std │ bench.out.jsonv2 │ bench.out.sonic │ bench.out.go_json │
│ sec/op │ sec/op vs base │ sec/op vs base │ sec/op vs base │
MapMarshalJSON1-4 1331.0n ± ∞ ¹ 1409.0n ± ∞ ¹ ~ (p=0.400 n=3) ² 929.1n ± ∞ ¹ ~ (p=0.100 n=3) ² 942.7n ± ∞ ¹ ~ (p=0.100 n=3) ²
MapMarshalJSON2-4 1.804µ ± ∞ ¹ 1.732µ ± ∞ ¹ ~ (p=0.200 n=3) ² 1.253µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.282µ ± ∞ ¹ ~ (p=0.100 n=3) ²
MapMarshalJSON3-4 2.214µ ± ∞ ¹ 2.044µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.532µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.657µ ± ∞ ¹ ~ (p=0.100 n=3) ²
MapMarshalJSON5-4 3.160µ ± ∞ ¹ 2.772µ ± ∞ ¹ ~ (p=0.100 n=3) ² 2.194µ ± ∞ ¹ ~ (p=0.100 n=3) ² 2.568µ ± ∞ ¹ ~ (p=0.700 n=3) ²
MapMarshalJSON8-4 4.396µ ± ∞ ¹ 3.625µ ± ∞ ¹ ~ (p=0.100 n=3) ² 2.788µ ± ∞ ¹ ~ (p=0.100 n=3) ² 3.452µ ± ∞ ¹ ~ (p=0.100 n=3) ²
MapUnmarshalJSON1-4 1.902µ ± ∞ ¹ 1.476µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.334µ ± ∞ ¹ ~ (p=0.100 n=3) ² 4.324µ ± ∞ ¹ ~ (p=0.100 n=3) ²
MapUnmarshalJSON2-4 2.401µ ± ∞ ¹ 1.862µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.869µ ± ∞ ¹ ~ (p=0.100 n=3) ² 2.449µ ± ∞ ¹ ~ (p=1.000 n=3) ²
MapUnmarshalJSON3-4 3.100µ ± ∞ ¹ 2.412µ ± ∞ ¹ ~ (p=0.100 n=3) ² 2.037µ ± ∞ ¹ ~ (p=0.100 n=3) ² 5.454µ ± ∞ ¹ ~ (p=0.100 n=3) ²
MapUnmarshalJSON5-4 4.413µ ± ∞ ¹ 3.327µ ± ∞ ¹ ~ (p=0.100 n=3) ² 3.161µ ± ∞ ¹ ~ (p=0.100 n=3) ² 3.375µ ± ∞ ¹ ~ (p=0.100 n=3) ²
MapUnmarshalJSON8-4 6.387µ ± ∞ ¹ 4.646µ ± ∞ ¹ ~ (p=0.100 n=3) ² 4.288µ ± ∞ ¹ ~ (p=0.100 n=3) ² 5.618µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONString1-4 326.0n ± ∞ ¹ 448.3n ± ∞ ¹ ~ (p=0.100 n=3) ² 336.1n ± ∞ ¹ ~ (p=0.200 n=3) ² 271.4n ± ∞ ¹ ~ (p=0.700 n=3) ²
SetMarshalJSONString2-4 1001.0n ± ∞ ¹ 1171.0n ± ∞ ¹ ~ (p=0.100 n=3) ² 709.2n ± ∞ ¹ ~ (p=0.100 n=3) ² 1112.0n ± ∞ ¹ ~ (p=0.700 n=3) ²
SetMarshalJSONString3-4 1065.0n ± ∞ ¹ 1356.0n ± ∞ ¹ ~ (p=0.100 n=3) ² 763.6n ± ∞ ¹ ~ (p=0.100 n=3) ² 1020.0n ± ∞ ¹ ~ (p=0.700 n=3) ²
SetMarshalJSONString5-4 1311.0n ± ∞ ¹ 1352.0n ± ∞ ¹ ~ (p=0.200 n=3) ² 852.6n ± ∞ ¹ ~ (p=0.100 n=3) ² 864.8n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONString8-4 1695.0n ± ∞ ¹ 1569.0n ± ∞ ¹ ~ (p=0.100 n=3) ² 971.9n ± ∞ ¹ ~ (p=0.100 n=3) ² 1069.0n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONInt1-4 291.2n ± ∞ ¹ 413.3n ± ∞ ¹ ~ (p=0.100 n=3) ² 319.0n ± ∞ ¹ ~ (p=0.200 n=3) ² 212.4n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONInt2-4 1004.0n ± ∞ ¹ 1280.0n ± ∞ ¹ ~ (p=0.100 n=3) ² 687.7n ± ∞ ¹ ~ (p=0.100 n=3) ² 681.1n ± ∞ ¹ ~ (p=0.400 n=3) ²
SetMarshalJSONInt3-4 1095.0n ± ∞ ¹ 1678.0n ± ∞ ¹ ~ (p=0.100 n=3) ² 704.7n ± ∞ ¹ ~ (p=0.100 n=3) ² 874.3n ± ∞ ¹ ~ (p=0.200 n=3) ²
SetMarshalJSONInt5-4 1199.0n ± ∞ ¹ 1993.0n ± ∞ ¹ ~ (p=0.100 n=3) ² 799.4n ± ∞ ¹ ~ (p=0.100 n=3) ² 1214.0n ± ∞ ¹ ~ (p=1.000 n=3) ²
SetMarshalJSONInt8-4 1374.0n ± ∞ ¹ 2133.0n ± ∞ ¹ ~ (p=0.100 n=3) ² 915.3n ± ∞ ¹ ~ (p=0.100 n=3) ² 1078.0n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONFloat1-4 398.0n ± ∞ ¹ 661.6n ± ∞ ¹ ~ (p=0.100 n=3) ² 334.4n ± ∞ ¹ ~ (p=0.100 n=3) ² 334.0n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONFloat2-4 1096.0n ± ∞ ¹ 1318.0n ± ∞ ¹ ~ (p=0.100 n=3) ² 701.2n ± ∞ ¹ ~ (p=0.100 n=3) ² 657.3n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONFloat3-4 1125.0n ± ∞ ¹ 1500.0n ± ∞ ¹ ~ (p=0.700 n=3) ² 715.3n ± ∞ ¹ ~ (p=0.100 n=3) ² 704.4n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONFloat5-4 1707.0n ± ∞ ¹ 1638.0n ± ∞ ¹ ~ (p=0.700 n=3) ² 815.2n ± ∞ ¹ ~ (p=0.100 n=3) ² 843.8n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONFloat8-4 1463.0n ± ∞ ¹ 2094.0n ± ∞ ¹ ~ (p=0.100 n=3) ² 933.4n ± ∞ ¹ ~ (p=0.100 n=3) ² 1144.0n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONUUID1-4 628.6n ± ∞ ¹ 606.7n ± ∞ ¹ ~ (p=1.000 n=3) ² 422.7n ± ∞ ¹ ~ (p=0.100 n=3) ² 359.8n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONUUID2-4 1490.0n ± ∞ ¹ 1436.0n ± ∞ ¹ ~ (p=0.100 n=3) ² 975.5n ± ∞ ¹ ~ (p=0.100 n=3) ² 828.4n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONUUID3-4 2.339µ ± ∞ ¹ 1.573µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.066µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.093µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetMarshalJSONUUID5-4 2.925µ ± ∞ ¹ 1.812µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.040µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.554µ ± ∞ ¹ ~ (p=0.700 n=3) ²
SetMarshalJSONUUID8-4 3.372µ ± ∞ ¹ 2.253µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.304µ ± ∞ ¹ ~ (p=0.100 n=3) ² 2.207µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONString1-4 501.3n ± ∞ ¹ 468.4n ± ∞ ¹ ~ (p=0.400 n=3) ² 533.2n ± ∞ ¹ ~ (p=0.700 n=3) ² 342.9n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONString2-4 1.483µ ± ∞ ¹ 1.155µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.158µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.288µ ± ∞ ¹ ~ (p=0.400 n=3) ²
SetUnmarshalJSONString3-4 1.853µ ± ∞ ¹ 1.350µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.441µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.375µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONString5-4 2.356µ ± ∞ ¹ 1.761µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.974µ ± ∞ ¹ ~ (p=0.700 n=3) ² 1.641µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONString8-4 2.939µ ± ∞ ¹ 2.057µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.775µ ± ∞ ¹ ~ (p=0.100 n=3) ² 2.133µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONInt1-4 493.5n ± ∞ ¹ 482.6n ± ∞ ¹ ~ (p=1.000 n=3) ² 412.3n ± ∞ ¹ ~ (p=0.100 n=3) ² 304.4n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONInt2-4 1.444µ ± ∞ ¹ 1.131µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.166µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.115µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONInt3-4 1.654µ ± ∞ ¹ 1.370µ ± ∞ ¹ ~ (p=0.400 n=3) ² 1.366µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.386µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONInt5-4 2.244µ ± ∞ ¹ 1.642µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.566µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.647µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONInt8-4 2.625µ ± ∞ ¹ 1.927µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.821µ ± ∞ ¹ ~ (p=0.100 n=3) ² 2.176µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONFloat1-4 585.2n ± ∞ ¹ 532.5n ± ∞ ¹ ~ (p=0.200 n=3) ² 444.1n ± ∞ ¹ ~ (p=0.100 n=3) ² 330.7n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONFloat2-4 1.597µ ± ∞ ¹ 1.311µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.231µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.317µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONFloat3-4 1.841µ ± ∞ ¹ 1.550µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.404µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.351µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONFloat5-4 2.462µ ± ∞ ¹ 1.971µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.600µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.765µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONFloat8-4 2.916µ ± ∞ ¹ 2.376µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.872µ ± ∞ ¹ ~ (p=0.100 n=3) ² 2.086µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONUUID1-4 890.2n ± ∞ ¹ 546.0n ± ∞ ¹ ~ (p=0.100 n=3) ² 536.3n ± ∞ ¹ ~ (p=0.100 n=3) ² 425.2n ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONUUID2-4 2.449µ ± ∞ ¹ 1.255µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.230µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.318µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONUUID3-4 3.216µ ± ∞ ¹ 1.522µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.388µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.643µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONUUID5-4 4.420µ ± ∞ ¹ 2.118µ ± ∞ ¹ ~ (p=0.100 n=3) ² 1.698µ ± ∞ ¹ ~ (p=0.100 n=3) ² 2.016µ ± ∞ ¹ ~ (p=0.100 n=3) ²
SetUnmarshalJSONUUID8-4 5.910µ ± ∞ ¹ 2.463µ ± ∞ ¹ ~ (p=0.100 n=3) ² 2.039µ ± ∞ ¹ ~ (p=0.100 n=3) ² 2.715µ ± ∞ ¹ ~ (p=0.100 n=3) ²
geomean 1.635µ 1.452µ -11.18% 1.077µ -34.11% 1.190µ -27.23%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
Contributor
Author
There was a problem hiding this comment.
Another way to avoid introducing all JSON libraries to go.mod is to export those fields to client options.
// internal/json/json.go
var (
Marshal = json.Marshal
Unmarshal = json.Unmarshal
)
type RawMessage = json.RawMessage
README.md
Outdated
|
|
||
| go build your application with -tags go_json or jsoniter | ||
|
|
||
| $ benchstat bench.out.std bench.out.jsoniter bench.out.go_json |
Collaborator
There was a problem hiding this comment.
It would be interesting to have a benchmark on slice of nested structs, that's where we observed in ovn-kubernetes things can get awful pretty quickly. For example a slice of operations with Mutations and Where conditions.
halfcrazy
changed the title
halfcrazy
force-pushed
the
json-adapter
branch
2 times, most recently
from
ea0f898 to
7cbd81b
Compare
halfcrazy
changed the title
halfcrazy
force-pushed
the
json-adapter
branch
from
e33dacf to
1e6b114
Compare
Contributor
Author
|
Now use sonic as the default. |
Signed-off-by: Yan Zhu <[email protected]>
halfcrazy
force-pushed
the
json-adapter
branch
from
1e6b114 to
eb15ef4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix https://github.com/ovn-org/libovsdb/issues/345
Use sonic as the default JSON lib.