Sonic slower than stdlib go in encoding

[d0x7]

I've written a benchmark to compare MessagePack, stdlib Json, Sonic Json and Protobuf to decide on a codec for a project, but I'm consistently getting unexpected results for Sonic vs. Go Json.

The decoding is faster than stdlib, just as expected. Advertised is 3,2x faster, I'm getting 3,2x faster, up to 3,9x faster.
But the encoding is weirdly slow. Go Json is 2x faster than sonic.

I thought it might be because the objects aren't really that complex, but the README says that:

For all sizes of json and all scenarios of usage, Sonic performs best.

So I would have thought that includes my scenario as well.

For the single benchmark I'm en/decoding the following item:

var ExampleItem = Item{
	ID:    1,
	Title: "Item 1",
	Desc:  "Description for item 1",
	Price: 10.5,
	Variants: []string{
		"Red",
		"Blue",
		"Green",
	},
}

The multiple one is using an type Items []Item with similar values to the item above.

Here are the benchmark results (along with the proto and msgpack ones)

bytes/obj is a metric I added to compare the resulting size.
Just to rule out the issue is with my setup, my struct or my code, I've also added Go Json and Json Iterator to the benchmark, as in Sonic' README.
If needed, I can add the benchmark code, but it's the exact same function for all (expect proto for obvious reasons), just with the respective (Un)Marshal function passed in, so basically encode(b, sonic.Marshal), etc.

❯ task bench                                    
task: [bench] go test -bench . -benchmem ./...
goos: darwin
goarch: arm64
pkg: json-messagepack-protobuf-bench
cpu: Apple M1 Pro
BenchmarkEncodeSingleMessagePack-8     	2745820	      432.3 ns/op      	  264 B/op	      4 allocs/op	      86.00 bytes/obj
BenchmarkEncodeSingleStdJson-8         	3445540	      351.9 ns/op      	  112 B/op	      1 allocs/op	      104.0 bytes/obj
BenchmarkEncodeSingleGoJson-8          	5325096	      223.0 ns/op      	  112 B/op	      1 allocs/op	      104.0 bytes/obj
BenchmarkEncodeSingleJsonIter-8        	3624213	      330.3 ns/op      	  120 B/op	      2 allocs/op	      104.0 bytes/obj
BenchmarkEncodeSingleSonicJson-8       	1702731	      704.5 ns/op     	  130 B/op	      2 allocs/op	      104.0 bytes/obj
BenchmarkEncodeSingleProtobuf-8        	7452826	      160.0 ns/op          64 B/op	      1 allocs/op	      57.00 bytes/obj

BenchmarkEncodeMultipleMessagePack-8   	1440144	      829.8 ns/op    	  544 B/op	      6 allocs/op	      177.0 bytes/obj
BenchmarkEncodeMultipleStdJson-8       	1808730	      661.9 ns/op         224 B/op	      1 allocs/op	      215.0 bytes/obj
BenchmarkEncodeMultipleGoJson-8        	2724446	      439.8 ns/op      	  224 B/op	      1 allocs/op	      215.0 bytes/obj
BenchmarkEncodeMultipleJsonIter-8      	1971601	      607.4 ns/op    	  232 B/op	      2 allocs/op	      215.0 bytes/obj
BenchmarkEncodeMultipleSonicJson-8     	 876168	      1343  ns/op         243 B/op	      2 allocs/op	      215.0 bytes/obj
BenchmarkEncodeMultipleProtobuf-8      	3691023	      324.7 ns/op         128 B/op	      1 allocs/op	      122.0 bytes/obj

BenchmarkDecodeSingleMessagePack-8     	2207197	      540.8 ns/op          93 B/op	      6 allocs/op
BenchmarkDecodeSingleStdJson-8         	 820467	      1451  ns/op         280 B/op	     10 allocs/op
BenchmarkDecodeSingleGoJson-8          	4756185	      250.8 ns/op         112 B/op	      1 allocs/op
BenchmarkDecodeSingleJsonIter-8        	2541705	      472.8 ns/op          72 B/op	     10 allocs/op
BenchmarkDecodeSingleSonicJson-8       	2648294	      454.7 ns/op         276 B/op	      3 allocs/op
BenchmarkDecodeSingleProtobuf-8        	3499012	      339.4 ns/op         157 B/op	      8 allocs/op

BenchmarkDecodeMultipleMessagePack-8   	1000000	      1105  ns/op         168 B/op	     12 allocs/op
BenchmarkDecodeMultipleStdJson-8       	 399679	      2965  ns/op         368 B/op	     16 allocs/op
BenchmarkDecodeMultipleGoJson-8        	2261152	      546.3 ns/op         224 B/op	      1 allocs/op
BenchmarkDecodeMultipleJsonIter-8      	1242776	      963.0 ns/op         160 B/op	     20 allocs/op
BenchmarkDecodeMultipleSonicJson-8     	1600255	      743.0 ns/op         400 B/op	      3 allocs/op
BenchmarkDecodeMultipleProtobuf-8      	1386304	      848.4 ns/op         568 B/op	     20 allocs/op
PASS
ok  	json-messagepack-protobuf-bench	39.391s

Does anyone has an idea why the encoding is so much faster in the stdlib Json, instead of Sonic like how I expected it to be?

[hanjk1234]

I’ve encountered the same issue while benchmarking Sonic against Go’s standard library JSON encoder
Go Version : go1.21.3 linux/amd64

type EventMetricsTab struct {
	MetricsID      uint64 `json:"metrics_id"`
	MetricsSource  int8   `json:"metrics_source"`
	EventId        uint64 `json:"event_id"`
	MetricsMode    int8   `json:"metrics_mode"`
	Status         int8   `json:"metrics_status"`
	Type           int8   `json:"metrics_type"`
	Params         string `json:"params"`
	ValueType      int8   `json:"value_type"`
	ItemValueType  int8   `json:"item_value_type"`
	MetricsName    string `json:"metrics_name"`
	Creator        string `json:"creator"`
	CreateTime     uint   `json:"create_time"`
	LastModifyTime uint   `json:"last_modify_time"`
	DeletedAt      *uint  `json:"deleted_at"`
	MetricsDesc    string `json:"metrics_desc"`

	MetricsServiceID        uint64 `json:"metrics_service_id"`
	ForceReport             int8   `json:"force_report"`
	ForceReportMC           int8   `json:"force_report_mc"`
	ForceReportPL           int8   `json:"force_report_pl"`
	ForceReportFI           int8   `json:"force_report_fi"`
	ForceReportPLExpiration uint32 `json:"force_report_pl_expiration"`
	WriteParams             string `json:"write_params"`
	ReviewStatus            int8   `json:"review_status"`
	NameForPL               string `json:"name_for_pl"`
	Regions                 string `json:"regions"`
	DefaultOutcome          string `json:"default_outcome"`
}


// 测试 JSON 序列化性能
func BenchmarkMetricsMarshal(b *testing.B) {
	// 加载测试数据，用于先反序列化获取对象
	jsonData, err := os.ReadFile("metrics_testdata_100kb.json")
	if err != nil {
		b.Fatalf("无法加载测试数据: %v", err)
	}

	// 解析为对象
	var metrics []model.EventMetricsTab
	err = json.Unmarshal(jsonData, &metrics)
	if err != nil {
		b.Fatalf("无法解析测试数据: %v", err)
	}
	b.Logf("测试数据包含 %d 个指标对象", len(metrics))

	// gojson序列化测试
	b.Run("gojson-Marshal-DisbleHtml", func(b *testing.B) {
		b.ReportAllocs()
		b.ResetTimer()

		for i := 0; i < b.N; i++ {
			_, err := gojson.MarshalWithOption(metrics, gojson.DisableHTMLEscape(), gojson.DisableNormalizeUTF8())
			if err != nil {
				b.Fatal(err)
			}
		}
	})
	// gojson2序列化测试
	b.Run("gojson-Marshal-default", func(b *testing.B) {
		b.ReportAllocs()
		b.ResetTimer()

		for i := 0; i < b.N; i++ {
			_, err := gojson.Marshal(metrics)
			if err != nil {
				b.Fatal(err)
			}
		}
	})
	// 标准库序列化测试
	b.Run("Standard-Marshal", func(b *testing.B) {
		b.ReportAllocs()
		b.ResetTimer()

		for i := 0; i < b.N; i++ {
			_, err := json.Marshal(metrics)
			if err != nil {
				b.Fatal(err)
			}
		}
	})

	// JSON-Iterator 序列化测试
	b.Run("JSON-Iterator-Marshal", func(b *testing.B) {
		b.ReportAllocs()
		b.ResetTimer()

		for i := 0; i < b.N; i++ {
			_, err := jsoniter.Marshal(metrics)
			if err != nil {
				b.Fatal(err)
			}
		}
	})

	// Sonic 序列化测试
	b.Run("Sonic-Marshal", func(b *testing.B) {
		b.ReportAllocs()
		b.ResetTimer()
		for i := 0; i < b.N; i++ {
			_, err := sonic_json.FastStringifier.Marshal(metrics)
			if err != nil {
				b.Fatal(err)
			}
		}
	})
}

[liuq19]

thanks, we will investigate it

[liuq19]

@hanjk1234 could you provide the json data and go env? I cannot reproduce the result on Linux/amd64

I used mocked data and my output is

cpu: AMD EPYC 7Y83 64-Core Processor                
BenchmarkMetricsMarshal
    benchmark_test.go:125: 测试数据包含 100 个指标对象
BenchmarkMetricsMarshal/gojson-Marshal-DisbleHtml
BenchmarkMetricsMarshal/gojson-Marshal-DisbleHtml-4                19160         62332 ns/op       73799 B/op          2 allocs/op
BenchmarkMetricsMarshal/gojson-Marshal-default
BenchmarkMetricsMarshal/gojson-Marshal-default-4                   17641         68165 ns/op       74013 B/op          2 allocs/op
BenchmarkMetricsMarshal/Standard-Marshal
BenchmarkMetricsMarshal/Standard-Marshal-4                          9410        118859 ns/op       73880 B/op          2 allocs/op
BenchmarkMetricsMarshal/JSON-Iterator-Marshal
BenchmarkMetricsMarshal/JSON-Iterator-Marshal-4                     7172        158637 ns/op       73846 B/op          2 allocs/op
BenchmarkMetricsMarshal/Sonic-Marshal
BenchmarkMetricsMarshal/Sonic-Marshal-4                            27506         47385 ns/op       76919 B/op          3 allocs/op
PASS
ok      github.com/bytedance/sonic/generic_test 7.780s
➜  sonic git:(main) ✗ go test -v -bench=BenchmarkMetricsMarsha

[hanjk1234]

have some update?

[hanjk1234]

metric_data.json
sorry, data file coming~

go version
go version go1.21.13 darwin/arm64

芯片： Apple M4 Pro
核总数： 14（10性能和4能效）
内存： 24 GB