Skip to content

Commit e52cb40

Browse files
paralinparalin
committed
chore: change more forked paths
Signed-off-by: Christian Stewart <christian@aperture.us>
1 parent e0a55a4 commit e52cb40

File tree

6 files changed

+45
-174
lines changed

6 files changed

+45
-174
lines changed

‎README.md

Lines changed: 15 additions & 139 deletions
Original file line numberDiff line numberDiff line change
@@ -19,28 +19,28 @@
1919
See [benchmarks](#benchmarks).
2020
* Parses arbitrary JSON without schema, reflection, struct magic and code generation
2121
contrary to [easyjson](https://github.com/mailru/easyjson).
22-
* Provides simple [API](http://godoc.org/github.com/valyala/fastjson).
22+
* Provides simple [API](http://godoc.org/github.com/aperturerobotics/fastjson).
2323
* Outperforms [jsonparser](https://github.com/buger/jsonparser) and [gjson](https://github.com/tidwall/gjson)
2424
when accessing multiple unrelated fields, since `fastjson` parses the input JSON only once.
2525
* Validates the parsed JSON unlike [jsonparser](https://github.com/buger/jsonparser)
2626
and [gjson](https://github.com/tidwall/gjson).
2727
* May quickly extract a part of the original JSON with `Value.Get(...).MarshalTo` and modify it
28-
with [Del](https://godoc.org/github.com/valyala/fastjson#Value.Del)
29-
and [Set](https://godoc.org/github.com/valyala/fastjson#Value.Set) functions.
28+
with [Del](https://godoc.org/github.com/aperturerobotics/fastjson#Value.Del)
29+
and [Set](https://godoc.org/github.com/aperturerobotics/fastjson#Value.Set) functions.
3030
* May parse array containing values with distinct types (aka non-homogenous types).
3131
For instance, `fastjson` easily parses the following JSON array `[123, "foo", [456], {"k": "v"}, null]`.
3232
* `fastjson` preserves the original order of object items when calling
33-
[Object.Visit](https://godoc.org/github.com/valyala/fastjson#Object.Visit).
33+
[Object.Visit](https://godoc.org/github.com/aperturerobotics/fastjson#Object.Visit).
3434

3535

3636
## Known limitations
3737

3838
* Requies extra care to work with - references to certain objects recursively
39-
returned by [Parser](https://godoc.org/github.com/valyala/fastjson#Parser)
40-
must be released before the next call to [Parse](https://godoc.org/github.com/valyala/fastjson#Parser.Parse).
41-
Otherwise the program may work improperly. The same applies to objects returned by [Arena](https://godoc.org/github.com/valyala/fastjson#Arena).
42-
Adhere recommendations from [docs](https://godoc.org/github.com/valyala/fastjson).
43-
* Cannot parse JSON from `io.Reader`. There is [Scanner](https://godoc.org/github.com/valyala/fastjson#Scanner)
39+
returned by [Parser](https://godoc.org/github.com/aperturerobotics/fastjson#Parser)
40+
must be released before the next call to [Parse](https://godoc.org/github.com/aperturerobotics/fastjson#Parser.Parse).
41+
Otherwise the program may work improperly. The same applies to objects returned by [Arena](https://godoc.org/github.com/aperturerobotics/fastjson#Arena).
42+
Adhere recommendations from [docs](https://godoc.org/github.com/aperturerobotics/fastjson).
43+
* Cannot parse JSON from `io.Reader`. There is [Scanner](https://godoc.org/github.com/aperturerobotics/fastjson#Scanner)
4444
for parsing stream of JSON values from a string.
4545

4646

@@ -82,7 +82,7 @@ Accessing multiple fields with error handling:
8282
// arr.1=foo
8383
```
8484

85-
See also [examples](https://godoc.org/github.com/valyala/fastjson#pkg-examples).
85+
See also [examples](https://godoc.org/github.com/aperturerobotics/fastjson#pkg-examples).
8686

8787

8888
## Security
@@ -96,139 +96,15 @@ See also [examples](https://godoc.org/github.com/valyala/fastjson#pkg-examples).
9696

9797
## Performance optimization tips
9898

99-
* Re-use [Parser](https://godoc.org/github.com/valyala/fastjson#Parser) and [Scanner](https://godoc.org/github.com/valyala/fastjson#Scanner)
99+
* Re-use [Parser](https://godoc.org/github.com/aperturerobotics/fastjson#Parser) and [Scanner](https://godoc.org/github.com/aperturerobotics/fastjson#Scanner)
100100
for parsing many JSONs. This reduces memory allocations overhead.
101-
[ParserPool](https://godoc.org/github.com/valyala/fastjson#ParserPool) may be useful in this case.
102-
* Prefer calling `Value.Get*` on the value returned from [Parser](https://godoc.org/github.com/valyala/fastjson#Parser)
101+
[ParserPool](https://godoc.org/github.com/aperturerobotics/fastjson#ParserPool) may be useful in this case.
102+
* Prefer calling `Value.Get*` on the value returned from [Parser](https://godoc.org/github.com/aperturerobotics/fastjson#Parser)
103103
instead of calling `Get*` one-liners when multiple fields
104104
must be obtained from JSON, since each `Get*` one-liner re-parses
105105
the input JSON again.
106-
* Prefer calling once [Value.Get](https://godoc.org/github.com/valyala/fastjson#Value.Get)
106+
* Prefer calling once [Value.Get](https://godoc.org/github.com/aperturerobotics/fastjson#Value.Get)
107107
for common prefix paths and then calling `Value.Get*` on the returned value
108108
for distinct suffix paths.
109-
* Prefer iterating over array returned from [Value.GetArray](https://godoc.org/github.com/valyala/fastjson#Object.Visit)
109+
* Prefer iterating over array returned from [Value.GetArray](https://godoc.org/github.com/aperturerobotics/fastjson#Object.Visit)
110110
with a range loop instead of calling `Value.Get*` for each array item.
111-
112-
## Fuzzing
113-
Install [go-fuzz](https://github.com/dvyukov/go-fuzz) & optionally the go-fuzz-corpus.
114-
115-
```bash
116-
go get -u github.com/dvyukov/go-fuzz/go-fuzz github.com/dvyukov/go-fuzz/go-fuzz-build
117-
```
118-
119-
Build using `go-fuzz-build` and run `go-fuzz` with an optional corpus.
120-
121-
```bash
122-
mkdir -p workdir/corpus
123-
cp $GOPATH/src/github.com/dvyukov/go-fuzz-corpus/json/corpus/* workdir/corpus
124-
go-fuzz-build github.com/valyala/fastjson
125-
go-fuzz -bin=fastjson-fuzz.zip -workdir=workdir
126-
```
127-
128-
## Benchmarks
129-
130-
Go 1.12 has been used for benchmarking.
131-
132-
Legend:
133-
134-
* `small` - parse [small.json](testdata/small.json) (190 bytes).
135-
* `medium` - parse [medium.json](testdata/medium.json) (2.3KB).
136-
* `large` - parse [large.json](testdata/large.json) (28KB).
137-
* `canada` - parse [canada.json](testdata/canada.json) (2.2MB).
138-
* `citm` - parse [citm_catalog.json](testdata/citm_catalog.json) (1.7MB).
139-
* `twitter` - parse [twitter.json](testdata/twitter.json) (617KB).
140-
141-
* `stdjson-map` - parse into a `map[string]interface{}` using `encoding/json`.
142-
* `stdjson-struct` - parse into a struct containing
143-
a subset of fields of the parsed JSON, using `encoding/json`.
144-
* `stdjson-empty-struct` - parse into an empty struct using `encoding/json`.
145-
This is the fastest possible solution for `encoding/json`, may be used
146-
for json validation. See also benchmark results for json validation.
147-
* `fastjson` - parse using `fastjson` without fields access.
148-
* `fastjson-get` - parse using `fastjson` with fields access similar to `stdjson-struct`.
149-
150-
```
151-
$ GOMAXPROCS=1 go test github.com/valyala/fastjson -bench='Parse$'
152-
goos: linux
153-
goarch: amd64
154-
pkg: github.com/valyala/fastjson
155-
BenchmarkParse/small/stdjson-map 200000 7305 ns/op 26.01 MB/s 960 B/op 51 allocs/op
156-
BenchmarkParse/small/stdjson-struct 500000 3431 ns/op 55.37 MB/s 224 B/op 4 allocs/op
157-
BenchmarkParse/small/stdjson-empty-struct 500000 2273 ns/op 83.58 MB/s 168 B/op 2 allocs/op
158-
BenchmarkParse/small/fastjson 5000000 347 ns/op 547.53 MB/s 0 B/op 0 allocs/op
159-
BenchmarkParse/small/fastjson-get 2000000 620 ns/op 306.39 MB/s 0 B/op 0 allocs/op
160-
BenchmarkParse/medium/stdjson-map 30000 40672 ns/op 57.26 MB/s 10196 B/op 208 allocs/op
161-
BenchmarkParse/medium/stdjson-struct 30000 47792 ns/op 48.73 MB/s 9174 B/op 258 allocs/op
162-
BenchmarkParse/medium/stdjson-empty-struct 100000 22096 ns/op 105.40 MB/s 280 B/op 5 allocs/op
163-
BenchmarkParse/medium/fastjson 500000 3025 ns/op 769.90 MB/s 0 B/op 0 allocs/op
164-
BenchmarkParse/medium/fastjson-get 500000 3211 ns/op 725.20 MB/s 0 B/op 0 allocs/op
165-
BenchmarkParse/large/stdjson-map 2000 614079 ns/op 45.79 MB/s 210734 B/op 2785 allocs/op
166-
BenchmarkParse/large/stdjson-struct 5000 298554 ns/op 94.18 MB/s 15616 B/op 353 allocs/op
167-
BenchmarkParse/large/stdjson-empty-struct 5000 268577 ns/op 104.69 MB/s 280 B/op 5 allocs/op
168-
BenchmarkParse/large/fastjson 50000 35210 ns/op 798.56 MB/s 5 B/op 0 allocs/op
169-
BenchmarkParse/large/fastjson-get 50000 35171 ns/op 799.46 MB/s 5 B/op 0 allocs/op
170-
BenchmarkParse/canada/stdjson-map 20 68147307 ns/op 33.03 MB/s 12260502 B/op 392539 allocs/op
171-
BenchmarkParse/canada/stdjson-struct 20 68044518 ns/op 33.08 MB/s 12260123 B/op 392534 allocs/op
172-
BenchmarkParse/canada/stdjson-empty-struct 100 17709250 ns/op 127.11 MB/s 280 B/op 5 allocs/op
173-
BenchmarkParse/canada/fastjson 300 4182404 ns/op 538.22 MB/s 254902 B/op 381 allocs/op
174-
BenchmarkParse/canada/fastjson-get 300 4274744 ns/op 526.60 MB/s 254902 B/op 381 allocs/op
175-
BenchmarkParse/citm/stdjson-map 50 27772612 ns/op 62.19 MB/s 5214163 B/op 95402 allocs/op
176-
BenchmarkParse/citm/stdjson-struct 100 14936191 ns/op 115.64 MB/s 1989 B/op 75 allocs/op
177-
BenchmarkParse/citm/stdjson-empty-struct 100 14946034 ns/op 115.56 MB/s 280 B/op 5 allocs/op
178-
BenchmarkParse/citm/fastjson 1000 1879714 ns/op 918.87 MB/s 17628 B/op 30 allocs/op
179-
BenchmarkParse/citm/fastjson-get 1000 1881598 ns/op 917.94 MB/s 17628 B/op 30 allocs/op
180-
BenchmarkParse/twitter/stdjson-map 100 11289146 ns/op 55.94 MB/s 2187878 B/op 31266 allocs/op
181-
BenchmarkParse/twitter/stdjson-struct 300 5779442 ns/op 109.27 MB/s 408 B/op 6 allocs/op
182-
BenchmarkParse/twitter/stdjson-empty-struct 300 5738504 ns/op 110.05 MB/s 408 B/op 6 allocs/op
183-
BenchmarkParse/twitter/fastjson 2000 774042 ns/op 815.86 MB/s 2541 B/op 2 allocs/op
184-
BenchmarkParse/twitter/fastjson-get 2000 777833 ns/op 811.89 MB/s 2541 B/op 2 allocs/op
185-
```
186-
187-
Benchmark results for json validation:
188-
189-
```
190-
$ GOMAXPROCS=1 go test github.com/valyala/fastjson -bench='Validate$'
191-
goos: linux
192-
goarch: amd64
193-
pkg: github.com/valyala/fastjson
194-
BenchmarkValidate/small/stdjson 2000000 955 ns/op 198.83 MB/s 72 B/op 2 allocs/op
195-
BenchmarkValidate/small/fastjson 5000000 384 ns/op 493.60 MB/s 0 B/op 0 allocs/op
196-
BenchmarkValidate/medium/stdjson 200000 10799 ns/op 215.66 MB/s 184 B/op 5 allocs/op
197-
BenchmarkValidate/medium/fastjson 300000 3809 ns/op 611.30 MB/s 0 B/op 0 allocs/op
198-
BenchmarkValidate/large/stdjson 10000 133064 ns/op 211.31 MB/s 184 B/op 5 allocs/op
199-
BenchmarkValidate/large/fastjson 30000 45268 ns/op 621.14 MB/s 0 B/op 0 allocs/op
200-
BenchmarkValidate/canada/stdjson 200 8470904 ns/op 265.74 MB/s 184 B/op 5 allocs/op
201-
BenchmarkValidate/canada/fastjson 500 2973377 ns/op 757.07 MB/s 0 B/op 0 allocs/op
202-
BenchmarkValidate/citm/stdjson 200 7273172 ns/op 237.48 MB/s 184 B/op 5 allocs/op
203-
BenchmarkValidate/citm/fastjson 1000 1684430 ns/op 1025.39 MB/s 0 B/op 0 allocs/op
204-
BenchmarkValidate/twitter/stdjson 500 2849439 ns/op 221.63 MB/s 312 B/op 6 allocs/op
205-
BenchmarkValidate/twitter/fastjson 2000 1036796 ns/op 609.10 MB/s 0 B/op 0 allocs/op
206-
```
207-
208-
## FAQ
209-
210-
* Q: _There are a ton of other high-perf packages for JSON parsing in Go. Why create yet another package?_
211-
A: Because other packages require either rigid JSON schema via struct magic
212-
and code generation or perform poorly when multiple unrelated fields
213-
must be obtained from the parsed JSON.
214-
Additionally, `fastjson` provides nicer [API](http://godoc.org/github.com/valyala/fastjson).
215-
216-
* Q: _What is the main purpose for `fastjson`?_
217-
A: High-perf JSON parsing for [RTB](https://www.iab.com/wp-content/uploads/2015/05/OpenRTB_API_Specification_Version_2_3_1.pdf)
218-
and other [JSON-RPC](https://en.wikipedia.org/wiki/JSON-RPC) services.
219-
220-
* Q: _Why fastjson doesn't provide fast marshaling (serialization)?_
221-
A: Actually it provides some sort of marshaling - see [Value.MarshalTo](https://godoc.org/github.com/valyala/fastjson#Value.MarshalTo).
222-
But I'd recommend using [quicktemplate](https://github.com/valyala/quicktemplate#use-cases)
223-
for high-performance JSON marshaling :)
224-
225-
* Q: _`fastjson` crashes my program!_
226-
A: There is high probability of improper use.
227-
* Make sure you don't hold references to objects recursively returned by `Parser` / `Scanner`
228-
beyond the next `Parser.Parse` / `Scanner.Next` call
229-
if such restriction is mentioned in [docs](https://github.com/valyala/fastjson/issues/new).
230-
* Make sure you don't access `fastjson` objects from concurrently running goroutines
231-
if such restriction is mentioned in [docs](https://github.com/valyala/fastjson/issues/new).
232-
* Build and run your program with [-race](https://golang.org/doc/articles/race_detector.html) flag.
233-
Make sure the race detector detects zero races.
234-
* If your program continue crashing after fixing issues mentioned above, [file a bug](https://github.com/valyala/fastjson/issues/new).

‎handy_example_test.go

Lines changed: 9 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,13 @@
1-
package fastjson_test
1+
package fastjson
22

33
import (
44
"fmt"
5-
"github.com/valyala/fastjson"
65
)
76

87
func ExampleGetString() {
98
data := []byte(`{"foo":{"bar":[123,"baz"]}}`)
109

11-
s := fastjson.GetString(data, "foo", "bar", "1")
10+
s := GetString(data, "foo", "bar", "1")
1211
fmt.Printf("data.foo.bar[1] = %s", s)
1312

1413
// Output:
@@ -18,10 +17,10 @@ func ExampleGetString() {
1817
func ExampleGetInt() {
1918
data := []byte(`{"foo": [233,true, {"bar": [2343]} ]}`)
2019

21-
n1 := fastjson.GetInt(data, "foo", "0")
20+
n1 := GetInt(data, "foo", "0")
2221
fmt.Printf("data.foo[0] = %d\n", n1)
2322

24-
n2 := fastjson.GetInt(data, "foo", "2", "bar", "0")
23+
n2 := GetInt(data, "foo", "2", "bar", "0")
2524
fmt.Printf("data.foo[2].bar[0] = %d\n", n2)
2625

2726
// Output:
@@ -32,11 +31,11 @@ func ExampleGetInt() {
3231
func ExampleExists() {
3332
data := []byte(`{"foo": [1.23,{"bar":33,"baz":null}]}`)
3433

35-
fmt.Printf("exists(data.foo) = %v\n", fastjson.Exists(data, "foo"))
36-
fmt.Printf("exists(data.foo[0]) = %v\n", fastjson.Exists(data, "foo", "0"))
37-
fmt.Printf("exists(data.foo[1].baz) = %v\n", fastjson.Exists(data, "foo", "1", "baz"))
38-
fmt.Printf("exists(data.foobar) = %v\n", fastjson.Exists(data, "foobar"))
39-
fmt.Printf("exists(data.foo.bar) = %v\n", fastjson.Exists(data, "foo", "bar"))
34+
fmt.Printf("exists(data.foo) = %v\n", Exists(data, "foo"))
35+
fmt.Printf("exists(data.foo[0]) = %v\n", Exists(data, "foo", "0"))
36+
fmt.Printf("exists(data.foo[1].baz) = %v\n", Exists(data, "foo", "1", "baz"))
37+
fmt.Printf("exists(data.foobar) = %v\n", Exists(data, "foobar"))
38+
fmt.Printf("exists(data.foo.bar) = %v\n", Exists(data, "foo", "bar"))
4039

4140
// Output:
4241
// exists(data.foo) = true

‎parser.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ package fastjson
22

33
import (
44
"fmt"
5-
"github.com/valyala/fastjson/fastfloat"
5+
"github.com/aperturerobotics/fastjson/fastfloat"
66
"strconv"
77
"strings"
88
"unicode/utf16"

‎parser_example_test.go

Lines changed: 9 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,13 @@
1-
package fastjson_test
1+
package fastjson
22

33
import (
44
"fmt"
5-
"github.com/valyala/fastjson"
65
"log"
76
"strconv"
87
)
98

109
func ExampleParser_Parse() {
11-
var p fastjson.Parser
10+
var p Parser
1211
v, err := p.Parse(`{"foo":"bar", "baz": 123}`)
1312
if err != nil {
1413
log.Fatalf("cannot parse json: %s", err)
@@ -21,7 +20,7 @@ func ExampleParser_Parse() {
2120
}
2221

2322
func ExampleParser_Parse_reuse() {
24-
var p fastjson.Parser
23+
var p Parser
2524

2625
// p may be re-used for parsing multiple json strings.
2726
// This improves parsing speed by reducing the number
@@ -61,7 +60,7 @@ func ExampleValue_MarshalTo() {
6160
}
6261
]
6362
}`
64-
var p fastjson.Parser
63+
var p Parser
6564
v, err := p.Parse(s)
6665
if err != nil {
6766
log.Fatalf("cannot parse json: %s", err)
@@ -82,7 +81,7 @@ func ExampleValue_MarshalTo() {
8281

8382
func ExampleValue_Get() {
8483
s := `{"foo":[{"bar":{"baz":123,"x":"434"},"y":[]},[null, false]],"qwe":true}`
85-
var p fastjson.Parser
84+
var p Parser
8685
v, err := p.Parse(s)
8786
if err != nil {
8887
log.Fatalf("cannot parse json: %s", err)
@@ -123,7 +122,7 @@ func ExampleValue_Type() {
123122
"null": null
124123
}`
125124

126-
var p fastjson.Parser
125+
var p Parser
127126
v, err := p.Parse(s)
128127
if err != nil {
129128
log.Fatalf("cannot parse json: %s", err)
@@ -154,7 +153,7 @@ func ExampleObject_Visit() {
154153
"str": "foobar"
155154
}`
156155

157-
var p fastjson.Parser
156+
var p Parser
158157
v, err := p.Parse(s)
159158
if err != nil {
160159
log.Fatalf("cannot parse json: %s", err)
@@ -164,7 +163,7 @@ func ExampleObject_Visit() {
164163
log.Fatalf("cannot obtain object from json value: %s", err)
165164
}
166165

167-
o.Visit(func(k []byte, v *fastjson.Value) {
166+
o.Visit(func(k []byte, v *Value) {
168167
switch string(k) {
169168
case "obj":
170169
fmt.Printf("object %s\n", v)
@@ -187,7 +186,7 @@ func ExampleValue_GetStringBytes() {
187186
[123, "baz"]
188187
]`
189188

190-
var p fastjson.Parser
189+
var p Parser
191190
v, err := p.Parse(s)
192191
if err != nil {
193192
log.Fatalf("cannot parse json: %s", err)

‎scanner_example_test.go

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,12 @@
1-
package fastjson_test
1+
package fastjson
22

33
import (
44
"fmt"
5-
"github.com/valyala/fastjson"
65
"log"
76
)
87

98
func ExampleScanner() {
10-
var sc fastjson.Scanner
9+
var sc Scanner
1110

1211
sc.Init(` {"foo": "bar" }[ ]
1312
12345"xyz" true false null `)
@@ -30,7 +29,7 @@ func ExampleScanner() {
3029
}
3130

3231
func ExampleScanner_reuse() {
33-
var sc fastjson.Scanner
32+
var sc Scanner
3433

3534
// The sc may be re-used in order to reduce the number
3635
// of memory allocations.

0 commit comments

Comments
 (0)