Skip to content

Commit 34a1f41

Browse files
committed
Improved readme
1 parent afbe130 commit 34a1f41

File tree

1 file changed

+35
-15
lines changed

1 file changed

+35
-15
lines changed

README.md

Lines changed: 35 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -19,19 +19,23 @@ SELECT protobuf_query_multi('MyProto:some_repeated_field[*].some_map[*].some_fie
1919

2020
## Why put protobufs in a database?
2121

22-
Protobufs in a database is in many ways similar to JSON in a database.
23-
The main advantage of protobufs is a compact and efficient representation.
22+
Protobufs in a database is in many ways similar to JSON in a database:
23+
- less code needed to convert data structures into database rows and back
24+
- less alter tables required as data fields are added and removed
25+
26+
The main advantage of protobufs over JSON is a compact and efficient representation.
2427
The main advantage of JSON is human-readability without extra steps.
2528

2629
### What about storing protobufs as JSON?
2730

2831
Protobufs have a well-defined JSON representation supported by most implementations.
29-
Using that for storage is a valid strategy if efficiency is not a major concern.
32+
Using that for storage (as [`json` or `jsonb`](https://www.postgresql.org/docs/current/datatype-json.html))
33+
is a valid strategy if efficiency is not a major concern.
3034

3135
Note that JSON protobufs store fields by name instead of by number,
3236
which means that field renames will break backwards- and forwards-compatibility.
3337

34-
## Installation
38+
## Installation from source
3539

3640
Requires Postgres 11 or newer.
3741

@@ -53,6 +57,19 @@ Then in Postgres:
5357
CREATE EXTENSION postgres_protobuf;
5458
```
5559

60+
## Installation from binary release
61+
62+
Prebuilt binaries are [here](https://github.com/mpartel/postgres-protobuf/releases).
63+
64+
Currently they are built with and tested on Ubuntu 18.04 only.
65+
You can build your own binary package from source with `make dist`.
66+
67+
On Ubuntu, install the contents of the binary package like this:
68+
- copy the contents of `lib/` to `/usr/lib/postgresql/11/lib/`
69+
- copy the contents of `extension/` to `/usr/share/postgresql/11/extension/`
70+
71+
Other distros may have those Postgres directories elsewhere.
72+
5673
## Usage
5774

5875
First, you need to tell the extension about your protobuf schema.
@@ -66,6 +83,7 @@ VALUES ('default', contents_of_file)
6683
```
6784

6885
where `contents_of_file` is a Postgres [byte array](https://www.postgresql.org/docs/current/datatype-binary.html).
86+
Commit the transaction if you're in one.
6987

7088
Now you can query protobufs described by your schema like this:
7189

@@ -105,8 +123,8 @@ before relying on it too heavily in a design.
105123

106124
This extension is fairly new and written in C++, so some caution is warranted.
107125
It's may be unwise to give it untrusted queries or protobuf data that you haven't
108-
parsed and reserialized first. Conversion to/from JSON should be safe, but see
109-
the note about memory management below.
126+
parsed and reserialized first. Conversion to/from JSON should be safer since it
127+
thinly wraps the well-tested protobuf library, but see the note about memory management below.
110128

111129
### Performance
112130

@@ -130,12 +148,14 @@ This memory might not be properly accounted for by Postgres's
130148
memory management and monitoring systems.
131149

132150
While care has been taken to avoid memory leaks and to tolerate memory exhaustion at any point,
133-
this has not been rigorously tested.
151+
this has not been rigorously tested, and I am unsure whether the protobuf library always cleans up
152+
correctly after a `bad_alloc` exception.
134153

135154
In the current version, protobuf [map](https://developers.google.com/protocol-buffers/docs/proto3#maps)
136-
values are buffered before being scanned. This means that huge protobufs whose bulk is under a
155+
values are buffered before being scanned by queries. This means that huge protobufs whose bulk is under a
137156
highly nested map may take a lot of memory. Note that an attacker can easily construct such a
138-
protobuf if the schema has recursion involving a map.
157+
protobuf if the schema has recursion involving a map. This caveat applies only to queries,
158+
not to JSON conversion.
139159

140160
Other than the above case, memory use is linear, or roughly
141161
`O(|descriptor sets| + |largest protobuf queried| + |result set|)`.
@@ -144,24 +164,24 @@ Other than the above case, memory use is linear, or roughly
144164

145165
Protobuf versions 2 and 3 should both work,
146166
but [groups](https://developers.google.com/protocol-buffers/docs/proto#groups)
147-
(a deprecated protobuf feature)
148-
are currently not supported and will cause the query to fail if encountered.
167+
(which have been deprecated for a long time)
168+
are currently not supported and will cause queries to fail if encountered.
149169

150170
Note that proto3 does not store default values for fields.
151171
The current implementation returns no result for missing values.
152172
This means that e.g. a numeric field whose value is 0 will not show up in results.
153-
A future version may change this default behaviour.
173+
A future version may change this default.
154174

155175
This extension has so far only been tested on AMD64.
156176

157177
### Advanced operations
158178

159-
The current version has no support for modifying the contents of protobufs,
179+
The current version does not provide functions for modifying the contents of protobufs,
160180
and there are no concrete plans to add such functonality.
161181
There are also no concrete plans to significantly extend the query language.
162182

163-
If you do need these features occasionally, consider converting to JSON,
164-
for which Postgres has [a wide range of operations](https://www.postgresql.org/docs/current/functions-json.html).
183+
If you do need these features occasionally, consider converting to JSON and back,
184+
since Postgres has [a wide range of JSON operations](https://www.postgresql.org/docs/current/functions-json.html).
165185

166186
## Comparison with pg_protobuf
167187

0 commit comments

Comments
 (0)