You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+8-17Lines changed: 8 additions & 17 deletions
Original file line number
Diff line number
Diff line change
@@ -184,17 +184,18 @@ CREATE TABLE http (
184
184
)
185
185
```
186
186
187
-
####HTTP status code handler (currently supported only for HTTP Sink)
188
-
Http Sink connector allows defining list of HTTP status codes that should be treated as errors.
187
+
## HTTP status code handler
188
+
Http Sink and Lookup Source connectors allow defining list of HTTP status codes that should be treated as errors.
189
189
By default all 400s and 500s response codes will be interpreted as error code.
190
190
191
-
This behavior can be changed by using below properties in table definition (DDL) or passing it via
191
+
This behavior can be changed by using below properties in table definition (DDL) for Sink and Lookup Source or passing it via
192
192
`setProperty' method from Sink's builder. The property names are:
193
-
-`gid.connector.http.sink.error.code` used to defined HTTP status code value that should be treated as error for example 404.
193
+
-`gid.connector.http.sink.error.code`and `gid.connector.http.source.lookup.error.code`used to defined HTTP status code value that should be treated as error for example 404.
194
194
Many status codes can be defined in one value, where each code should be separated with comma, for example:
195
195
`401, 402, 403`. User can use this property also to define a type code mask. In that case, all codes from given HTTP response type will be treated as errors.
196
196
An example of such a mask would be `3XX, 4XX, 5XX`. In this case, all 300s, 400s and 500s status codes will be treated as errors.
197
-
-`gid.connector.http.sink.error.code.exclude` used to exclude a HTTP code from error list. Many status codes can be defined in one value, where each code should be separated with comma, for example:
197
+
-`gid.connector.http.sink.error.code.exclude` and `gid.connector.http.source.lookup.error.code.exclude` used to exclude a HTTP code from error list.
198
+
Many status codes can be defined in one value, where each code should be separated with comma, for example:
198
199
`401, 402, 403`. In this example, codes 401, 402 and 403 would not be interpreted as error codes.
199
200
200
201
@@ -291,21 +292,13 @@ Issue was discussed on Flink's user mailing list - https://lists.apache.org/thre
291
292
Implementation of an HTTP Sink is based on Flink's `AsyncSinkBase` introduced in Flink 1.15 [3, 4].
292
293
293
294
#### Http Response to Table schema mapping
294
-
The mapping from Http Json Response to SQL table schema is done via Json Paths [5].
295
-
This is achieved thanks to `com.jayway.jsonpath:json-path` library.
296
-
297
-
If no `root` or `field.#.path` option is defined, the connector will use the column name as json path and will try to look for Json Node with that name in received Json. If no node with a given name is found, the connector will return `null` as value for this field.
298
-
299
-
If the `field.#.path` option is defined, connector will use given Json path from option's value in order to find Json data that should be used for this column.
300
-
For example `'field.isActive.path' = '$.details.isActive'` - the value for table column `isActive` will be taken from `$.details.isActive` node from received Json.
295
+
The mapping from Http Json Response to SQL table schema is done via Flink's Json Format [5].
301
296
302
297
## TODO
303
298
304
299
### HTTP TableLookup Source
305
300
- Implement caches.
306
-
- Add support for other Flink types. Currently, STRING type is only fully supported.
307
301
- Think about Retry Policy for Http Request
308
-
- Use Flink Format [7] to parse Json response
309
302
- Add Configurable Timeout value
310
303
- Check other `//TODO`'s.
311
304
@@ -321,9 +314,7 @@ For example `'field.isActive.path' = '$.details.isActive'` - the value for table
0 commit comments