Skip to content

Commit 12b03f6

Browse files
committed
DEVEXP-457 Added docs for error handling
Really not a lot to say here, other than errors being thrown to the user with hopefully helpful error messages.
1 parent 7921c80 commit 12b03f6

File tree

2 files changed

+27
-0
lines changed

2 files changed

+27
-0
lines changed

docs/reading.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,17 @@ spark.readStream \
127127
.load()
128128
```
129129

130+
## Error handling
131+
132+
When reading data with the connector, any error that occurs - whether it is before any data is read or during a request
133+
to MarkLogic to read data - is immediately thrown to the user and the read operation ends without any result being
134+
returned. This is consistent with the design of Spark connectors, where a partition reader is allowed to throw
135+
exceptions and is not given any mechanism for reporting an error and continuing to read data. The connector will strive
136+
to provide meaningful context when an error occurs to assist with debugging the cause of the error.
137+
138+
In practice, it is expected that most errors will be a result of a misconfiguration. For example, the connection and
139+
authentication options may be incorrect, or the Optic DSL query may have a syntax error. Any errors that cannot be
140+
fixed via changes to the options passed to the connector should be reported as new issues to this GitHub repository.
130141

131142
## Tuning performance
132143

docs/writing.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,22 @@ connector and into MarkLogic. This will result 100 new JSON documents in the `st
121121
The ability to stream data into MarkLogic can make Spark an effective tool for obtaining data from a variety of data
122122
sources and writing it directly to MarkLogic.
123123

124+
## Error handling
125+
126+
The connector may throw an error during one of two phases of operation - before it begins to write data to MarkLogic,
127+
and during the writing of data to MarkLogic.
128+
129+
For the first kind of error, the error will be immediately returned to the user and no data will have been written.
130+
Such errors are often due to misconfiguration of the connector options and should be fixable.
131+
132+
For the second kind of error, the error will eventually be returned to the user, usually within seconds of it
133+
occurring. The slight delay is due to the asynchronous nature of data being written by the connector. The error will
134+
be logged by the connector and the write operation will be aborted. Any batches of documents that were written
135+
successfully prior to the error occurring will still exist in the database.
136+
137+
Similar to errors with reading data, the connector will strive to provide meaningful context when an error occurs to
138+
assist with debugging the cause of the error. Any errors that cannot be fixed via changes to the options passed to the
139+
connector should be reported as new issues to this GitHub repository.
124140

125141
## Tuning performance
126142

0 commit comments

Comments
 (0)