Skip to content

Commit 4d09e57

Browse files
authored
Merge pull request #49 from marklogic/feature/457-error-handling-docs
DEVEXP-457 Added docs for error handling
2 parents 2d5bff9 + 12b03f6 commit 4d09e57

File tree

2 files changed

+27
-0
lines changed

2 files changed

+27
-0
lines changed

docs/reading.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,17 @@ spark.readStream \
128128
.load()
129129
```
130130

131+
## Error handling
132+
133+
When reading data with the connector, any error that occurs - whether it is before any data is read or during a request
134+
to MarkLogic to read data - is immediately thrown to the user and the read operation ends without any result being
135+
returned. This is consistent with the design of Spark connectors, where a partition reader is allowed to throw
136+
exceptions and is not given any mechanism for reporting an error and continuing to read data. The connector will strive
137+
to provide meaningful context when an error occurs to assist with debugging the cause of the error.
138+
139+
In practice, it is expected that most errors will be a result of a misconfiguration. For example, the connection and
140+
authentication options may be incorrect, or the Optic DSL query may have a syntax error. Any errors that cannot be
141+
fixed via changes to the options passed to the connector should be reported as new issues to this GitHub repository.
131142

132143
## Tuning performance
133144

docs/writing.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,22 @@ connector and into MarkLogic. This will result 100 new JSON documents in the `st
121121
The ability to stream data into MarkLogic can make Spark an effective tool for obtaining data from a variety of data
122122
sources and writing it directly to MarkLogic.
123123

124+
## Error handling
125+
126+
The connector may throw an error during one of two phases of operation - before it begins to write data to MarkLogic,
127+
and during the writing of data to MarkLogic.
128+
129+
For the first kind of error, the error will be immediately returned to the user and no data will have been written.
130+
Such errors are often due to misconfiguration of the connector options and should be fixable.
131+
132+
For the second kind of error, the error will eventually be returned to the user, usually within seconds of it
133+
occurring. The slight delay is due to the asynchronous nature of data being written by the connector. The error will
134+
be logged by the connector and the write operation will be aborted. Any batches of documents that were written
135+
successfully prior to the error occurring will still exist in the database.
136+
137+
Similar to errors with reading data, the connector will strive to provide meaningful context when an error occurs to
138+
assist with debugging the cause of the error. Any errors that cannot be fixed via changes to the options passed to the
139+
connector should be reported as new issues to this GitHub repository.
124140

125141
## Tuning performance
126142

0 commit comments

Comments
 (0)