Merge pull request #106 from marklogic/feature/625-docs

DEVEXP-625 Reworked docs to use docker-compose

Author: rjrudin

docs/configuration.md

@@ -56,7 +56,7 @@ Using this convenience can provide a much more succinct set of options - for exa
 
 ```
 df = spark.read.format("com.marklogic.spark")\
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020")\
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003")\
     .option("spark.marklogic.read.opticQuery", "op.fromView('example', 'employee')")\
     .load()
 ```
@@ -155,4 +155,4 @@ The following options control how rows can be processed with custom code in Mark
 | spark.marklogic.write.xquery | XQuery code to execute. |
 | spark.marklogic.write.externalVariableName | Name of the external variable in custom code that is populated with row values; defaults to `URI`. |
 | spark.marklogic.write.externalVariableDelimiter | Delimiter used when multiple row values are sent in a single call; defaults to a comma. |
-| spark.marklogic.write.vars. | Prefix for user-defined variables to be sent to the custom code. |
+| spark.marklogic.write.vars. | Prefix for user-defined variables to be sent to the custom code. |

docs/getting-started/pyspark.md

@@ -52,7 +52,7 @@ paste the following Python statement into PySpark, adjusting the host and passwo
 ```
 df = spark.read.format("com.marklogic.spark") \
     .option("spark.marklogic.client.host", "localhost") \
-    .option("spark.marklogic.client.port", "8020") \
+    .option("spark.marklogic.client.port", "8003") \
     .option("spark.marklogic.client.username", "spark-example-user") \
     .option("spark.marklogic.client.password", "password") \
     .option("spark.marklogic.read.opticQuery", "op.fromView('example', 'employee')") \
@@ -64,7 +64,7 @@ client options in one option:
 
 ```
 df = spark.read.format("com.marklogic.spark") \
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020") \
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003") \
     .option("spark.marklogic.read.opticQuery", "op.fromView('example', 'employee')") \
     .load()
 ```
@@ -91,7 +91,7 @@ paste the following into PySpark, adjusting the host and password values as need
 
 ```
 df.write.format("com.marklogic.spark") \
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020") \
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003") \
     .option("spark.marklogic.write.collections", "write-test") \
     .option("spark.marklogic.write.permissions", "rest-reader,read,rest-writer,update") \
     .option("spark.marklogic.write.uriPrefix", "/write/") \

docs/getting-started/setup.md

@@ -25,32 +25,32 @@ projects data from documents in MarkLogic into rows.
 
 To facilitate trying out the connector, perform the following steps to deploy an example application to your
 MarkLogic server that includes a
-[TDE view](https://docs.marklogic.com/guide/app-dev/TDE) and some documents that conform to that view.
+[TDE view](https://docs.marklogic.com/guide/app-dev/TDE) and some documents that conform to that view. These instructions depend on 
+[using Docker](https://docs.docker.com/get-docker/) to install and initialize an instance of MarkLogic. If you already
+have an instance of MarkLogic running, you can skip step 4 below, but ensure that the `gradle.properties` file in the
+extracted directory contains valid connection properties for your instance of MarkLogic.
 
 1. From [this repository's Releases page](https://github.com/marklogic/marklogic-spark-connector/releases), select 
-   the latest release and download the `marklogic-spark-getting-started-2.0.0.zip` file.
+   the latest release and download the `marklogic-spark-getting-started-2.1.0.zip` file.
 2. Extract the contents of the downloaded zip file. 
 3. Open a terminal window and go to the directory created by extracting the zip file; the directory should have a 
-   name of "marklogic-spark-getting-started-2.0.0".
-4. Create a file named `gradle-local.properties` and add `mlPassword=changeme`, changing the text "changeme" to the
-   password of your MarkLogic `admin` user.
-5. Open the `gradle.properties` file and verify that the value of the `mlPort` property is an available port on the
-   machine running your MarkLogic server; the default port is 8020.
-6. Ensure that the `./gradlew` file is executable; depending on your operating system, you may need to run
+   name of "marklogic-spark-getting-started-2.1.0".
+4. Run `docker-compose up -d` to start an instance of MarkLogic
+5. Ensure that the `./gradlew` file is executable; depending on your operating system, you may need to run
    `chmod 755 gradlew` to make the file executable.
-7. Run `./gradlew -i mlDeploy` to deploy the example application.
+6. Run `./gradlew -i mlDeploy` to deploy the example application.
 
 After the deployment finishes, your MarkLogic server will now have the following:
 
-- An app server named `spark-example` listening on port 8020 (or the port you chose if you modified the `mlPort`
-  property).
-- A database named `spark-example-content` that contains 1000 JSON documents in the `employee` collection.
+- An app server named `spark-example` listening on port 8003.
+- A database named `spark-example-content` that contains 1000 JSON documents in a collection named `employee`.
 - A TDE with a schema name of `example` and a view name of `employee`.
-- A user named `spark-example-user` that can be used with the Spark connector and [MarkLogic's qconsole tool](https://docs.marklogic.com/guide/qconsole/intro).
+- A user named `spark-example-user` with a password of `password` that can be used with the Spark connector and [MarkLogic's qconsole tool](https://docs.marklogic.com/guide/qconsole/intro).
+
+To verify that your application was deployed correctly, access your MarkLogic server's qconsole tool 
+via <http://localhost:8000/qconsole> . You can authenticate as the `spark-example-user` user that was created above, 
+as it's generally preferable to test as a non-admin user. 
 
-To verify that your application was deployed correctly, access your MarkLogic server's qconsole tool - for example,
-if your MarkLogic server is deployed locally, you will go to <http://localhost:8000/qconsole> . You can authenticate as 
-the `spark-example-user` user that was created above, as it's generally preferable to test as a non-admin user. 
 After authenticating, perform the following steps:
 
 1. In the "Database" dropdown, select `spark-example-content`.

docs/reading.md

@@ -22,7 +22,7 @@ see the next section for more information), and zero or more other options:
 
 ```
 df = spark.read.format("com.marklogic.spark") \
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020") \
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003") \
     .option("spark.marklogic.read.opticQuery", "op.fromView('example', 'employee')") \
     .load()
 ```
@@ -42,7 +42,7 @@ the Optic query):
 query = "op.fromView('example', 'employee').where(cts.wordQuery('Drive'))"
 
 df = spark.read.format("com.marklogic.spark") \
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020") \
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003") \
     .option("spark.marklogic.read.opticQuery", query) \
     .load()
 ```
@@ -85,7 +85,7 @@ be provided within PySpark; this assumes that you have deployed the application
 from pyspark.sql.types import StructField, StructType, StringType
 df = spark.read.format("com.marklogic.spark") \
     .schema(StructType([StructField("example.employee.GivenName", StringType()), StructField("example.employee.Surname", StringType())])) \
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020") \
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003") \
     .option("spark.marklogic.read.opticQuery", "op.fromView('example', 'employee')") \
     .load()
 ```
@@ -105,7 +105,7 @@ op.fromView('example', 'employee', '', joinCol) \
   .select('doc')"
 
 df = spark.read.format("com.marklogic.spark") \
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020") \
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003") \
     .option("spark.marklogic.read.opticQuery", query) \
     .load()
 ```
@@ -128,7 +128,7 @@ deployed in the [Getting Started with PySpark guide](getting-started/pyspark.md)
 ```
 stream = spark.readStream \
     .format("com.marklogic.spark") \
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020") \
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003") \
     .option("spark.marklogic.read.numPartitions", 2) \
     .option("spark.marklogic.read.opticQuery", "op.fromView('example', 'employee')") \
     .load() \
@@ -173,7 +173,7 @@ rows being returned to Spark and far less work having to be done by Spark:
 
 ```
 spark.read.format("com.marklogic.spark") \
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020") \
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003") \
     .option("spark.marklogic.read.opticQuery", "op.fromView('example', 'employee', '')") \
     .load() \
     .filter("HiredDate < '2020-01-01'") \
@@ -286,7 +286,7 @@ configuring the `spark.marklogic.read.javascript` option:
 
 ```
 df = spark.read.format("com.marklogic.spark") \
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020") \
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003") \
     .option("spark.marklogic.read.javascript", "cts.uris(null, null, cts.collectionQuery('employee'))") \
     .load()
 ```
@@ -296,7 +296,7 @@ Or code can be [written in XQuery](https://docs.marklogic.com/guide/getting-star
 
 ```
 df = spark.read.format("com.marklogic.spark") \
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020") \
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003") \
     .option("spark.marklogic.read.xquery", "cts:uris((), (), cts:collection-query('employee'))") \
     .load()
 ```
@@ -306,7 +306,7 @@ You can also invoke a JavaScript or XQuery module in your application's modules
 
 ```
 df = spark.read.format("com.marklogic.spark") \
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020") \
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003") \
     .option("spark.marklogic.read.invoke", "/read.sjs") \
     .load()
 ```
@@ -326,7 +326,7 @@ JSON objects with columns that conform to the given schema:
 ```
 from pyspark.sql.types import StructField, StructType, IntegerType, StringType
 df = spark.read.format("com.marklogic.spark") \
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020") \
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003") \
     .option("spark.marklogic.read.invoke", "/read-custom-schema.sjs") \
     .schema(StructType([StructField("id", IntegerType()), StructField("name", StringType())])) \
     .load()
@@ -343,14 +343,14 @@ The following demonstrates two custom external variables being configured and us
 
 ```
 df = spark.read.format("com.marklogic.spark") \
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020") \
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003") \
     .option("spark.marklogic.read.vars.var1", "Engineering") \
     .option("spark.marklogic.read.vars.var2", "Marketing") \
     .option("spark.marklogic.read.javascript", "var var1, var2; cts.uris(null, null, cts.wordQuery([var1, var2]))") \
     .load()
 ```
 
-### Streaming with custom code
+### Streaming support
 
 Spark's support for [streaming reads](https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html) 
 from MarkLogic can be useful when your custom code for reading data may take a long time to execute. Or, based on the
@@ -384,7 +384,7 @@ sent to the writer, which in this example are then printed to the console:
 ```
 stream = spark.readStream \
     .format("com.marklogic.spark") \
-    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8020") \
+    .option("spark.marklogic.client.uri", "spark-example-user:password@localhost:8003") \
     .option("spark.marklogic.read.batchIds.javascript", "xdmp.databaseForests(xdmp.database('spark-example-content'))") \
     .option("spark.marklogic.read.javascript", "cts.uris(null, null, cts.collectionQuery('employee'), null, [BATCH_ID]);") \
     .load() \