@@ -122,7 +122,7 @@ This will produce a single jar file for the connector in the `./build/libs` dire
122
122
123
123
You can then launch PySpark with the connector available via:
124
124
125
- pyspark --jars build/libs/marklogic-spark-connector-2.3 -SNAPSHOT.jar
125
+ pyspark --jars build/libs/marklogic-spark-connector-2.4 -SNAPSHOT.jar
126
126
127
127
The below command is an example of loading data from the test application deployed via the instructions at the top of
128
128
this page.
@@ -198,7 +198,7 @@ The Spark master GUI is at <http://localhost:8080>. You can use this to view det
198
198
199
199
Now that you have a Spark cluster running, you just need to tell PySpark to connect to it:
200
200
201
- pyspark --master spark://NYWHYC3G0W:7077 --jars build/libs/marklogic-spark-connector-2.3 -SNAPSHOT.jar
201
+ pyspark --master spark://NYWHYC3G0W:7077 --jars build/libs/marklogic-spark-connector-2.4 -SNAPSHOT.jar
202
202
203
203
You can then run the same commands as shown in the PySpark section above. The Spark master GUI will allow you to
204
204
examine details of each of the commands that you run.
@@ -217,12 +217,12 @@ You will need the connector jar available, so run `./gradlew clean shadowJar` if
217
217
You can then run a test Python program in this repository via the following (again, change the master address as
218
218
needed); note that you run this outside of PySpark, and ` spark-submit ` is available after having installed PySpark:
219
219
220
- spark-submit --master spark://NYWHYC3G0W:7077 --jars build/libs/marklogic-spark-connector-2.3 -SNAPSHOT.jar src/test/python/test_program.py
220
+ spark-submit --master spark://NYWHYC3G0W:7077 --jars build/libs/marklogic-spark-connector-2.4 -SNAPSHOT.jar src/test/python/test_program.py
221
221
222
222
You can also test a Java program. To do so, first move the ` com.marklogic.spark.TestProgram ` class from ` src/test/java `
223
223
to ` src/main/java ` . Then run ` ./gradlew clean shadowJar ` to rebuild the connector jar. Then run the following:
224
224
225
- spark-submit --master spark://NYWHYC3G0W:7077 --class com.marklogic.spark.TestProgram build/libs/marklogic-spark-connector-2.3 -SNAPSHOT.jar
225
+ spark-submit --master spark://NYWHYC3G0W:7077 --class com.marklogic.spark.TestProgram build/libs/marklogic-spark-connector-2.4 -SNAPSHOT.jar
226
226
227
227
Be sure to move ` TestProgram ` back to ` src/test/java ` when you are done.
228
228
0 commit comments