how to select gpu

[aapst12]

In spark NLP, how do I select the gpu I want to target? I tried
tokenizer.setGpu(0) , but that did not work.

In spark NLP, how do I select the cuda device I want to target (in both scala and python)? For example, if a node has 2 gpus, how do I select gpu 0 or gpu 1?

What I have observed so far, working with the NerDL benchmark/example, with scala:
The spark session will find both GPUs, and the training will be done on gpu0; the training is successful, and the engagement of the gpu can be seen with the utilization of the gpu 0 going up to 25%.
I tried “tokenizer.setGpu(1) “, but that did not work (the setGpu attribute does not exist).
I tried setting the environment variable with “export CUDA_VISIBLE_DEVICES=1” before running the scala commands, but the training still ran on GPU 0 (perhaps the default spark session configuration flips it back?).
spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-gpu_2.12:5.0.1 --conf CUDA_VISIBLE_DEVICES=1
scala> System.setProperty("CUDA_VISIBLE_DEVICES", "1")
spark-shell --conf spark.executorEnv.CUDA_VISIBLE_DEVICES=1
…neither of the above commands worked; the training was still carried out on gpu 0.

[maziyarpanahi]

Hi,

a few points:

• Just to be sure, there is no multi-gpu support in Spark NLP. Neither for training nor for inference.
• Allowing a config to set which GPU device should be used in TensorFlow and ONNX inside Spark NLP is under development and hopefully be released within a month. By default it is always gpu:0
• In the meantime, CUDA_VISIBLE_DEVICES=1 should work (tried in Python for PySpark). However, is Spark via Scala, the ENV you set this variable and the ENV the Scala/JVM is running are different.
• Also, there is no way you can pass it via --conf. If really required, this ENV must be set in the same env that is launching the Spark application. (hopefully this works on Scala, if not we are working on this config to allow this easily be set)