rename docker image tags

Author: zigzagcai

README.md

@@ -122,7 +122,7 @@ echo $LOCAL_DIR
 echo $MODEL_NAME
 ```
 **Important Notes**: 
-1. Users need to have adequate write permissions to the folders on NFS. IT policies of NFS may prvent you from mounting volumes on NFS to containers. Please refer to the [Troubleshooting](#troubleshooting) section if you ever ran into "permission denied" error when running the pipelines.
+1. Users need to have adequate write permissions to the folders on NFS. IT policies of NFS may prevent you from mounting volumes on NFS to containers. Please refer to the [Troubleshooting](#troubleshooting) section if you ever ran into "permission denied" error when running the pipelines.
 2. It is important to set up the $LOCAL_DIR variable. Make sure you have it set up.
 3. You may want to double check your http and https proxies on your machines to make sure you can download models from external internet, more specifically, from Hugging Face model hub and from PaddleOCR.
 
@@ -213,8 +213,8 @@ docker compose build
 OR 
 
 ```bash
-docker pull intel/ai-workflows:beta-doc-automation-fine-tuning
-docker pull intel/ai-workflows:beta-doc-automation-indexing
+docker pull intel/ai-workflows:doc-automation-fine-tuning
+docker pull intel/ai-workflows:doc-automation-indexing
 ```
 
 ### Run Single-Node Preprocessing Pipeline 
@@ -670,7 +670,7 @@ docker run -a stdout ${DOCKER_RUN_ENVS} \
            -v ${SAVEPATH}:/home/user/output/processed_data \
            --privileged --network host --init -it --rm --pull always \
            -w /home/user/application \
-           intel/ai-workflows:beta-doc-automation-fine-tuning \
+           intel/ai-workflows:doc-automation-fine-tuning \
            /bin/bash
 ```
 Then you will be taken inside the container and you can run the command below for pre-processing:
@@ -687,7 +687,7 @@ To read about other use cases and workflows examples, see these resources:
 
 
 ## Troubleshooting
-1. If you got "permissions denied" error to write to the output folder in the fine-tuning pipeline, it is very likely that you do not have adequate write permissions to write to the folder on NFS. You can use `chmod` to change the permissions of the output folder. If you cannot change the permissions, you can try to set up the work directory on a local disk where you have adequate write permissions.
+1. If you got "permissions denied" error to write to the output folder in the preprocessing or fine-tuning pipeline, it is very likely that you do not have adequate write permissions to write to the folder on NFS. You can use `chmod` to change the permissions of the output folder. If you cannot change the permissions, you can try to set up the work directory on a local disk where you have adequate write permissions.
 2. If you got errors about not being able to write to databases,or if either postgresql or elasticsearch container did not get started with docker-compose commands, the errors are likely due to the postgresql container user and the elasticsearch container user being different, and they have different/inadequate permissions to write to the database directory that you have set up on your machine. Change the write permissions of the `$LOCAL_DIR` with `chmod` commands so that both postgresql and elasticsearch containers can write to it.
 3. If you got error from docker daemon when running distributed indexing pipeline that "mkdir permission denied", it is due to NFS policy not allowing mounting folders on NFS to docker containers. Contact your IT to get permission or change to an NFS that allows container volume mounting.
 4. If you got out of memory (OOM) error from Ray actors, or the ray process got stuck for a very long time, try reducing the number of actors and increasing the number of CPUs per actor. As a rule of thumb, max_num_ray_actors * num_cpus_per_actor should not exceed the total number of threads of your CPU. For example, you have 48 CPU cores in your system and have hyperthreading turned on, in this case you have in total 48 * 2 = 96 threads, max_num_ray_actors * num_cpus_per_actor should not exceed 96.

chart/templates/workflowTemplate.yaml

@@ -12,15 +12,15 @@ spec:
             arguments:
               parameters:
                 - name: workflow
-                  value: beta-doc-automation-fine-tuning
+                  value: doc-automation-fine-tuning
                 - name: cmd
                   value: 'bash scripts/run_process_dataset_argo.sh'
           - name: fine-tuning
             template: fine-tuning-phase
             arguments:
               parameters:
                 - name: workflow
-                  value: beta-doc-automation-fine-tuning
+                  value: doc-automation-fine-tuning
                 - name: cmd
                   value: 'bash scripts/run_dpr_training.sh'
             dependencies:
@@ -38,7 +38,7 @@ spec:
             arguments:
               parameters:
                - name: workflow
-                 value: beta-doc-automation-indexing
+                 value: doc-automation-indexing
                - name: postgre_ip
                  value: "{{ `{{tasks.postgresql-db.ip}}` }}"
                - name: es_ip
@@ -54,7 +54,7 @@ spec:
             arguments:
              parameters:
                - name: workflow
-                 value: beta-doc-automation-indexing
+                 value: doc-automation-indexing
                - name: postgre_ip
                  value: "{{ `{{tasks.postgresql-db.ip}}` }}"
                - name: es_ip
@@ -72,7 +72,7 @@ spec:
                 - name: postgre_ip
                   value: "{{ `{{tasks.postgresql-db.ip}}` }}"
                 - name: workflow
-                  value: beta-doc-automation-indexing
+                  value: doc-automation-indexing
             dependencies:
               - performance-retrieval
           - name: ui

docker/docker-compose.yml

@@ -11,7 +11,7 @@ services:
         no_proxy: ${no_proxy}
       context: ../../document-automation
       dockerfile: docker/Dockerfile.fine-tuning
-    image: intel/ai-workflows:beta-doc-automation-fine-tuning
+    image: intel/ai-workflows:doc-automation-fine-tuning
     network_mode: "host"
     privileged: true
     command: sh -c "bash scripts/run_process_dataset.sh"
@@ -26,7 +26,7 @@ services:
     working_dir: /home/user/application
 
   fine-tuning:
-    image: intel/ai-workflows:beta-doc-automation-fine-tuning
+    image: intel/ai-workflows:doc-automation-fine-tuning
     network_mode: "host"
     privileged: true
     command: sh -c "bash scripts/run_dpr_training.sh"
@@ -76,7 +76,7 @@ services:
         no_proxy: ${no_proxy}
       dockerfile: docker/Dockerfile.indexing
       context: ../../document-automation
-    image: intel/ai-workflows:beta-doc-automation-indexing
+    image: intel/ai-workflows:doc-automation-indexing
     privileged: true
     network_mode: "host"
     command: sh -c "ray start --node-ip-address=${HEAD_IP} --head --dashboard-host='0.0.0.0' --dashboard-port=8265 --disable-usage-stats && \ python src/test_pocr.py && \ bash scripts/run_distributed_indexing.sh"
@@ -100,7 +100,7 @@ services:
     working_dir: /home/user/application
 
   performance-retrieval:
-    image: intel/ai-workflows:beta-doc-automation-indexing
+    image: intel/ai-workflows:doc-automation-indexing
     privileged: true
     depends_on:
       - indexing
@@ -119,7 +119,7 @@ services:
 
   dev:
     command: sh -c "${WORKFLOW:-bash scripts/run_dpr_training.sh}"
-    image: intel/ai-workflows:${TAG:-beta-doc-automation-fine-tuning}
+    image: intel/ai-workflows:${TAG:-doc-automation-fine-tuning}
     environment:
       - MODEL_NAME=${MODEL_NAME:-my_dpr_model}
       - http_proxy=${http_proxy}
@@ -145,7 +145,7 @@ services:
         no_proxy: ${no_proxy}
       dockerfile: docker/Dockerfile.indexing
       context: ../../document-automation
-    image: intel/ai-workflows:beta-doc-automation-indexing
+    image: intel/ai-workflows:doc-automation-indexing
     privileged: true
     network_mode: "host"
     command: sh -c "ray start --node-ip-address=${HEAD_IP} --head --dashboard-host='0.0.0.0' --dashboard-port=8265 --disable-usage-stats && \ python src/test_pocr.py && \ bash scripts/run_distributed_indexing.sh && \ bash scripts/make_retrieval_eval_csv.sh && \ bash scripts/run_retrieval_eval.sh"

scripts/run-ray-cluster.sh

@@ -209,7 +209,7 @@ if [[ $run_type = "startup_all" ]]; then
     docker run -itd -p 8265:8265 --cap-add=NET_ADMIN --network host -v ${workspace}:/home/user/application -v ${dataset}:/home/user/dataset -v ${output}:/home/user/output -v ${docvqa}:/home/user/docvqa \
             --cpuset-cpus=${ray_head_cores_range} \
             --env HEAD_IP=$HEAD_IP --env http_proxy=$http_proxy --env https_proxy=$https_proxy \
-            --shm-size=64gb --name $head_name intel/ai-workflows:beta-doc-automation-indexing /bin/bash & #intel/ai-workflows:odqa-haystack-api 
+            --shm-size=64gb --name $head_name intel/ai-workflows:doc-automation-indexing /bin/bash & #intel/ai-workflows:odqa-haystack-api 
     sleep 5
     #docker exec -d $head_name /bin/bash -c "ip link del dev eth1"
     docker exec -d $head_name /bin/bash -c "ray start --node-ip-address=${head_address} --head --dashboard-host='0.0.0.0' --dashboard-port=8265"
@@ -227,7 +227,7 @@ if [[ $run_type = "startup_all" ]]; then
         docker run -itd --cpuset-cpus=${ray_worker_cores_range} --cap-add=NET_ADMIN --network host -v ${workspace}:/home/user/application -v ${dataset}:/home/user/dataset -v ${output}:/home/user/output \
         --env HEAD_IP=$HEAD_IP --env http_proxy=$http_proxy --env https_proxy=$https_proxy \
                 --shm-size=2gb \
-                --name $worker_name  intel/ai-workflows:beta-doc-automation-indexing  /bin/bash &
+                --name $worker_name  intel/ai-workflows:doc-automation-indexing  /bin/bash &
         sleep 5
         #docker exec -d $worker_name /bin/bash -c "ip link del dev eth1"
         docker exec -d $worker_name /bin/bash -c "ray start --address=$head_address"
@@ -251,7 +251,7 @@ elif [[ $run_type = "startup_workers" ]]; then
         docker run -itd --cpuset-cpus=${ray_worker_cores_range} --cap-add=NET_ADMIN --network host -v ${workspace}:/home/user/application -v ${dataset}:/home/user/dataset -v ${output}:/home/user/output\
         --env HEAD_IP=$HEAD_IP --env http_proxy=$http_proxy --env https_proxy=$https_proxy \
                 --shm-size=2gb \
-                --name $worker_name  intel/ai-workflows:beta-doc-automation-indexing /bin/bash &
+                --name $worker_name  intel/ai-workflows:doc-automation-indexing /bin/bash &
         sleep 5
         #docker exec -d $worker_name /bin/bash -c "ip link del dev eth1"
         docker exec -d $worker_name /bin/bash -c "ray start --address=$head_address"