From e9ba40bc244078c146b08c9b66740f0f2a2bb19d Mon Sep 17 00:00:00 2001 From: AlexZakIntel Date: Mon, 28 Feb 2022 12:51:49 -0500 Subject: [PATCH 01/13] New Jupyter Notebooks ipynb Semantic Segmentation Training files - work in progress --- .../train_ss_model_using_pytorch.ipynb | 376 ++++++++++++++++++ .../train_ss_model_using_tensorflow.ipynb | 352 ++++++++++++++++ 2 files changed, 728 insertions(+) create mode 100644 docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb create mode 100644 docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb diff --git a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb new file mode 100644 index 000000000..98fac6bfb --- /dev/null +++ b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb @@ -0,0 +1,376 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "f6567deb", + "metadata": {}, + "source": [ + "# Training a Semantic Segmentation Model Using PyTorch" + ] + }, + { + "cell_type": "markdown", + "id": "33cfdf83", + "metadata": {}, + "source": [ + "In this tutorial, we will learn how to train a semantic segmentation model using PyTorch in a Jupyter Notebook. We assume that you are familiar with Jupyter Notebook and have created a folder notebooks in a folder that is relative to ml3d.\n", + "\n", + "Before you begin, ensure that you have *PyTorch* installed. To install a compatible version of PyTorch, use the requirement file:\n", + "\n", + "```sh\n", + "pip install -r requirements-torch-cuda.txt\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "cfa831d8", + "metadata": {}, + "source": [ + "At a high-level, we will:\n", + "\n", + "- Read a dataset and get a training split. For this example, we will use `SemanticKITTI` dataset.\n", + "- Run a pre-trained model. For this example, we will use the `RandLANet` model.\n", + "- Train a model. We will train a model using the `SemanticKITTI` dataset and `RandLANet` model.\n", + "- Run an inference and run a test. We will run an inference using the 'training' split that use a pointcloud and display a result. However, a test is run on a pre-defined test set rather than a pass pointcloud.\n" + ] + }, + { + "cell_type": "markdown", + "id": "95993f3e", + "metadata": {}, + "source": [ + "## Reading a dataset" + ] + }, + { + "cell_type": "markdown", + "id": "c210491d", + "metadata": {}, + "source": [ + "You can use any dataset available in the `ml3d.datasets` dataset namespace. For this example, we will use the `SemanticKITTI` dataset and visualize it. You can use any of the other dataset to load data. However, you must understand that the parameters may vary for each dataset.\n", + "\n", + "We will read the dataset by specifying its path and then get all splits." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "54e07a36", + "metadata": {}, + "outputs": [], + "source": [ + "#Training Semantic Segmentation Model using PyTorch\n", + "\n", + "#import torch\n", + "import open3d.ml.torch as ml3d\n", + "\n", + "#Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", + "\n", + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n", + "#Split the dataset for 'training'. You can get the other splits by passing 'validation' or 'test'\n", + "train_split = dataset.get_split('training')\n", + "\n", + "#support of Open3d-ML visualizer in Jupyter Notebooks is in progress\n", + "#view the frames using the visualizer\n", + "#vis = ml3d.vis.Visualizer()\n", + "#vis.visualize_dataset(dataset, 'training',indices=range(len(train_split)))" + ] + }, + { + "cell_type": "markdown", + "id": "0b5eccfd", + "metadata": {}, + "source": [ + "Now that you have visualized the dataset for training, let us train the model." + ] + }, + { + "cell_type": "markdown", + "id": "43a70525", + "metadata": {}, + "source": [ + "## Training a model" + ] + }, + { + "cell_type": "markdown", + "id": "d33ddf74", + "metadata": {}, + "source": [ + "Before you train a model, you must decide which model you want to use. For this example, we will use the `RandLANet` model. To use models, you must import the model from `open3d.ml.torch.models`.\n", + "\n", + "After you load a dataset, you can initialize any model and then train the model. The following example shows how you can train a model:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "b15cace7", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Jupyter environment detected. Enabling Open3D WebVisualizer.\n", + "[Open3D INFO] WebRTC GUI backend enabled.\n", + "[Open3D INFO] WebRTCWindowSystem: HTTP handshake server disabled.\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\n", + " Using the Open3D PyTorch ops with CUDA 11 may have stability issues!\n", + "\n", + " We recommend to compile PyTorch from source with compile flags\n", + " '-Xcompiler -fno-gnu-unique'\n", + "\n", + " or use the PyTorch wheels at\n", + " https://github.com/isl-org/open3d_downloads/releases/tag/torch1.8.2\n", + "\n", + "\n", + " Ignore this message if PyTorch has been compiled with the aforementioned\n", + " flags.\n", + "\n", + " See https://github.com/isl-org/Open3D/issues/3324 and\n", + " https://github.com/pytorch/pytorch/issues/52663 for more information on this\n", + " problem.\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2022-02-28 12:25:57.683214: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", + "2022-02-28 12:25:57.683243: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n", + "INFO - 2022-02-28 12:25:59,912 - semantic_segmentation - DEVICE : cpu\n", + "INFO - 2022-02-28 12:25:59,913 - semantic_segmentation - Logging in file : ./logs/RandLANet_SemanticKITTI_torch/log_train_2022-02-28_12:25:59.txt\n", + "INFO - 2022-02-28 12:25:59,914 - semantickitti - Found 1 pointclouds for train\n", + "INFO - 2022-02-28 12:25:59,915 - semantickitti - Found 1 pointclouds for validation\n", + "INFO - 2022-02-28 12:25:59,917 - semantic_segmentation - ckpt_path not given. Restore from the latest ckpt\n", + "INFO - 2022-02-28 12:25:59,917 - semantic_segmentation - Loading checkpoint ./logs/RandLANet_SemanticKITTI_torch/checkpoint/ckpt_00000.pth\n", + "INFO - 2022-02-28 12:25:59,979 - semantic_segmentation - Loading checkpoint optimizer_state_dict\n", + "INFO - 2022-02-28 12:26:00,004 - semantic_segmentation - Loading checkpoint scheduler_state_dict\n", + "INFO - 2022-02-28 12:26:00,010 - semantic_segmentation - Writing summary in train_log/00003_RandLANet_SemanticKITTI_torch.\n", + "INFO - 2022-02-28 12:26:00,011 - semantic_segmentation - Started training\n", + "INFO - 2022-02-28 12:26:00,013 - semantic_segmentation - === EPOCH 0/3 ===\n", + "training: 0%| | 0/1 [00:00\u001b[0;34m\u001b[0m\n\u001b[1;32m 7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[0;31m#Get pipeline, model, and dataset.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 9\u001b[0;31m \u001b[0mPipeline\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mget_module\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"pipeline\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"SemanticSegmentation\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"torch\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 10\u001b[0m \u001b[0mModel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mget_module\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"model\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"RandLANet\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"torch\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 11\u001b[0m \u001b[0mDataset\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mget_module\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"dataset\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"SemanticKITTI\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;31mNameError\u001b[0m: name 'get_module' is not defined" + ] + } + ], + "source": [ + "#Training Semantic Segmentation Model using PyTorch\n", + "\n", + "#Import torch and the model to use for training\n", + "import open3d.ml.torch as ml3d\n", + "from open3d.ml.torch.models import RandLANet\n", + "from open3d.ml.torch.pipelines import SemanticSegmentation\n", + "\n", + "#Get pipeline, model, and dataset.\n", + "Pipeline = get_module(\"pipeline\", \"SemanticSegmentation\", \"torch\")\n", + "Model = get_module(\"model\", \"RandLANet\", \"torch\")\n", + "Dataset = get_module(\"dataset\", \"SemanticKITTI\")\n", + "\n", + "#Create a checkpoint\n", + "RandLANet = Model(ckpt_path=args.path_ckpt_randlanet)\n", + "SemanticKITTI = Dataset(args.path_semantickitti, use_cache=False)\n", + "pipeline = Pipeline(model=RandLANet, dataset=SemanticKITTI)\n", + "\n", + "#Get data from the SemanticKITTI dataset using the \"train\" split\n", + "train_split = SemanticKITTI.get_split(\"train\")\n", + "data = train_split.get_data(0)\n", + "\n", + "#Run the test\n", + "pipeline.run_test(data)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0f03fae8", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb b/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb new file mode 100644 index 000000000..724f63f6f --- /dev/null +++ b/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb @@ -0,0 +1,352 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "c8d52012", + "metadata": {}, + "source": [ + "# Training a Semantic Segmentation Model Using TensorFlow" + ] + }, + { + "cell_type": "markdown", + "id": "04ed6952", + "metadata": {}, + "source": [ + "In this tutorial, we will learn how to train a semantic segmentation model using `TensorFlow` in a Jupyter Notebook. We assume that you are familiar with Jupyter Notebook and have created a folder *notebooks* in a folder that is relative to *ml3d*.\n", + "\n", + "Before you begin, ensure that you have `TensorFlow` installed. To install a compatible version of `TensorFlow`, use the requirement file:\n", + "\n", + "```sh\n", + "pip install -r requirements-tensorflow.txt\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "ec60b6a0", + "metadata": {}, + "source": [ + "At a high-level, we will:\n", + "\n", + "- Read a dataset and get a training split. For this example, we will use `SemanticKITTI` dataset.\n", + "- Run a pre-trained model. For this example, we will use the `RandLANet` model.\n", + "- Train a model. We will train a model using the `SemanticKITTI` dataset and `RandLANet` model.\n", + "- Run an inference and run a test. We will run an inference using the 'training' split that use a pointcloud and display a result. However, a test is run on a pre-defined test set rather than a pass pointcloud.\n" + ] + }, + { + "cell_type": "markdown", + "id": "8055b308", + "metadata": {}, + "source": [ + "## Reading a dataset" + ] + }, + { + "cell_type": "markdown", + "id": "5e33bf9d", + "metadata": {}, + "source": [ + "You can use any dataset available in the `ml3d.datasets` dataset namespace. For this example, we will use the `SemanticKITTI` dataset and visualize it. You can use any of the other dataset to load data. However, you must understand that the parameters may vary for each dataset.\n", + "\n", + "We will read the dataset by specifying its path and then get all splits." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "b4e344db", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Jupyter environment detected. Enabling Open3D WebVisualizer.\n", + "[Open3D INFO] WebRTC GUI backend enabled.\n", + "[Open3D INFO] WebRTCWindowSystem: HTTP handshake server disabled.\n" + ] + }, + { + "ename": "ModuleNotFoundError", + "evalue": "No module named 'tensorflow'", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)", + "\u001b[0;32m/tmp/ipykernel_9847/1245153815.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;31m#import tensorflow\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mml3d\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0;31m#Read a dataset by specifying the path. We are also providing the cache directory and training split.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m~/miniconda3/envs/open3d/lib/python3.8/site-packages/open3d/ml/tf/__init__.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 27\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 28\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mos\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_os\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 29\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mtensorflow\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m__version__\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_tf_version\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 30\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m_build_config\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 31\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'tensorflow'" + ] + } + ], + "source": [ + "#Training Semantic Segmentation Model using TensorFlow\n", + "\n", + "#import tensorflow\n", + "import open3d.ml.tf as ml3d\n", + "\n", + "#Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='../datasets/', cache_dir='./logs/cache',training_split=['00', '01', '02', '03', '04', '05', '06', '07', '09', '10'])\n", + "#Split the dataset for 'training'. You can get the other splits by passing 'validation' or 'test'\n", + "train_split = dataset.get_split('training')\n", + "\n", + "#view the first 1000 frames using the visualizer\n", + "MyVis = ml3d.vis.Visualizer()\n", + "vis.visualize_dataset(dataset, 'training',indices=range(100))" + ] + }, + { + "cell_type": "markdown", + "id": "1138db9e", + "metadata": {}, + "source": [ + "Now that you have visualized the dataset for training, let us train the model." + ] + }, + { + "cell_type": "markdown", + "id": "258186a2", + "metadata": {}, + "source": [ + "## Training a model" + ] + }, + { + "cell_type": "markdown", + "id": "9653bd32", + "metadata": {}, + "source": [ + "`TensorFlow` maps nearly all of GPU memory by default. This may result in out_of_memory error if some of the ops allocate memory independent to tensorflow. You may want to limit memory usage as and when needed by the process. Use following code right after importing tensorflow:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "c220cfb6", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2022-02-24 12:27:34.379346: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", + "2022-02-24 12:27:34.379376: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n", + "2022-02-24 12:27:35.764206: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory\n", + "2022-02-24 12:27:35.764236: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)\n", + "2022-02-24 12:27:35.764259: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (intel-VirtualBox): /proc/driver/nvidia/version does not exist\n" + ] + } + ], + "source": [ + "import tensorflow as tf\n", + "gpus = tf.config.experimental.list_physical_devices('GPU')\n", + "if gpus:\n", + " try:\n", + " for gpu in gpus:\n", + " tf.config.experimental.set_memory_growth(gpu, True)\n", + " except RuntimeError as e:\n", + " print(e)" + ] + }, + { + "cell_type": "markdown", + "id": "12d8271a", + "metadata": {}, + "source": [ + "Refer to [this link](https://www.tensorflow.org/guide/gpu#limiting_gpu_memory_growth) for more details.\n", + "\n", + "Before you train a model, you must decide which model you want to use. For this example, we will use `RandLANet` model. To use models, you must import the model from `open3d.ml.tf.models`.\n", + "\n", + "After you load a dataset, you can initialize any model and then train the model. The following example shows how you can train a model:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "caeea522", + "metadata": {}, + "outputs": [ + { + "ename": "ModuleNotFoundError", + "evalue": "No module named 'tensorflow'", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)", + "\u001b[0;32m/tmp/ipykernel_9847/122240413.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;31m#Import tensorflow and the model to use for training\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mml3d\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 5\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmodels\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mRandLANet\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpipelines\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mSemanticSegmentation\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m~/miniconda3/envs/open3d/lib/python3.8/site-packages/open3d/ml/tf/__init__.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 27\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 28\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mos\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_os\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 29\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mtensorflow\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m__version__\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_tf_version\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 30\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m_build_config\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 31\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'tensorflow'" + ] + } + ], + "source": [ + "#Training Semantic Segmentation Model using TensorFlow\n", + "\n", + "#Import tensorflow and the model to use for training\n", + "import open3d.ml.tf as ml3d\n", + "from open3d.ml.tf.models import RandLANet\n", + "from open3d.ml.tf.pipelines import SemanticSegmentation\n", + "\n", + "#Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='../datasets/', cache_dir='./logs/cache',training_split=['00', '01', '02', '03', '04', '05', '06', '07', '09', '10'])\n", + "#Initialize the RandLANet model with three layers.\n", + "model = RandLANet(dim_input=3)\n", + "pipeline = SemanticSegmentation(model=model, dataset=dataset, max_epoch=100)\n", + "#Run the training\n", + "pipeline.run_train()" + ] + }, + { + "cell_type": "markdown", + "id": "5adc93ed", + "metadata": {}, + "source": [ + "## Running an Inference" + ] + }, + { + "cell_type": "markdown", + "id": "515a986a", + "metadata": {}, + "source": [ + "An inference processes point cloud and displays the results based on the trained model. For this example, we will use a trained `RandLANet` model.\n", + "\n", + "This example gets the pipeline, model, and dataset based on our previous training example. It runs the inference based the \"train\" split and prints the results." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "3ec86a88", + "metadata": {}, + "outputs": [ + { + "ename": "ModuleNotFoundError", + "evalue": "No module named 'tensorflow'", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)", + "\u001b[0;32m/tmp/ipykernel_9847/822280479.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;31m#Import tensorflow and the model to use for training\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mml3d\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 5\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmodels\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mRandLANet\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpipelines\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mSemanticSegmentation\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m~/miniconda3/envs/open3d/lib/python3.8/site-packages/open3d/ml/tf/__init__.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 27\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 28\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mos\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_os\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 29\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mtensorflow\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m__version__\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_tf_version\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 30\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m_build_config\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 31\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'tensorflow'" + ] + } + ], + "source": [ + "#Training Semantic Segmentation Model using TensorFlow\n", + "\n", + "#Import tensorflow and the model to use for training\n", + "import open3d.ml.tf as ml3d\n", + "from open3d.ml.tf.models import RandLANet\n", + "from open3d.ml.tf.pipelines import SemanticSegmentation\n", + "\n", + "#Get pipeline, model, and dataset.\n", + "Pipeline = get_module(\"pipeline\", \"SemanticSegmentation\", \"tf\")\n", + "Model = get_module(\"model\", \"RandLANet\", \"tf\")\n", + "Dataset = get_module(\"dataset\", \"SemanticKITTI\")\n", + "\n", + "#Create a checkpoint\n", + "RandLANet = Model(ckpt_path=args.path_ckpt_randlanet)\n", + "SemanticKITTI = Dataset(args.path_semantickitti, use_cache=False)\n", + "pipeline = Pipeline(model=RandLANet, dataset=SemanticKITTI)\n", + "\n", + "#Get data from the SemanticKITTI dataset using the \"train\" split\n", + "train_split = SemanticKITTI.get_split(\"train\")\n", + "data = train_split.get_data(0)\n", + "\n", + "#Run the inference\n", + "results = pipeline.run_inference(data)\n", + "\n", + "#Print the results\n", + "print(results)" + ] + }, + { + "cell_type": "markdown", + "id": "7f7d59ce", + "metadata": {}, + "source": [ + "## Running a test" + ] + }, + { + "cell_type": "markdown", + "id": "9648d5da", + "metadata": {}, + "source": [ + "Running a test is very similar to running an inference on Jupyter.\n", + "\n", + "This example gets the pipeline, model, and dataset based on our previous training example. It runs the test based the \"train\" split." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "7a81f7f2", + "metadata": {}, + "outputs": [ + { + "ename": "ModuleNotFoundError", + "evalue": "No module named 'tensorflow'", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)", + "\u001b[0;32m/tmp/ipykernel_9847/2840162364.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;31m#Import tensorflow and the model to use for training\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mml3d\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 5\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmodels\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mRandLANet\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpipelines\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mSemanticSegmentation\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;32m~/miniconda3/envs/open3d/lib/python3.8/site-packages/open3d/ml/tf/__init__.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 27\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 28\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mos\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_os\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 29\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mtensorflow\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m__version__\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_tf_version\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 30\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m_build_config\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 31\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'tensorflow'" + ] + } + ], + "source": [ + "#Training Semantic Segmentation Model using TensorFlow\n", + "\n", + "#Import tensorflow and the model to use for training\n", + "import open3d.ml.tf as ml3d\n", + "from open3d.ml.tf.models import RandLANet\n", + "from open3d.ml.tf.pipelines import SemanticSegmentation\n", + "\n", + "#Get pipeline, model, and dataset.\n", + "Pipeline = get_module(\"pipeline\", \"SemanticSegmentation\", \"tf\")\n", + "Model = get_module(\"model\", \"RandLANet\", \"tf\")\n", + "Dataset = get_module(\"dataset\", \"SemanticKITTI\")\n", + "\n", + "#Create a checkpoint\n", + "RandLANet = Model(ckpt_path=args.path_ckpt_randlanet)\n", + "SemanticKITTI = Dataset(args.path_semantickitti, use_cache=False)\n", + "pipeline = Pipeline(model=RandLANet, dataset=SemanticKITTI)\n", + "\n", + "#Get data from the SemanticKITTI dataset using the \"train\" split\n", + "train_split = SemanticKITTI.get_split(\"train\")\n", + "data = train_split.get_data(0)\n", + "\n", + "#Run the test\n", + "pipeline.run_test(data)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} From de5f733e10198624303e083bd683d17062754c4c Mon Sep 17 00:00:00 2001 From: Sanskar Agrawal Date: Thu, 10 Mar 2022 00:28:25 +0530 Subject: [PATCH 02/13] add new files --- .../Inference_on_a_custom_dataset.ipynb | 51 +++ .../notebook/reading_a_config_file.ipynb | 328 ++++++++++++++++++ .../train_ss_model_using_pytorch.ipynb | 21 +- 3 files changed, 381 insertions(+), 19 deletions(-) create mode 100644 docs/tutorial/notebook/Inference_on_a_custom_dataset.ipynb create mode 100644 docs/tutorial/notebook/reading_a_config_file.ipynb diff --git a/docs/tutorial/notebook/Inference_on_a_custom_dataset.ipynb b/docs/tutorial/notebook/Inference_on_a_custom_dataset.ipynb new file mode 100644 index 000000000..d1e4cc33e --- /dev/null +++ b/docs/tutorial/notebook/Inference_on_a_custom_dataset.ipynb @@ -0,0 +1,51 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "#Import torch and the model to use for training\n", + "import open3d.ml.torch as ml3d\n", + "from open3d.ml.torch.models import RandLANet\n", + "from open3d.ml.torch.pipelines import SemanticSegmentation\n", + "\n", + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n", + "model = RandLANet(in_channels=3)\n", + "pipeline = SemanticSegmentation(model=model, dataset=dataset, max_epoch=3, optimizer={'lr': 0.001}, num_workers=0)\n", + "\n", + "#Get data from the SemanticKITTI dataset using the \"train\" split\n", + "train_split = dataset.get_split(\"test\")\n", + "data = train_split.get_data(0)\n", + "\n", + "#Run the inference\n", + "results = pipeline.run_inference(data)\n", + "\n", + "#Print the results\n", + "print(results)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.7" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/docs/tutorial/notebook/reading_a_config_file.ipynb b/docs/tutorial/notebook/reading_a_config_file.ipynb new file mode 100644 index 000000000..3011b4336 --- /dev/null +++ b/docs/tutorial/notebook/reading_a_config_file.ipynb @@ -0,0 +1,328 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [], + "source": [ + "from open3d.ml import utils\n", + "import open3d.ml.torch as ml3d" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [], + "source": [ + "cfg_file = \"../../../ml3d/configs/randlanet_semantickitti.yml\"\n", + "cfg = utils.Config.load_from_file(cfg_file)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "cfg" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'name': 'SemanticKITTI',\n", + " 'dataset_path': None,\n", + " 'cache_dir': './logs/cache',\n", + " 'class_weights': [55437630,\n", + " 320797,\n", + " 541736,\n", + " 2578735,\n", + " 3274484,\n", + " 552662,\n", + " 184064,\n", + " 78858,\n", + " 240942562,\n", + " 17294618,\n", + " 170599734,\n", + " 6369672,\n", + " 230413074,\n", + " 101130274,\n", + " 476491114,\n", + " 9833174,\n", + " 129609852,\n", + " 4506626,\n", + " 1168181],\n", + " 'test_result_folder': './test',\n", + " 'test_split': ['11',\n", + " '12',\n", + " '13',\n", + " '14',\n", + " '15',\n", + " '16',\n", + " '17',\n", + " '18',\n", + " '19',\n", + " '20',\n", + " '21'],\n", + " 'training_split': ['00',\n", + " '01',\n", + " '02',\n", + " '03',\n", + " '04',\n", + " '05',\n", + " '06',\n", + " '07',\n", + " '09',\n", + " '10'],\n", + " 'all_split': ['00',\n", + " '01',\n", + " '02',\n", + " '03',\n", + " '04',\n", + " '05',\n", + " '06',\n", + " '07',\n", + " '09',\n", + " '08',\n", + " '10',\n", + " '11',\n", + " '12',\n", + " '13',\n", + " '14',\n", + " '15',\n", + " '16',\n", + " '17',\n", + " '18',\n", + " '19',\n", + " '20',\n", + " '21'],\n", + " 'validation_split': ['08'],\n", + " 'use_cache': True,\n", + " 'sampler': {'name': 'SemSegRandomSampler'}}" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "cfg.dataset" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'name': 'RandLANet',\n", + " 'batcher': 'DefaultBatcher',\n", + " 'ckpt_path': None,\n", + " 'num_neighbors': 16,\n", + " 'num_layers': 4,\n", + " 'num_points': 45056,\n", + " 'num_classes': 19,\n", + " 'ignored_label_inds': [0],\n", + " 'sub_sampling_ratio': [4, 4, 4, 4],\n", + " 'in_channels': 3,\n", + " 'dim_features': 8,\n", + " 'dim_output': [16, 64, 128, 256],\n", + " 'grid_size': 0.06,\n", + " 'augment': {'recenter': {'dim': [0, 1]}}}" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "cfg.model" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'name': 'SemanticSegmentation',\n", + " 'optimizer': {'lr': 0.001},\n", + " 'batch_size': 4,\n", + " 'main_log_dir': './logs',\n", + " 'max_epoch': 100,\n", + " 'save_ckpt_freq': 5,\n", + " 'scheduler_gamma': 0.9886,\n", + " 'test_batch_size': 1,\n", + " 'train_sum_dir': 'train_log',\n", + " 'val_batch_size': 2,\n", + " 'summary': {'record_for': [],\n", + " 'max_pts': None,\n", + " 'use_reference': False,\n", + " 'max_outputs': 1}}" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "cfg.pipeline" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "dataset = ml3d.datasets.SemanticKITTI(cfg.dataset)" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'cfg': ,\n", + " 'name': 'SemanticKITTI',\n", + " 'rng': Generator(PCG64) at 0x7FDA4855C450,\n", + " 'label_to_names': {0: 'unlabeled',\n", + " 1: 'car',\n", + " 2: 'bicycle',\n", + " 3: 'motorcycle',\n", + " 4: 'truck',\n", + " 5: 'other-vehicle',\n", + " 6: 'person',\n", + " 7: 'bicyclist',\n", + " 8: 'motorcyclist',\n", + " 9: 'road',\n", + " 10: 'parking',\n", + " 11: 'sidewalk',\n", + " 12: 'other-ground',\n", + " 13: 'building',\n", + " 14: 'fence',\n", + " 15: 'vegetation',\n", + " 16: 'trunk',\n", + " 17: 'terrain',\n", + " 18: 'pole',\n", + " 19: 'traffic-sign'},\n", + " 'num_classes': 20,\n", + " 'remap_lut_val': array([ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 0, 5, 0, 3, 5,\n", + " 0, 4, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 7, 8, 0,\n", + " 0, 0, 0, 0, 0, 0, 9, 0, 0, 0, 10, 0, 0, 0, 11, 12, 13,\n", + " 14, 0, 0, 0, 0, 0, 0, 0, 0, 9, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 15, 16, 17, 0, 0, 0, 0, 0, 0, 0, 18, 19, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 7, 6,\n", + " 8, 5, 5, 4, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0], dtype=int32),\n", + " 'remap_lut': array([ 0, 10, 11, 15, 18, 20, 30, 31, 32, 40, 44, 48, 49, 50, 51, 70, 71,\n", + " 72, 80, 81, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", + " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n", + " dtype=int32)}" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "vars(dataset)" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "20" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dataset.num_classes" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.7" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb index 98fac6bfb..a2cda6052 100644 --- a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb +++ b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb @@ -2,7 +2,6 @@ "cells": [ { "cell_type": "markdown", - "id": "f6567deb", "metadata": {}, "source": [ "# Training a Semantic Segmentation Model Using PyTorch" @@ -10,7 +9,6 @@ }, { "cell_type": "markdown", - "id": "33cfdf83", "metadata": {}, "source": [ "In this tutorial, we will learn how to train a semantic segmentation model using PyTorch in a Jupyter Notebook. We assume that you are familiar with Jupyter Notebook and have created a folder notebooks in a folder that is relative to ml3d.\n", @@ -24,7 +22,6 @@ }, { "cell_type": "markdown", - "id": "cfa831d8", "metadata": {}, "source": [ "At a high-level, we will:\n", @@ -37,7 +34,6 @@ }, { "cell_type": "markdown", - "id": "95993f3e", "metadata": {}, "source": [ "## Reading a dataset" @@ -45,7 +41,6 @@ }, { "cell_type": "markdown", - "id": "c210491d", "metadata": {}, "source": [ "You can use any dataset available in the `ml3d.datasets` dataset namespace. For this example, we will use the `SemanticKITTI` dataset and visualize it. You can use any of the other dataset to load data. However, you must understand that the parameters may vary for each dataset.\n", @@ -56,7 +51,6 @@ { "cell_type": "code", "execution_count": null, - "id": "54e07a36", "metadata": {}, "outputs": [], "source": [ @@ -79,7 +73,6 @@ }, { "cell_type": "markdown", - "id": "0b5eccfd", "metadata": {}, "source": [ "Now that you have visualized the dataset for training, let us train the model." @@ -87,7 +80,6 @@ }, { "cell_type": "markdown", - "id": "43a70525", "metadata": {}, "source": [ "## Training a model" @@ -95,7 +87,6 @@ }, { "cell_type": "markdown", - "id": "d33ddf74", "metadata": {}, "source": [ "Before you train a model, you must decide which model you want to use. For this example, we will use the `RandLANet` model. To use models, you must import the model from `open3d.ml.torch.models`.\n", @@ -106,7 +97,6 @@ { "cell_type": "code", "execution_count": 1, - "id": "b15cace7", "metadata": {}, "outputs": [ { @@ -204,7 +194,6 @@ }, { "cell_type": "markdown", - "id": "0aed5aef", "metadata": {}, "source": [ "## Running an Inference" @@ -212,7 +201,6 @@ }, { "cell_type": "markdown", - "id": "ef6472cc", "metadata": {}, "source": [ "An inference processes point cloud and displays the results based on the trained model. For this example, we will use a trained `RandLANet` model.\n", @@ -223,7 +211,6 @@ { "cell_type": "code", "execution_count": 2, - "id": "edd51edf", "metadata": {}, "outputs": [ { @@ -283,7 +270,6 @@ }, { "cell_type": "markdown", - "id": "9553a541", "metadata": {}, "source": [ "## Running a Test" @@ -291,7 +277,6 @@ }, { "cell_type": "markdown", - "id": "625f66bc", "metadata": {}, "source": [ "Running a test is very similar to running an inference on Jupyter.\n", @@ -302,7 +287,6 @@ { "cell_type": "code", "execution_count": 3, - "id": "2a8c5649", "metadata": {}, "outputs": [ { @@ -346,7 +330,6 @@ { "cell_type": "code", "execution_count": null, - "id": "0f03fae8", "metadata": {}, "outputs": [], "source": [] @@ -354,7 +337,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "Python 3", "language": "python", "name": "python3" }, @@ -368,7 +351,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.12" + "version": "3.7.7" } }, "nbformat": 4, From ad88ffc7809e040dc29eb9c68c04c97e13a614c4 Mon Sep 17 00:00:00 2001 From: AlexZakIntel Date: Mon, 21 Mar 2022 17:13:13 -0400 Subject: [PATCH 03/13] 3-21-2021 - Reading a Config File Tutorial - work in progress --- .../notebook/reading_a_config_file.ipynb | 330 ++++++++++++++++-- 1 file changed, 308 insertions(+), 22 deletions(-) diff --git a/docs/tutorial/notebook/reading_a_config_file.ipynb b/docs/tutorial/notebook/reading_a_config_file.ipynb index 3011b4336..dcc03e36a 100644 --- a/docs/tutorial/notebook/reading_a_config_file.ipynb +++ b/docs/tutorial/notebook/reading_a_config_file.ipynb @@ -1,18 +1,111 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Reading a *config* file" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In other examples, we examine the following snippet of code which creates a dataset:\n", + "\n", + "```py\n", + "# Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", + "\n", + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n", + "```\n", + "\n", + "In the code above, the `dataset` object is created by explicitly passing dataset-spcific parameters into `ml3d.datasets.SemanticKITTI()` method.\n", + "\n", + "Instead of passing a bunch of parameters to a function call as part of `dataset` object creation, we can supply `dataset` information from a specific *config* file. Each *config* file contains information specific to a given dataset.\n", + "\n", + ">***Config* files pass `dataset` information into Open3D-ML in YAML format.**\n", + "\n", + ">>>Do they hold any configuration-specific information???\n", + "\n", + "// When a `dataset` object data is read by Open3D-ML, `dataset` parameters are parsed by Open3D-ML subsystem based on configuration information in the _config_ YAML files.\n", + "\n", + "// In this example, we will be reading a *config* file and parsing `dataset` information from it.\n", + "\n", + "In this example, we will:\n", + "\n", + "- Load a *config* `cfg_file` into an `cfg` object;\n", + "- Parse `dataset` dictionaries from the `cfg` object;\n", + "- Access individual dictionaries in the `cfg` object;\n", + "- Access individual elements from within dictionaries.\n", + "\n", + "\n", + "## Loading a *config* file" + ] + }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 1, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Jupyter environment detected. Enabling Open3D WebVisualizer.\n", + "[Open3D INFO] WebRTC GUI backend enabled.\n", + "[Open3D INFO] WebRTCWindowSystem: HTTP handshake server disabled.\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\n", + " Using the Open3D PyTorch ops with CUDA 11 may have stability issues!\n", + "\n", + " We recommend to compile PyTorch from source with compile flags\n", + " '-Xcompiler -fno-gnu-unique'\n", + "\n", + " or use the PyTorch wheels at\n", + " https://github.com/isl-org/open3d_downloads/releases/tag/torch1.8.2\n", + "\n", + "\n", + " Ignore this message if PyTorch has been compiled with the aforementioned\n", + " flags.\n", + "\n", + " See https://github.com/isl-org/Open3D/issues/3324 and\n", + " https://github.com/pytorch/pytorch/issues/52663 for more information on this\n", + " problem.\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2022-03-21 11:19:34.531469: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", + "2022-03-21 11:19:34.531795: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n" + ] + } + ], "source": [ "from open3d.ml import utils\n", "import open3d.ml.torch as ml3d" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here, we instantiate two objects:\n", + " \n", + " 1. `utils` - Open3D-ML utilities\n", + " 2. `ml3d` - Open3D-ML Torch library object\n", + "\n", + "Now, we'll create *config*-specific objects:" + ] + }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ @@ -20,6 +113,18 @@ "cfg = utils.Config.load_from_file(cfg_file)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `cfg_file` holds the full path to the particular *config* file - `randlanet_semantickitti.yml`.\n", + "The `cfg` object is initialized by the Open3D-ML `utils.Config.load_from_file()` method to hold configuration data read from the `cfg_file`.\n", + "\n", + "## Examining Dataset Dictionaries in the `cfg` Object\n", + "\n", + "To find out what type the `cfg` object is, we can enter it in the next code cell:" + ] + }, { "cell_type": "code", "execution_count": 3, @@ -28,7 +133,7 @@ { "data": { "text/plain": [ - "" + "" ] }, "execution_count": 3, @@ -40,9 +145,18 @@ "cfg" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Open3D-ML confirms the `cfg` is of `ml3d.utils.config.Config` type. The `cfg` object contains three dictionaries: `dataset`, `model`, and `pipeline`.\n", + "\n", + "Now, let's explore them. The first one to look at is the `cfg.dataset`:" + ] + }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 4, "metadata": {}, "outputs": [ { @@ -119,7 +233,7 @@ " 'sampler': {'name': 'SemSegRandomSampler'}}" ] }, - "execution_count": 18, + "execution_count": 4, "metadata": {}, "output_type": "execute_result" } @@ -128,10 +242,50 @@ "cfg.dataset" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Accessing Individual Dictionary Elements\n", + "\n", + "We can access individual items of the `cfg.dataset` dictionary like so: " + ] + }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 6, "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'SemanticKITTI'" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "cfg.dataset['name']" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## `cfg.model` and `cfg.pipeline` Dictionaries\n", + "\n", + "We'll later revisit the `cfg.dataset`. Next, let's look at the `cfg.model` dictionary:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "scrolled": true + }, "outputs": [ { "data": { @@ -152,7 +306,7 @@ " 'augment': {'recenter': {'dim': [0, 1]}}}" ] }, - "execution_count": 20, + "execution_count": 8, "metadata": {}, "output_type": "execute_result" } @@ -161,9 +315,43 @@ "cfg.model" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Just as in the case of `cfg.dataset`, we can access `cfg.model` dictionary items by referencing them individually:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[4, 4, 4, 4]" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "cfg.model['sub_sampling_ratio']" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`cfg.pipeline` is the last dictionary item of the `cfg` object:" + ] + }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 11, "metadata": {}, "outputs": [ { @@ -185,7 +373,7 @@ " 'max_outputs': 1}}" ] }, - "execution_count": 9, + "execution_count": 11, "metadata": {}, "output_type": "execute_result" } @@ -194,26 +382,71 @@ "cfg.pipeline" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Likewise, individual dictionary items in `cfg.pipeline` can be accesed just like those of `cfg.model` and `cfg.dataset`:" + ] + }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'SemanticSegmentation'" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "cfg.pipeline['name']" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Working with Individual Datasets\n", + "\n", + "Next, we explicitly create the `dataset` object which will hold all information from `cfg.dataset`:" + ] + }, + { + "cell_type": "code", + "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "dataset = ml3d.datasets.SemanticKITTI(cfg.dataset)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the above code segment we initialize the `dataset` object with the contents of `cfg.dataset` dictionary. Next, we'll look at what properties the newly-created `dataset` object exposes with the `vars()` function:\n", + "\n", + "> Is `vars()` a Python-specific function???" + ] + }, { "cell_type": "code", - "execution_count": 22, + "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "{'cfg': ,\n", + "{'cfg': ,\n", " 'name': 'SemanticKITTI',\n", - " 'rng': Generator(PCG64) at 0x7FDA4855C450,\n", + " 'rng': Generator(PCG64) at 0x7F75F536DE50,\n", " 'label_to_names': {0: 'unlabeled',\n", " 1: 'car',\n", " 2: 'bicycle',\n", @@ -267,7 +500,7 @@ " dtype=int32)}" ] }, - "execution_count": 22, + "execution_count": 18, "metadata": {}, "output_type": "execute_result" } @@ -276,9 +509,16 @@ "vars(dataset)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can reference any property of the dataset by using *`object.property`* syntax. For example, to find out what value the `num_classes` property holds, we type in:" + ] + }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 19, "metadata": {}, "outputs": [ { @@ -287,7 +527,7 @@ "20" ] }, - "execution_count": 13, + "execution_count": 19, "metadata": {}, "output_type": "execute_result" } @@ -296,17 +536,63 @@ "dataset.num_classes" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Likewise, to extract information from a `label_to_names` **property ?????** which acts as a legend mapping a numeric code for a recognized object to a human-readable label, we type in:" + ] + }, { "cell_type": "code", - "execution_count": null, + "execution_count": 20, "metadata": {}, - "outputs": [], - "source": [] + "outputs": [ + { + "data": { + "text/plain": [ + "{0: 'unlabeled',\n", + " 1: 'car',\n", + " 2: 'bicycle',\n", + " 3: 'motorcycle',\n", + " 4: 'truck',\n", + " 5: 'other-vehicle',\n", + " 6: 'person',\n", + " 7: 'bicyclist',\n", + " 8: 'motorcyclist',\n", + " 9: 'road',\n", + " 10: 'parking',\n", + " 11: 'sidewalk',\n", + " 12: 'other-ground',\n", + " 13: 'building',\n", + " 14: 'fence',\n", + " 15: 'vegetation',\n", + " 16: 'trunk',\n", + " 17: 'terrain',\n", + " 18: 'pole',\n", + " 19: 'traffic-sign'}" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dataset.label_to_names" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Experiment with other `dataset` properties to see how convenient it is to reference them." + ] } ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -320,7 +606,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.7" + "version": "3.8.12" } }, "nbformat": 4, From d3f3bed1e69da5aefec1490763a7e846f149d022 Mon Sep 17 00:00:00 2001 From: AlexZakIntel Date: Tue, 22 Mar 2022 13:49:10 -0400 Subject: [PATCH 04/13] Added a final version of the Config File Tutorial - 3-22-2022 --- .../notebook/reading_a_config_file.ipynb | 34 ++++++++----------- 1 file changed, 14 insertions(+), 20 deletions(-) diff --git a/docs/tutorial/notebook/reading_a_config_file.ipynb b/docs/tutorial/notebook/reading_a_config_file.ipynb index dcc03e36a..d4263c94e 100644 --- a/docs/tutorial/notebook/reading_a_config_file.ipynb +++ b/docs/tutorial/notebook/reading_a_config_file.ipynb @@ -19,23 +19,19 @@ "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n", "```\n", "\n", - "In the code above, the `dataset` object is created by explicitly passing dataset-spcific parameters into `ml3d.datasets.SemanticKITTI()` method.\n", + "In the code above, the `dataset` object is created by explicitly passing dataset-specific parameters into `ml3d.datasets.SemanticKITTI()` method.\n", "\n", - "Instead of passing a bunch of parameters to a function call as part of `dataset` object creation, we can supply `dataset` information from a specific *config* file. Each *config* file contains information specific to a given dataset.\n", + "Instead of passing a bunch of parameters to a function call as part of `dataset` object creation, we can supply `dataset` information from a specific *config* file. Each *config* file contains parameters for for a dataset, model and pipeline.\n", "\n", - ">***Config* files pass `dataset` information into Open3D-ML in YAML format.**\n", "\n", - ">>>Do they hold any configuration-specific information???\n", + ">***Config* files pass information into Open3D-ML in YAML format.**\n", "\n", - "// When a `dataset` object data is read by Open3D-ML, `dataset` parameters are parsed by Open3D-ML subsystem based on configuration information in the _config_ YAML files.\n", - "\n", - "// In this example, we will be reading a *config* file and parsing `dataset` information from it.\n", "\n", "In this example, we will:\n", "\n", - "- Load a *config* `cfg_file` into an `cfg` object;\n", - "- Parse `dataset` dictionaries from the `cfg` object;\n", - "- Access individual dictionaries in the `cfg` object;\n", + "- Load a *config* `cfg_file` into a `Config` class object;\n", + "- Parse `dataset` dictionaries from the `Config` object;\n", + "- Access individual dictionaries in the `Config` object;\n", "- Access individual elements from within dictionaries.\n", "\n", "\n", @@ -95,10 +91,10 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Here, we instantiate two objects:\n", + "Here, we import two modules:\n", " \n", " 1. `utils` - Open3D-ML utilities\n", - " 2. `ml3d` - Open3D-ML Torch library object\n", + " 2. `ml3d` - Open3D-ML Torch library\n", "\n", "Now, we'll create *config*-specific objects:" ] @@ -118,7 +114,7 @@ "metadata": {}, "source": [ "The `cfg_file` holds the full path to the particular *config* file - `randlanet_semantickitti.yml`.\n", - "The `cfg` object is initialized by the Open3D-ML `utils.Config.load_from_file()` method to hold configuration data read from the `cfg_file`.\n", + "The `cfg` object is initialized by the Open3D-ML `utils.Config.load_from_file()` method to hold parameters that are read from the `cfg_file`.\n", "\n", "## Examining Dataset Dictionaries in the `cfg` Object\n", "\n", @@ -246,7 +242,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Accessing Individual Dictionary Elements\n", + "### Accessing Individual Dictionary Items\n", "\n", "We can access individual items of the `cfg.dataset` dictionary like so: " ] @@ -413,9 +409,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Working with Individual Datasets\n", + "## Initializing Datasets from *Config* Files\n", "\n", - "Next, we explicitly create the `dataset` object which will hold all information from `cfg.dataset`:" + "Next, we explicitly create the `dataset` object which will hold all information from the `cfg.dataset` dictionary:" ] }, { @@ -431,9 +427,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In the above code segment we initialize the `dataset` object with the contents of `cfg.dataset` dictionary. Next, we'll look at what properties the newly-created `dataset` object exposes with the `vars()` function:\n", - "\n", - "> Is `vars()` a Python-specific function???" + "Next, we'll look at what properties the newly-created `dataset` object exposes with the Python `vars()` function:" ] }, { @@ -540,7 +534,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Likewise, to extract information from a `label_to_names` **property ?????** which acts as a legend mapping a numeric code for a recognized object to a human-readable label, we type in:" + "Likewise, to extract information from a `label_to_names` property which maps labels to the objects names, we call:" ] }, { From 8babd52d8495809085c4405adb0a41cade32e6d5 Mon Sep 17 00:00:00 2001 From: Sanskar Agrawal Date: Thu, 24 Mar 2022 20:03:08 +0530 Subject: [PATCH 05/13] add reading_a_dataset ipynb --- .../tutorial/notebook/reading_a_dataset.ipynb | 198 ++++++++++++++++++ 1 file changed, 198 insertions(+) create mode 100644 docs/tutorial/notebook/reading_a_dataset.ipynb diff --git a/docs/tutorial/notebook/reading_a_dataset.ipynb b/docs/tutorial/notebook/reading_a_dataset.ipynb new file mode 100644 index 000000000..698a5dfee --- /dev/null +++ b/docs/tutorial/notebook/reading_a_dataset.ipynb @@ -0,0 +1,198 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "#import torch\n", + "import open3d.ml.torch as ml3d" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "#Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='/Users/sanskara/data/SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO - 2022-03-24 19:48:01,057 - semantickitti - Found 1 pointclouds for training\n" + ] + } + ], + "source": [ + "#Split the dataset for 'training'. You can get the other splits by passing 'validation' or 'test'\n", + "train_split = dataset.get_split('training')" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO - 2022-03-24 19:49:34,283 - semantickitti - Found 1 pointclouds for validation\n" + ] + } + ], + "source": [ + "#Similarly, get validataion split.\n", + "val_split = dataset.get_split('validation')" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO - 2022-03-24 19:47:58,953 - semantickitti - Found 1 pointclouds for test\n" + ] + } + ], + "source": [ + "#get test split\n", + "test_split = dataset.get_split('test')" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1\n", + "1\n", + "1\n" + ] + } + ], + "source": [ + "# Get length of splits\n", + "print(len(train_split))\n", + "print(len(val_split))\n", + "print(len(test_split))" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'point': array([[ 5.2305943e+01, 2.2989707e-02, 1.9779946e+00],\n", + " [ 5.3259735e+01, 1.0695236e-01, 2.0099745e+00],\n", + " [ 5.3284321e+01, 2.7487758e-01, 2.0109341e+00],\n", + " ...,\n", + " [ 3.8249431e+00, -1.4261885e+00, -1.7655631e+00],\n", + " [ 3.8495324e+00, -1.4222100e+00, -1.7755738e+00],\n", + " [ 3.8631279e+00, -1.4142324e+00, -1.7805853e+00]], dtype=float32), 'feat': None, 'label': array([0, 0, 0, ..., 9, 9, 9], dtype=int32)}\n" + ] + } + ], + "source": [ + "# Query splits for data, index should be from `0` to `len(split) - 1`\n", + "for i in range(len(train_split)):\n", + " data = train_split.get_data(i)\n", + " print(data)\n", + " break" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "dict_keys(['point', 'feat', 'label'])\n" + ] + } + ], + "source": [ + "data = train_split.get_data(0) # Dictionary of `point`, `feat`, and `label`\n", + "print(data.keys())" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'idx': 0, 'name': '00_000001', 'path': '/Users/sanskara/data/SemanticKITTI/dataset/sequences/00/velodyne/000001.bin', 'split': 'training'}\n" + ] + } + ], + "source": [ + "attr = train_split.get_attr(0)\n", + "print(attr) # Dictionary containing information about the data e.g. name, path, split, etc." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "#support of Open3d-ML visualizer in Jupyter Notebooks is in progress\n", + "#view the frames using the visualizer\n", + "#vis = ml3d.vis.Visualizer()\n", + "#vis.visualize_dataset(dataset, 'training',indices=range(len(train_split)))" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.7" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} From b8df5639fa1165fe986426fe30015a184a2c2878 Mon Sep 17 00:00:00 2001 From: AlexZakIntel Date: Tue, 29 Mar 2022 10:55:23 -0400 Subject: [PATCH 06/13] 3-29-2022 - Added a new Reading a Dataset tutorial --- .../tutorial/notebook/reading_a_dataset.ipynb | 176 ++++++++++++++++-- 1 file changed, 156 insertions(+), 20 deletions(-) diff --git a/docs/tutorial/notebook/reading_a_dataset.ipynb b/docs/tutorial/notebook/reading_a_dataset.ipynb index 698a5dfee..0ac5ea519 100644 --- a/docs/tutorial/notebook/reading_a_dataset.ipynb +++ b/docs/tutorial/notebook/reading_a_dataset.ipynb @@ -1,8 +1,40 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Reading a Dataset" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this tutorial, we are going to learn how to read Open3D-ML datasets.\n", + "\n", + "You can use any dataset available in the `ml3d.datasets` dataset namespace. For this example, we will use the `SemanticKITTI` dataset. You can use any of the other datasets to load data. However, you must understand that the parameters may vary for each dataset.\n", + "\n", + "In order to read a dataset in this example, we will supply the following parameter variables:\n", + "\n", + "- Dataset path (`dataset_path`)\n", + "- Cache directory (`cache_dir`)\n", + "- Dataset Splits (for training, validation, and testing)\n", + "\n", + "> **For more theoretical background information on dataset splitting, please refer to these articles:**\n", + ">\n", + "> https://machinelearningcompass.com/dataset_optimization/split_data_the_right_way/\n", + ">\n", + "> https://www.freecodecamp.org/news/key-machine-learning-concepts-explained-dataset-splitting-and-random-forest/\n", + "\n", + "## Creating a Global Dataset Object\n", + "\n", + "First, we declare the `ml3d` object to be of `open3d.ml.torch` library:" + ] + }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 5, "metadata": {}, "outputs": [], "source": [ @@ -10,14 +42,50 @@ "import open3d.ml.torch as ml3d" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We then create a global `dataset` object, initializing it with *path, cache directory*, and *splits*. This `dataset` consists of all files specified in the `dataset_path=''` variable:" + ] + }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 6, "metadata": {}, "outputs": [], "source": [ - "#Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", - "dataset = ml3d.datasets.SemanticKITTI(dataset_path='/Users/sanskara/data/SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])" + "# Read a dataset by specifying the path. We are also providing the cache directory and splits.\n", + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache', training_split=['00'], validation_split=['01'], test_split=['01'])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A couple of words regarding the *splits* variables: here, we isolate different portions of the `SemanticKITTI` dataset content and divide them into 3 different parts:\n", + "\n", + "1. `training_split` for data training. This part usually contains 70-75% of the global `dataset` content.\n", + "2. `validation_split` for data validation. This part accounts for 10-15% of the global `dataset` content;\n", + "3. `test_split` for testing. It contains test data and its size varies.\n", + "\n", + "Note the `SemanticKITTI` dataset folder structure:\n", + "\n", + "![dataset_splits](https://user-images.githubusercontent.com/93158890/160633509-e190d8ea-7d3c-4eea-a00f-29ba215b7e69.jpg)\n", + "\n", + "The three different *split* parameter variables instruct Open3D-ML subsystem to reference the following folder locations:\n", + "\n", + "- `training_split=['00']` points to `'SemanticKITTI/dataset/sequences/00/'`\n", + "- `validation_split=['01']` points to `'SemanticKITTI/dataset/sequences/01/'`\n", + "- `test_split=['01']` points to `'SemanticKITTI/dataset/sequences/01/'`\n", + "\n", + "> Note: **Dataset split directories usually contain numerous pointcloud files.** In our example we included only one pointcloud file for extra speed and convenience.\n", + "\n", + "## Creating Dataset Split Objects to Query the Data\n", + "\n", + "Next, we will create **dedicated** dataset split objects for specifying which split portion we would like to query.\n", + "\n", + "First, we create a `train_split` subset for training from the global `dataset` content we have initialized above using the `get_split()` method:" ] }, { @@ -31,54 +99,77 @@ "name": "stderr", "output_type": "stream", "text": [ - "INFO - 2022-03-24 19:48:01,057 - semantickitti - Found 1 pointclouds for training\n" + "INFO - 2022-03-29 10:31:54,916 - semantickitti - Found 1 pointclouds for training\n" ] } ], "source": [ - "#Split the dataset for 'training'. You can get the other splits by passing 'validation' or 'test'\n", + "# Split the dataset for 'training'. You can get the other splits by passing 'validation' or 'test'\n", "train_split = dataset.get_split('training')" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, let's do the same for validation:" + ] + }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "INFO - 2022-03-24 19:49:34,283 - semantickitti - Found 1 pointclouds for validation\n" + "INFO - 2022-03-29 10:31:54,926 - semantickitti - Found 1 pointclouds for validation\n" ] } ], "source": [ - "#Similarly, get validataion split.\n", + "# Similarly, get validataion split.\n", "val_split = dataset.get_split('validation')" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we create a `test_split` subset for testing:" + ] + }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "INFO - 2022-03-24 19:47:58,953 - semantickitti - Found 1 pointclouds for test\n" + "INFO - 2022-03-29 10:31:54,939 - semantickitti - Found 1 pointclouds for test\n" ] } ], "source": [ - "#get test split\n", + "# Get test split\n", "test_split = dataset.get_split('test')" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Determining the Size of Dataset Splits\n", + "\n", + "Let's see how large out *split* portions are:" + ] + }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 10, "metadata": {}, "outputs": [ { @@ -98,9 +189,23 @@ "print(len(test_split))" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Above, Open3D-ML prints out the number of pointcloud files it found in `'SemanticKITTI/dataset/sequences/'`' `'/00/'` and `'/01/'` subdirectories we have specified earlier in `training_split=['00'], validation_split=['01'], test_split=['01']` varables for the `dataset`.\n", + "\n", + "## Querying Dataset Splits for Data\n", + "\n", + "In this section, we are using the `train_split` dataset split object as an example. The procedure would be identical for all other dataset splits - `val_split` and `test_split`.\n", + "\n", + "In order to extract the data from the `train_split`, we can iterate through the `train_split` with the index `i` (ranging from `0` - `len(train_split)-1`) using the `get_data()` method.\n", + ":" + ] + }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 11, "metadata": {}, "outputs": [ { @@ -125,9 +230,16 @@ " break" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`data` objects from the above `for` loop return a dictionary of points (`'point'`), features (`'feat'`), and labels (`'label'`), as we will see below:" + ] + }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 12, "metadata": {}, "outputs": [ { @@ -143,16 +255,33 @@ "print(data.keys())" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "- The **`'point'`** key value contains a set of 3D points / coordinates - X, Y, and Z:\n", + "\n", + "![dataset_coordinates](https://user-images.githubusercontent.com/93158890/160503607-76e77f7a-56be-4fba-91c5-7e35019f68e8.jpg)\n", + "\n", + "- The **`'feat'`** (*features*) key value contains RGB color information for each of the above points.\n", + "\n", + "- The **`'label'`** key value represents which class the dataset content belongs to, i.e.: *pedestrian, vehicle, traffic light*, etc.\n", + "\n", + "### Querying Dataset Splits for Attributes\n", + "\n", + "We can also extract corresponding pointcloud information:" + ] + }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "{'idx': 0, 'name': '00_000001', 'path': '/Users/sanskara/data/SemanticKITTI/dataset/sequences/00/velodyne/000001.bin', 'split': 'training'}\n" + "{'idx': 0, 'name': '00_000001', 'path': 'SemanticKITTI/dataset/sequences/00/velodyne/000001.bin', 'split': 'training'}\n" ] } ], @@ -161,9 +290,16 @@ "print(attr) # Dictionary containing information about the data e.g. name, path, split, etc." ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Atttributes returned are: `'idx'`(*index*), `'name'`, `'path'`, and `'split'`." + ] + }, { "cell_type": "code", - "execution_count": null, + "execution_count": 14, "metadata": {}, "outputs": [], "source": [ @@ -176,7 +312,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -190,7 +326,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.7" + "version": "3.8.12" } }, "nbformat": 4, From c4279307f87d5051cfef65df7e3fadd821436200 Mon Sep 17 00:00:00 2001 From: Sanskar Agrawal Date: Fri, 1 Apr 2022 20:18:30 +0530 Subject: [PATCH 07/13] add inference_on_custom_data.ipynb --- .../notebook/Inference_on_a_custom_data.ipynb | 210 ++++++++++++++++++ .../Inference_on_a_custom_dataset.ipynb | 51 ----- 2 files changed, 210 insertions(+), 51 deletions(-) create mode 100644 docs/tutorial/notebook/Inference_on_a_custom_data.ipynb delete mode 100644 docs/tutorial/notebook/Inference_on_a_custom_dataset.ipynb diff --git a/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb b/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb new file mode 100644 index 000000000..fc344c522 --- /dev/null +++ b/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb @@ -0,0 +1,210 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Using external Open3D-ML in /Users/sanskara/Workspace/Open3D-ML\n" + ] + } + ], + "source": [ + "import open3d.ml.torch as ml3d # just switch to open3d.ml.tf for tf usage\n", + "import numpy as np" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "# Download weights using link from model zoo(collection of weights for all combinations of model and dataset).\n", + "# Link : https://github.com/isl-org/Open3D-ML#semantic-segmentation-1\n", + "# Randlanet SemanticKITTI[PyTorch] : https://storage.googleapis.com/open3d-releases/model-zoo/randlanet_semantickitti_202201071330utc.pth\n", + "ckpt_path = './randlanet_semantickitti_202201071330utc.pth'" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "# We define dataset (similar to train_ss_using_pytorch tutorial)\n", + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='/Users/sanskara/data/SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "# Initializing the model and pipeline\n", + "model = ml3d.models.RandLANet(in_channels=3)\n", + "pipeline = ml3d.pipelines.SemanticSegmentation(model)" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO - 2022-04-01 19:58:03,220 - semantic_segmentation - Loading checkpoint ./randlanet_semantickitti_202201071330utc.pth\n" + ] + } + ], + "source": [ + "# Load checkpoint using `load_ckpt` method (restoring weights for inference)\n", + "pipeline.load_ckpt(ckpt_path=ckpt_path)" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO - 2022-04-01 20:12:49,159 - semantickitti - Found 1 pointclouds for test\n" + ] + } + ], + "source": [ + "test_data = dataset.get_split('test')\n", + "data = test_data.get_data(0)" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'point': array([[ 5.2305943e+01, 2.2989707e-02, 1.9779946e+00],\n", + " [ 5.3259735e+01, 1.0695236e-01, 2.0099745e+00],\n", + " [ 5.3284321e+01, 2.7487758e-01, 2.0109341e+00],\n", + " ...,\n", + " [ 3.8249431e+00, -1.4261885e+00, -1.7655631e+00],\n", + " [ 3.8495324e+00, -1.4222100e+00, -1.7755738e+00],\n", + " [ 3.8631279e+00, -1.4142324e+00, -1.7805853e+00]], dtype=float32), 'feat': None, 'label': array([0, 0, 0, ..., 9, 9, 9], dtype=int32)}\n" + ] + } + ], + "source": [ + "# For inference on custom data, you can convert your pointcloud in this format.\n", + "# Dictionary with keys {'point', 'feat', 'label'}\n", + "# If you already have the ground truth labels, you can add it to data to get accuracy and IoU.\n", + "# otherwise pass labels as `None`.\n", + "print(data)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "test 0/1: 100%|██████████| 83500/83500 [03:51<00:00, 360.18it/s] \n", + "test 0/1: 100%|██████████| 83500/83500 [00:15<00:00, 303.57it/s] INFO - 2022-04-01 20:09:24,024 - semantic_segmentation - Accuracy : [0.963756553453385, nan, nan, nan, nan, nan, nan, 0.3561643835616438, 0.9540482683792264, 0.9261289480553381, 0.4870199692780338, nan, 0.9735662148070907, 0.966321243523316, 0.825527363917982, 0.8268921095008052, 0.9399638336347197, 0.9401709401709402, 0.62, 0.81496331902354]\n", + "INFO - 2022-04-01 20:09:24,025 - semantic_segmentation - IoU : [0.9090518168135885, nan, 0.0, nan, 0.0, nan, 0.0, 0.3561643835616438, 0.9177174127669178, 0.21501727168050422, 0.47786276800180866, 0.0, 0.9568537022802972, 0.10855646100116415, 0.813112967671631, 0.7186843946815955, 0.6830486202365309, 0.843558282208589, 0.543859649122807, 0.4714679831266924]\n" + ] + } + ], + "source": [ + "# Running inference on test data\n", + "results = pipeline.run_inference(data)\n", + "# prints per class accuracy and IoU. Last entry is mean accuracy and mean IoU.\n", + "# We get several `nan` outputs for missing classes in the input data." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'predict_labels': array([12, 12, 12, ..., 8, 8, 8], dtype=int16),\n", + " 'predict_scores': array([[2.1815e-05, 1.1027e-05, 7.1526e-07, ..., 1.4412e-04, 3.7403e-03,\n", + " 1.8473e-03],\n", + " [1.3769e-05, 5.7220e-06, 5.3644e-07, ..., 1.2094e-04, 4.3449e-03,\n", + " 2.5768e-03],\n", + " [8.1062e-06, 4.7088e-06, 3.5763e-07, ..., 1.0777e-04, 5.9471e-03,\n", + " 3.2253e-03],\n", + " ...,\n", + " [0.0000e+00, 0.0000e+00, 0.0000e+00, ..., 0.0000e+00, 0.0000e+00,\n", + " 0.0000e+00],\n", + " [0.0000e+00, 0.0000e+00, 0.0000e+00, ..., 0.0000e+00, 0.0000e+00,\n", + " 0.0000e+00],\n", + " [0.0000e+00, 0.0000e+00, 0.0000e+00, ..., 0.0000e+00, 0.0000e+00,\n", + " 0.0000e+00]], dtype=float16)}" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\r", + "test 0/1: 100%|██████████| 83500/83500 [00:32<00:00, 303.57it/s]" + ] + } + ], + "source": [ + "# Dictionary of predicted labels and predicted probabilities per class\n", + "results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.7" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/docs/tutorial/notebook/Inference_on_a_custom_dataset.ipynb b/docs/tutorial/notebook/Inference_on_a_custom_dataset.ipynb deleted file mode 100644 index d1e4cc33e..000000000 --- a/docs/tutorial/notebook/Inference_on_a_custom_dataset.ipynb +++ /dev/null @@ -1,51 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "#Import torch and the model to use for training\n", - "import open3d.ml.torch as ml3d\n", - "from open3d.ml.torch.models import RandLANet\n", - "from open3d.ml.torch.pipelines import SemanticSegmentation\n", - "\n", - "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n", - "model = RandLANet(in_channels=3)\n", - "pipeline = SemanticSegmentation(model=model, dataset=dataset, max_epoch=3, optimizer={'lr': 0.001}, num_workers=0)\n", - "\n", - "#Get data from the SemanticKITTI dataset using the \"train\" split\n", - "train_split = dataset.get_split(\"test\")\n", - "data = train_split.get_data(0)\n", - "\n", - "#Run the inference\n", - "results = pipeline.run_inference(data)\n", - "\n", - "#Print the results\n", - "print(results)" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.7.7" - } - }, - "nbformat": 4, - "nbformat_minor": 4 -} From 1fdbf8b3a24c24825bd336fe0540e8fe1ec91114 Mon Sep 17 00:00:00 2001 From: AlexZakIntel Date: Tue, 5 Apr 2022 10:57:28 -0400 Subject: [PATCH 08/13] 4-5-2022 - Added Inference on a Custom Data tutorial & edited other 3 tutorials --- .../notebook/Inference_on_a_custom_data.ipynb | 191 +++++++++++++----- .../train_ss_model_using_pytorch.ipynb | 67 +++--- .../train_ss_model_using_tensorflow.ipynb | 49 ++--- .../train_ss_model_using_tensorflow.rst | 2 +- 4 files changed, 206 insertions(+), 103 deletions(-) diff --git a/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb b/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb index fc344c522..4e116a14a 100644 --- a/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb +++ b/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb @@ -1,48 +1,97 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Running an Inference on Custom Data" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "> **Note:** In this tutorial, we will be using a sample `RandLANet` `SemanticKITTI` weight file which we need to:\n", + ">\n", + "> 1. Download from: https://storage.googleapis.com/open3d-releases/model-zoo/randlanet_semantickitti_202201071330utc.pth\n", + "> 2. Place the downloaded `randlanet_semantickitti_202201071330utc.pth` file into `'Open3D-ML/docs/tutorial/notebook/'` subdirectory, or any other place and change the `ckpt_path` accordingly.\n", + ">\n", + "> For other model/dataset weight files, please check out https://github.com/isl-org/Open3D-ML#semantic-segmentation-1\n", + "\n", + "\n", + "An inference predicts the results based on the trained model. **Inference always depends on a prior step - _training_**.\n", + "\n", + "> **Please see the [Training a Semantic Segmentation Model Using PyTorch](train_ss_model_using_pytorch.ipynb) and [Training a Semantic Segmentation Model Using TensorFlow](train_ss_model_using_tensorflow.ipynb) for training tutorials.**\n", + "\n", + "To recap the model training process, - in our data model, we:\n", + "\n", + "1. define a dataset;\n", + "2. create a training split part of the data;\n", + "3. define a model;\n", + "4. create a `SemanticSegmentation` `pipeline` object;\n", + "5. execute the `pipeline.run_train()` method which runs the training process.\n", + "\n", + "While training, the model saves the checkpoint files every few epochs, in the *logs* directory. We use these trained weights to restore the model for inference.\n", + "\n", + "Our first step in Inference on a Custom Data implementation is to import *Open3D-ML* and *Numpy* libraries: \n" + ] + }, { "cell_type": "code", - "execution_count": 1, + "execution_count": 3, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Using external Open3D-ML in /Users/sanskara/Workspace/Open3D-ML\n" - ] - } - ], + "outputs": [], "source": [ "import open3d.ml.torch as ml3d # just switch to open3d.ml.tf for tf usage\n", "import numpy as np" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We then create a checkpoint path pointing to the data weights file we downloaded (generated at the end of the Training stage):" + ] + }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 9, "metadata": {}, "outputs": [], "source": [ - "# Download weights using link from model zoo(collection of weights for all combinations of model and dataset).\n", + "# Download weights using link from model zoo (collection of weights for all combinations of model and dataset).\n", "# Link : https://github.com/isl-org/Open3D-ML#semantic-segmentation-1\n", "# Randlanet SemanticKITTI[PyTorch] : https://storage.googleapis.com/open3d-releases/model-zoo/randlanet_semantickitti_202201071330utc.pth\n", "ckpt_path = './randlanet_semantickitti_202201071330utc.pth'" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, we define a `dataset` object identical to how it was done in our previous *Training a Semantic Segmentation Model* tutorials:" + ] + }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "# We define dataset (similar to train_ss_using_pytorch tutorial)\n", - "dataset = ml3d.datasets.SemanticKITTI(dataset_path='/Users/sanskara/data/SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n" + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then, initialize `model` and `pipeline` objects:" ] }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 11, "metadata": {}, "outputs": [], "source": [ @@ -51,16 +100,23 @@ "pipeline = ml3d.pipelines.SemanticSegmentation(model)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, we restore the model with our data weights file with `pipeline.load_ckpt()` method:" + ] + }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "INFO - 2022-04-01 19:58:03,220 - semantic_segmentation - Loading checkpoint ./randlanet_semantickitti_202201071330utc.pth\n" + "INFO - 2022-04-04 17:06:01,353 - semantic_segmentation - Loading checkpoint ./randlanet_semantickitti_202201071330utc.pth\n" ] } ], @@ -70,15 +126,24 @@ ] }, { - "cell_type": "code", - "execution_count": 23, + "cell_type": "markdown", "metadata": {}, + "source": [ + "Now, lets query the first pointcloud from the `test` split." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "scrolled": true + }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "INFO - 2022-04-01 20:12:49,159 - semantickitti - Found 1 pointclouds for test\n" + "INFO - 2022-04-04 17:13:43,378 - semantickitti - Found 1 pointclouds for test\n" ] } ], @@ -87,9 +152,16 @@ "data = test_data.get_data(0)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's display the content `data` contains:" + ] + }, { "cell_type": "code", - "execution_count": 26, + "execution_count": 23, "metadata": {}, "outputs": [ { @@ -107,70 +179,79 @@ } ], "source": [ - "# For inference on custom data, you can convert your pointcloud in this format.\n", - "# Dictionary with keys {'point', 'feat', 'label'}\n", - "# If you already have the ground truth labels, you can add it to data to get accuracy and IoU.\n", - "# otherwise pass labels as `None`.\n", - "print(data)\n" + "print(data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For inference on custom data, you can convert your pointcloud into this format:\n", + "\n", + "**Dictionary with keys {'point', 'feat', 'label'}**\n", + "\n", + "If you already have the *ground truth labels*, you can add them to data to get accuracy and IoU (Intersection over Union). Otherwise, pass labels as `None`.\n", + "\n", + "And now - the main topic of our tutorial - running an inference on the test data. You can call the `run_inference()` method with your data, it will print accuracy per class and Intersection over Union (IoU) metrics. The last entry in the list is *mean accuracy* and *mean IoU*:" ] }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 24, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "test 0/1: 100%|██████████| 83500/83500 [03:51<00:00, 360.18it/s] \n", - "test 0/1: 100%|██████████| 83500/83500 [00:15<00:00, 303.57it/s] INFO - 2022-04-01 20:09:24,024 - semantic_segmentation - Accuracy : [0.963756553453385, nan, nan, nan, nan, nan, nan, 0.3561643835616438, 0.9540482683792264, 0.9261289480553381, 0.4870199692780338, nan, 0.9735662148070907, 0.966321243523316, 0.825527363917982, 0.8268921095008052, 0.9399638336347197, 0.9401709401709402, 0.62, 0.81496331902354]\n", - "INFO - 2022-04-01 20:09:24,025 - semantic_segmentation - IoU : [0.9090518168135885, nan, 0.0, nan, 0.0, nan, 0.0, 0.3561643835616438, 0.9177174127669178, 0.21501727168050422, 0.47786276800180866, 0.0, 0.9568537022802972, 0.10855646100116415, 0.813112967671631, 0.7186843946815955, 0.6830486202365309, 0.843558282208589, 0.543859649122807, 0.4714679831266924]\n" + "test 0/1: 100%|██████████████████████████| 83500/83500 [09:03<00:00, 153.51it/s]\n", + "test 0/1: 100%|████████████████████████▉| 83258/83500 [00:12<00:00, 6721.37it/s]INFO - 2022-04-04 17:28:38,137 - semantic_segmentation - Accuracy : [0.972646455436517, nan, nan, nan, nan, nan, nan, 0.0, 0.9576552007272514, 0.9159488384233881, 0.4735023041474654, nan, 0.9099061522419186, 0.9974093264248705, 0.8342307124944682, 0.8268921095008052, 0.9240506329113924, 0.9487179487179487, 0.67, 0.7859133067521687]\n", + "INFO - 2022-04-04 17:28:38,140 - semantic_segmentation - IoU : [0.8774419082870656, nan, 0.0, nan, nan, nan, nan, 0.0, 0.900385993934381, 0.21923028864175934, 0.4655992749792312, 0.0, 0.8965375526559128, 0.08926501275214468, 0.8220736272122687, 0.7171787709497207, 0.6831550802139037, 0.8712715855572999, 0.5826086956521739, 0.5089105564882759]\n", + "test 0/1: 100%|█████████████████████████| 83500/83500 [00:30<00:00, 6721.37it/s]" ] } ], "source": [ "# Running inference on test data\n", "results = pipeline.run_inference(data)\n", - "# prints per class accuracy and IoU. Last entry is mean accuracy and mean IoU.\n", + "# prints per class accuracy and IoU (Intersection over Union). Last entry is mean accuracy and mean IoU.\n", "# We get several `nan` outputs for missing classes in the input data." ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `results` object will return dictionary of predicted labels and predicted probabilities per point:" + ] + }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'predict_labels': array([12, 12, 12, ..., 8, 8, 8], dtype=int16),\n", - " 'predict_scores': array([[2.1815e-05, 1.1027e-05, 7.1526e-07, ..., 1.4412e-04, 3.7403e-03,\n", - " 1.8473e-03],\n", - " [1.3769e-05, 5.7220e-06, 5.3644e-07, ..., 1.2094e-04, 4.3449e-03,\n", - " 2.5768e-03],\n", - " [8.1062e-06, 4.7088e-06, 3.5763e-07, ..., 1.0777e-04, 5.9471e-03,\n", - " 3.2253e-03],\n", + " 'predict_scores': array([[1.061e-05, 6.437e-06, 2.980e-07, ..., 5.573e-05, 1.382e-03,\n", + " 7.524e-04],\n", + " [6.795e-06, 7.331e-06, 2.980e-07, ..., 7.313e-05, 2.205e-03,\n", + " 8.492e-04],\n", + " [6.199e-06, 6.318e-06, 2.980e-07, ..., 7.361e-05, 3.147e-03,\n", + " 1.205e-03],\n", " ...,\n", - " [0.0000e+00, 0.0000e+00, 0.0000e+00, ..., 0.0000e+00, 0.0000e+00,\n", - " 0.0000e+00],\n", - " [0.0000e+00, 0.0000e+00, 0.0000e+00, ..., 0.0000e+00, 0.0000e+00,\n", - " 0.0000e+00],\n", - " [0.0000e+00, 0.0000e+00, 0.0000e+00, ..., 0.0000e+00, 0.0000e+00,\n", - " 0.0000e+00]], dtype=float16)}" + " [0.000e+00, 0.000e+00, 0.000e+00, ..., 0.000e+00, 0.000e+00,\n", + " 0.000e+00],\n", + " [0.000e+00, 0.000e+00, 0.000e+00, ..., 0.000e+00, 0.000e+00,\n", + " 0.000e+00],\n", + " [0.000e+00, 0.000e+00, 0.000e+00, ..., 0.000e+00, 0.000e+00,\n", + " 0.000e+00]], dtype=float16)}" ] }, - "execution_count": 21, + "execution_count": 25, "metadata": {}, "output_type": "execute_result" - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\r", - "test 0/1: 100%|██████████| 83500/83500 [00:32<00:00, 303.57it/s]" - ] } ], "source": [ @@ -188,7 +269,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -202,7 +283,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.7" + "version": "3.8.12" } }, "nbformat": 4, diff --git a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb index a2cda6052..c6fe9515b 100644 --- a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb +++ b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb @@ -2,6 +2,7 @@ "cells": [ { "cell_type": "markdown", + "id": "48589547", "metadata": {}, "source": [ "# Training a Semantic Segmentation Model Using PyTorch" @@ -9,6 +10,7 @@ }, { "cell_type": "markdown", + "id": "49b68041", "metadata": {}, "source": [ "In this tutorial, we will learn how to train a semantic segmentation model using PyTorch in a Jupyter Notebook. We assume that you are familiar with Jupyter Notebook and have created a folder notebooks in a folder that is relative to ml3d.\n", @@ -22,6 +24,7 @@ }, { "cell_type": "markdown", + "id": "2c10c2d2", "metadata": {}, "source": [ "At a high-level, we will:\n", @@ -34,6 +37,7 @@ }, { "cell_type": "markdown", + "id": "0f51bcdd", "metadata": {}, "source": [ "## Reading a dataset" @@ -41,6 +45,7 @@ }, { "cell_type": "markdown", + "id": "b69972ae", "metadata": {}, "source": [ "You can use any dataset available in the `ml3d.datasets` dataset namespace. For this example, we will use the `SemanticKITTI` dataset and visualize it. You can use any of the other dataset to load data. However, you must understand that the parameters may vary for each dataset.\n", @@ -51,18 +56,19 @@ { "cell_type": "code", "execution_count": null, + "id": "d8090925", "metadata": {}, "outputs": [], "source": [ - "#Training Semantic Segmentation Model using PyTorch\n", + "# Training Semantic Segmentation Model using PyTorch\n", "\n", - "#import torch\n", + "# import torch\n", "import open3d.ml.torch as ml3d\n", "\n", - "#Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", - "\n", + "# Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n", - "#Split the dataset for 'training'. You can get the other splits by passing 'validation' or 'test'\n", + "\n", + "# Split the dataset for 'training'. You can get the other splits by passing 'validation' or 'test'\n", "train_split = dataset.get_split('training')\n", "\n", "#support of Open3d-ML visualizer in Jupyter Notebooks is in progress\n", @@ -73,6 +79,7 @@ }, { "cell_type": "markdown", + "id": "b965946b", "metadata": {}, "source": [ "Now that you have visualized the dataset for training, let us train the model." @@ -80,6 +87,7 @@ }, { "cell_type": "markdown", + "id": "d077a5a2", "metadata": {}, "source": [ "## Training a model" @@ -87,6 +95,7 @@ }, { "cell_type": "markdown", + "id": "bb442bcd", "metadata": {}, "source": [ "Before you train a model, you must decide which model you want to use. For this example, we will use the `RandLANet` model. To use models, you must import the model from `open3d.ml.torch.models`.\n", @@ -97,6 +106,7 @@ { "cell_type": "code", "execution_count": 1, + "id": "989a6579", "metadata": {}, "outputs": [ { @@ -175,25 +185,28 @@ } ], "source": [ - "#Training Semantic Segmentation Model using PyTorch\n", + "# Training Semantic Segmentation Model using PyTorch\n", "\n", - "#Import torch and the model to use for training\n", + "# Import torch and the model to use for training\n", "import open3d.ml.torch as ml3d\n", "from open3d.ml.torch.models import RandLANet\n", "from open3d.ml.torch.pipelines import SemanticSegmentation\n", "\n", - "#Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", - "#dataset = ml3d.datasets.SemanticKITTI(dataset_path='/Users/sanskara/data/SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n", + "# Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", + "# dataset = ml3d.datasets.SemanticKITTI(dataset_path='/Users/sanskara/data/SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n", "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n", - "#Initialize the RandLANet model with three layers.\n", + "\n", + "# Initialize the RandLANet model with three layers.\n", "model = RandLANet(in_channels=3)\n", "pipeline = SemanticSegmentation(model=model, dataset=dataset, max_epoch=3, optimizer={'lr': 0.001}, num_workers=0)\n", - "#Run the training\n", + "\n", + "# Run the training\n", "pipeline.run_train()" ] }, { "cell_type": "markdown", + "id": "55c5bdba", "metadata": {}, "source": [ "## Running an Inference" @@ -201,6 +214,7 @@ }, { "cell_type": "markdown", + "id": "5404cae2", "metadata": {}, "source": [ "An inference processes point cloud and displays the results based on the trained model. For this example, we will use a trained `RandLANet` model.\n", @@ -211,6 +225,7 @@ { "cell_type": "code", "execution_count": 2, + "id": "2e6aad28", "metadata": {}, "outputs": [ { @@ -246,9 +261,9 @@ } ], "source": [ - "#Training Semantic Segmentation Model using PyTorch\n", + "# Training Semantic Segmentation Model using PyTorch\n", "\n", - "#Import torch and the model to use for training\n", + "# Import torch and the model to use for training\n", "import open3d.ml.torch as ml3d\n", "from open3d.ml.torch.models import RandLANet\n", "from open3d.ml.torch.pipelines import SemanticSegmentation\n", @@ -257,19 +272,20 @@ "model = RandLANet(in_channels=3)\n", "pipeline = SemanticSegmentation(model=model, dataset=dataset, max_epoch=3, optimizer={'lr': 0.001}, num_workers=0)\n", "\n", - "#Get data from the SemanticKITTI dataset using the \"train\" split\n", + "# Get data from the SemanticKITTI dataset using the \"train\" split\n", "train_split = dataset.get_split(\"test\")\n", "data = train_split.get_data(0)\n", "\n", - "#Run the inference\n", + "# Run the inference\n", "results = pipeline.run_inference(data)\n", "\n", - "#Print the results\n", + "# Print the results\n", "print(results)" ] }, { "cell_type": "markdown", + "id": "deb61bc0", "metadata": {}, "source": [ "## Running a Test" @@ -277,6 +293,7 @@ }, { "cell_type": "markdown", + "id": "ad40d3ea", "metadata": {}, "source": [ "Running a test is very similar to running an inference on Jupyter.\n", @@ -287,6 +304,7 @@ { "cell_type": "code", "execution_count": 3, + "id": "9269a2df", "metadata": {}, "outputs": [ { @@ -302,34 +320,35 @@ } ], "source": [ - "#Training Semantic Segmentation Model using PyTorch\n", + "# Training Semantic Segmentation Model using PyTorch\n", "\n", - "#Import torch and the model to use for training\n", + "# Import torch and the model to use for training\n", "import open3d.ml.torch as ml3d\n", "from open3d.ml.torch.models import RandLANet\n", "from open3d.ml.torch.pipelines import SemanticSegmentation\n", "\n", - "#Get pipeline, model, and dataset.\n", + "# Get pipeline, model, and dataset.\n", "Pipeline = get_module(\"pipeline\", \"SemanticSegmentation\", \"torch\")\n", "Model = get_module(\"model\", \"RandLANet\", \"torch\")\n", "Dataset = get_module(\"dataset\", \"SemanticKITTI\")\n", "\n", - "#Create a checkpoint\n", + "# Create a checkpoint\n", "RandLANet = Model(ckpt_path=args.path_ckpt_randlanet)\n", "SemanticKITTI = Dataset(args.path_semantickitti, use_cache=False)\n", "pipeline = Pipeline(model=RandLANet, dataset=SemanticKITTI)\n", "\n", - "#Get data from the SemanticKITTI dataset using the \"train\" split\n", + "# Get data from the SemanticKITTI dataset using the \"train\" split\n", "train_split = SemanticKITTI.get_split(\"train\")\n", "data = train_split.get_data(0)\n", "\n", - "#Run the test\n", + "# Run the test\n", "pipeline.run_test(data)" ] }, { "cell_type": "code", "execution_count": null, + "id": "b85be785", "metadata": {}, "outputs": [], "source": [] @@ -337,7 +356,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -351,7 +370,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.7" + "version": "3.8.12" } }, "nbformat": 4, diff --git a/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb b/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb index 724f63f6f..3f51a0f13 100644 --- a/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb +++ b/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb @@ -82,17 +82,18 @@ } ], "source": [ - "#Training Semantic Segmentation Model using TensorFlow\n", + "# Training Semantic Segmentation Model using TensorFlow\n", "\n", - "#import tensorflow\n", + "# import tensorflow\n", "import open3d.ml.tf as ml3d\n", "\n", - "#Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", + "# Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", "dataset = ml3d.datasets.SemanticKITTI(dataset_path='../datasets/', cache_dir='./logs/cache',training_split=['00', '01', '02', '03', '04', '05', '06', '07', '09', '10'])\n", - "#Split the dataset for 'training'. You can get the other splits by passing 'validation' or 'test'\n", + "\n", + "# Split the dataset for 'training'. You can get the other splits by passing 'validation' or 'test'\n", "train_split = dataset.get_split('training')\n", "\n", - "#view the first 1000 frames using the visualizer\n", + "# view the first 1000 frames using the visualizer\n", "MyVis = ml3d.vis.Visualizer()\n", "vis.visualize_dataset(dataset, 'training',indices=range(100))" ] @@ -182,19 +183,21 @@ } ], "source": [ - "#Training Semantic Segmentation Model using TensorFlow\n", + "# Training Semantic Segmentation Model using TensorFlow\n", "\n", - "#Import tensorflow and the model to use for training\n", + "# Import tensorflow and the model to use for training\n", "import open3d.ml.tf as ml3d\n", "from open3d.ml.tf.models import RandLANet\n", "from open3d.ml.tf.pipelines import SemanticSegmentation\n", "\n", - "#Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", + "# Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", "dataset = ml3d.datasets.SemanticKITTI(dataset_path='../datasets/', cache_dir='./logs/cache',training_split=['00', '01', '02', '03', '04', '05', '06', '07', '09', '10'])\n", - "#Initialize the RandLANet model with three layers.\n", + "\n", + "# Initialize the RandLANet model with three layers.\n", "model = RandLANet(dim_input=3)\n", "pipeline = SemanticSegmentation(model=model, dataset=dataset, max_epoch=100)\n", - "#Run the training\n", + "\n", + "# Run the training\n", "pipeline.run_train()" ] }, @@ -236,31 +239,31 @@ } ], "source": [ - "#Training Semantic Segmentation Model using TensorFlow\n", + "# Training Semantic Segmentation Model using TensorFlow\n", "\n", - "#Import tensorflow and the model to use for training\n", + "# Import tensorflow and the model to use for training\n", "import open3d.ml.tf as ml3d\n", "from open3d.ml.tf.models import RandLANet\n", "from open3d.ml.tf.pipelines import SemanticSegmentation\n", "\n", - "#Get pipeline, model, and dataset.\n", + "# Get pipeline, model, and dataset.\n", "Pipeline = get_module(\"pipeline\", \"SemanticSegmentation\", \"tf\")\n", "Model = get_module(\"model\", \"RandLANet\", \"tf\")\n", "Dataset = get_module(\"dataset\", \"SemanticKITTI\")\n", "\n", - "#Create a checkpoint\n", + "# Create a checkpoint\n", "RandLANet = Model(ckpt_path=args.path_ckpt_randlanet)\n", "SemanticKITTI = Dataset(args.path_semantickitti, use_cache=False)\n", "pipeline = Pipeline(model=RandLANet, dataset=SemanticKITTI)\n", "\n", - "#Get data from the SemanticKITTI dataset using the \"train\" split\n", + "# Get data from the SemanticKITTI dataset using the \"train\" split\n", "train_split = SemanticKITTI.get_split(\"train\")\n", "data = train_split.get_data(0)\n", "\n", - "#Run the inference\n", + "# Run the inference\n", "results = pipeline.run_inference(data)\n", "\n", - "#Print the results\n", + "# Print the results\n", "print(results)" ] }, @@ -302,28 +305,28 @@ } ], "source": [ - "#Training Semantic Segmentation Model using TensorFlow\n", + "# Training Semantic Segmentation Model using TensorFlow\n", "\n", - "#Import tensorflow and the model to use for training\n", + "# Import tensorflow and the model to use for training\n", "import open3d.ml.tf as ml3d\n", "from open3d.ml.tf.models import RandLANet\n", "from open3d.ml.tf.pipelines import SemanticSegmentation\n", "\n", - "#Get pipeline, model, and dataset.\n", + "# Get pipeline, model, and dataset.\n", "Pipeline = get_module(\"pipeline\", \"SemanticSegmentation\", \"tf\")\n", "Model = get_module(\"model\", \"RandLANet\", \"tf\")\n", "Dataset = get_module(\"dataset\", \"SemanticKITTI\")\n", "\n", - "#Create a checkpoint\n", + "# Create a checkpoint\n", "RandLANet = Model(ckpt_path=args.path_ckpt_randlanet)\n", "SemanticKITTI = Dataset(args.path_semantickitti, use_cache=False)\n", "pipeline = Pipeline(model=RandLANet, dataset=SemanticKITTI)\n", "\n", - "#Get data from the SemanticKITTI dataset using the \"train\" split\n", + "# Get data from the SemanticKITTI dataset using the \"train\" split\n", "train_split = SemanticKITTI.get_split(\"train\")\n", "data = train_split.get_data(0)\n", "\n", - "#Run the test\n", + "# Run the test\n", "pipeline.run_test(data)" ] } diff --git a/docs/tutorial/notebook/train_ss_model_using_tensorflow.rst b/docs/tutorial/notebook/train_ss_model_using_tensorflow.rst index 8cd249dc3..2c7b91082 100644 --- a/docs/tutorial/notebook/train_ss_model_using_tensorflow.rst +++ b/docs/tutorial/notebook/train_ss_model_using_tensorflow.rst @@ -1,7 +1,7 @@ .. _train_ss_model_using_tensorflow: Train a Semantic Segmentation Model Using TensorFlow -------------------------------------------------- +---------------------------------------------------- In this tutorial, we will learn how to train a semantic segmentation model using TensorFlow in a Jupyter Notebook. We assume that you are familiar with Jupyter Notebook and have created a folder `notebooks` in a folder that is relative to `ml3d`. Before you begin, ensure that you have TensorFlow installed. To install a compatible version of TensorFlow, use the requirement file: From ff5a38695d905745fb993d3cf14f8dbcb77c9c61 Mon Sep 17 00:00:00 2001 From: AlexZakIntel Date: Thu, 7 Apr 2022 10:37:00 -0400 Subject: [PATCH 09/13] 4-7-2022 - Code bug removals and content updates * Fixed code errors in train_ss_model_using_pytorch.ipynb * Fixed code errors in train_ss_model_using_tensorflow.ipynb * Updated content in all tutorials --- .../notebook/Inference_on_a_custom_data.ipynb | 94 ++++-- .../notebook/reading_a_config_file.ipynb | 44 +-- .../tutorial/notebook/reading_a_dataset.ipynb | 65 ++++- .../train_ss_model_using_pytorch.ipynb | 247 +++++++--------- .../train_ss_model_using_tensorflow.ipynb | 271 ++++++++++-------- 5 files changed, 394 insertions(+), 327 deletions(-) diff --git a/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb b/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb index 4e116a14a..34e9257a6 100644 --- a/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb +++ b/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb @@ -38,9 +38,48 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 1, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Jupyter environment detected. Enabling Open3D WebVisualizer.\n", + "[Open3D INFO] WebRTC GUI backend enabled.\n", + "[Open3D INFO] WebRTCWindowSystem: HTTP handshake server disabled.\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\n", + " Using the Open3D PyTorch ops with CUDA 11 may have stability issues!\n", + "\n", + " We recommend to compile PyTorch from source with compile flags\n", + " '-Xcompiler -fno-gnu-unique'\n", + "\n", + " or use the PyTorch wheels at\n", + " https://github.com/isl-org/open3d_downloads/releases/tag/torch1.8.2\n", + "\n", + "\n", + " Ignore this message if PyTorch has been compiled with the aforementioned\n", + " flags.\n", + "\n", + " See https://github.com/isl-org/Open3D/issues/3324 and\n", + " https://github.com/pytorch/pytorch/issues/52663 for more information on this\n", + " problem.\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2022-04-06 15:53:47.653694: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", + "2022-04-06 15:53:47.653724: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n" + ] + } + ], "source": [ "import open3d.ml.torch as ml3d # just switch to open3d.ml.tf for tf usage\n", "import numpy as np" @@ -55,7 +94,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ @@ -74,7 +113,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 3, "metadata": {}, "outputs": [], "source": [ @@ -91,7 +130,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 4, "metadata": {}, "outputs": [], "source": [ @@ -109,14 +148,14 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "INFO - 2022-04-04 17:06:01,353 - semantic_segmentation - Loading checkpoint ./randlanet_semantickitti_202201071330utc.pth\n" + "INFO - 2022-04-06 15:54:37,266 - semantic_segmentation - Loading checkpoint ./randlanet_semantickitti_202201071330utc.pth\n" ] } ], @@ -134,7 +173,7 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 6, "metadata": { "scrolled": true }, @@ -143,7 +182,7 @@ "name": "stderr", "output_type": "stream", "text": [ - "INFO - 2022-04-04 17:13:43,378 - semantickitti - Found 1 pointclouds for test\n" + "INFO - 2022-04-06 15:54:40,026 - semantickitti - Found 1 pointclouds for test\n" ] } ], @@ -161,7 +200,7 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 7, "metadata": {}, "outputs": [ { @@ -192,22 +231,25 @@ "\n", "If you already have the *ground truth labels*, you can add them to data to get accuracy and IoU (Intersection over Union). Otherwise, pass labels as `None`.\n", "\n", - "And now - the main topic of our tutorial - running an inference on the test data. You can call the `run_inference()` method with your data, it will print accuracy per class and Intersection over Union (IoU) metrics. The last entry in the list is *mean accuracy* and *mean IoU*:" + "And now - the main topic of our tutorial - running an inference on the test data. You can call the `run_inference()` method with your data, - it will print accuracy per class and Intersection over Union (IoU) metrics. The last entry in the list is *mean accuracy* and *mean IoU*:" ] }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "test 0/1: 100%|██████████████████████████| 83500/83500 [09:03<00:00, 153.51it/s]\n", - "test 0/1: 100%|████████████████████████▉| 83258/83500 [00:12<00:00, 6721.37it/s]INFO - 2022-04-04 17:28:38,137 - semantic_segmentation - Accuracy : [0.972646455436517, nan, nan, nan, nan, nan, nan, 0.0, 0.9576552007272514, 0.9159488384233881, 0.4735023041474654, nan, 0.9099061522419186, 0.9974093264248705, 0.8342307124944682, 0.8268921095008052, 0.9240506329113924, 0.9487179487179487, 0.67, 0.7859133067521687]\n", - "INFO - 2022-04-04 17:28:38,140 - semantic_segmentation - IoU : [0.8774419082870656, nan, 0.0, nan, nan, nan, nan, 0.0, 0.900385993934381, 0.21923028864175934, 0.4655992749792312, 0.0, 0.8965375526559128, 0.08926501275214468, 0.8220736272122687, 0.7171787709497207, 0.6831550802139037, 0.8712715855572999, 0.5826086956521739, 0.5089105564882759]\n", - "test 0/1: 100%|█████████████████████████| 83500/83500 [00:30<00:00, 6721.37it/s]" + "\n", + "test 0/1: 100%|█████████████████████████| 83500/83500 [01:14<00:00, 1115.74it/s]\u001b[A\n", + "\n", + "test 0/1: 100%|████████████████████████▉| 83262/83500 [00:09<00:00, 9105.02it/s]\u001b[AINFO - 2022-04-06 15:57:33,626 - semantic_segmentation - Accuracy : [0.9721905630271256, nan, nan, nan, nan, nan, nan, 0.0, 0.9554558517345533, 0.9201252936570086, 0.47223502304147463, nan, 0.9598018769551616, 0.9974093264248705, 0.8371072429561882, 0.8510466988727858, 0.9349005424954792, 0.9367521367521368, 0.67, 0.7922520463263987]\n", + "INFO - 2022-04-06 15:57:33,627 - semantic_segmentation - IoU : [0.9032189750105888, nan, 0.0, nan, 0.0, nan, nan, 0.0, 0.9118182072593961, 0.21114106019766396, 0.464405755504362, 0.0, 0.9425046078230596, 0.11663132384126022, 0.8259888650340235, 0.753922967189729, 0.6790123456790124, 0.8404907975460123, 0.5775862068965517, 0.481781407465444]\n", + "\n", + "test 0/1: 100%|█████████████████████████| 83500/83500 [00:25<00:00, 9105.02it/s]\u001b[A" ] } ], @@ -227,29 +269,29 @@ }, { "cell_type": "code", - "execution_count": 25, + "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'predict_labels': array([12, 12, 12, ..., 8, 8, 8], dtype=int16),\n", - " 'predict_scores': array([[1.061e-05, 6.437e-06, 2.980e-07, ..., 5.573e-05, 1.382e-03,\n", - " 7.524e-04],\n", - " [6.795e-06, 7.331e-06, 2.980e-07, ..., 7.313e-05, 2.205e-03,\n", - " 8.492e-04],\n", - " [6.199e-06, 6.318e-06, 2.980e-07, ..., 7.361e-05, 3.147e-03,\n", - " 1.205e-03],\n", + " 'predict_scores': array([[2.044e-05, 1.186e-05, 8.345e-07, ..., 1.608e-04, 2.861e-03,\n", + " 1.152e-03],\n", + " [1.615e-05, 1.079e-05, 8.345e-07, ..., 2.084e-04, 5.005e-03,\n", + " 2.203e-03],\n", + " [1.317e-05, 9.954e-06, 6.557e-07, ..., 1.574e-04, 6.054e-03,\n", + " 2.481e-03],\n", " ...,\n", " [0.000e+00, 0.000e+00, 0.000e+00, ..., 0.000e+00, 0.000e+00,\n", " 0.000e+00],\n", - " [0.000e+00, 0.000e+00, 0.000e+00, ..., 0.000e+00, 0.000e+00,\n", + " [5.960e-08, 0.000e+00, 0.000e+00, ..., 0.000e+00, 0.000e+00,\n", " 0.000e+00],\n", - " [0.000e+00, 0.000e+00, 0.000e+00, ..., 0.000e+00, 0.000e+00,\n", + " [5.960e-08, 0.000e+00, 0.000e+00, ..., 0.000e+00, 0.000e+00,\n", " 0.000e+00]], dtype=float16)}" ] }, - "execution_count": 25, + "execution_count": 10, "metadata": {}, "output_type": "execute_result" } diff --git a/docs/tutorial/notebook/reading_a_config_file.ipynb b/docs/tutorial/notebook/reading_a_config_file.ipynb index d4263c94e..8efd66efd 100644 --- a/docs/tutorial/notebook/reading_a_config_file.ipynb +++ b/docs/tutorial/notebook/reading_a_config_file.ipynb @@ -77,8 +77,8 @@ "name": "stderr", "output_type": "stream", "text": [ - "2022-03-21 11:19:34.531469: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", - "2022-03-21 11:19:34.531795: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n" + "2022-04-06 16:18:39.509582: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", + "2022-04-06 16:18:39.509610: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n" ] } ], @@ -129,7 +129,7 @@ { "data": { "text/plain": [ - "" + "" ] }, "execution_count": 3, @@ -249,7 +249,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 5, "metadata": {}, "outputs": [ { @@ -258,7 +258,7 @@ "'SemanticKITTI'" ] }, - "execution_count": 6, + "execution_count": 5, "metadata": {}, "output_type": "execute_result" } @@ -278,7 +278,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 6, "metadata": { "scrolled": true }, @@ -302,7 +302,7 @@ " 'augment': {'recenter': {'dim': [0, 1]}}}" ] }, - "execution_count": 8, + "execution_count": 6, "metadata": {}, "output_type": "execute_result" } @@ -320,7 +320,7 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 7, "metadata": {}, "outputs": [ { @@ -329,7 +329,7 @@ "[4, 4, 4, 4]" ] }, - "execution_count": 13, + "execution_count": 7, "metadata": {}, "output_type": "execute_result" } @@ -347,7 +347,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 8, "metadata": {}, "outputs": [ { @@ -369,7 +369,7 @@ " 'max_outputs': 1}}" ] }, - "execution_count": 11, + "execution_count": 8, "metadata": {}, "output_type": "execute_result" } @@ -387,7 +387,7 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 9, "metadata": {}, "outputs": [ { @@ -396,7 +396,7 @@ "'SemanticSegmentation'" ] }, - "execution_count": 16, + "execution_count": 9, "metadata": {}, "output_type": "execute_result" } @@ -416,7 +416,7 @@ }, { "cell_type": "code", - "execution_count": 17, + "execution_count": 10, "metadata": {}, "outputs": [], "source": [ @@ -432,15 +432,15 @@ }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "{'cfg': ,\n", + "{'cfg': ,\n", " 'name': 'SemanticKITTI',\n", - " 'rng': Generator(PCG64) at 0x7F75F536DE50,\n", + " 'rng': Generator(PCG64) at 0x7F42605AFA90,\n", " 'label_to_names': {0: 'unlabeled',\n", " 1: 'car',\n", " 2: 'bicycle',\n", @@ -494,7 +494,7 @@ " dtype=int32)}" ] }, - "execution_count": 18, + "execution_count": 11, "metadata": {}, "output_type": "execute_result" } @@ -512,7 +512,7 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 12, "metadata": {}, "outputs": [ { @@ -521,7 +521,7 @@ "20" ] }, - "execution_count": 19, + "execution_count": 12, "metadata": {}, "output_type": "execute_result" } @@ -539,7 +539,7 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 13, "metadata": {}, "outputs": [ { @@ -567,7 +567,7 @@ " 19: 'traffic-sign'}" ] }, - "execution_count": 20, + "execution_count": 13, "metadata": {}, "output_type": "execute_result" } diff --git a/docs/tutorial/notebook/reading_a_dataset.ipynb b/docs/tutorial/notebook/reading_a_dataset.ipynb index 0ac5ea519..ccbe3a907 100644 --- a/docs/tutorial/notebook/reading_a_dataset.ipynb +++ b/docs/tutorial/notebook/reading_a_dataset.ipynb @@ -34,9 +34,48 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 1, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Jupyter environment detected. Enabling Open3D WebVisualizer.\n", + "[Open3D INFO] WebRTC GUI backend enabled.\n", + "[Open3D INFO] WebRTCWindowSystem: HTTP handshake server disabled.\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\n", + " Using the Open3D PyTorch ops with CUDA 11 may have stability issues!\n", + "\n", + " We recommend to compile PyTorch from source with compile flags\n", + " '-Xcompiler -fno-gnu-unique'\n", + "\n", + " or use the PyTorch wheels at\n", + " https://github.com/isl-org/open3d_downloads/releases/tag/torch1.8.2\n", + "\n", + "\n", + " Ignore this message if PyTorch has been compiled with the aforementioned\n", + " flags.\n", + "\n", + " See https://github.com/isl-org/Open3D/issues/3324 and\n", + " https://github.com/pytorch/pytorch/issues/52663 for more information on this\n", + " problem.\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2022-04-06 16:20:21.687818: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", + "2022-04-06 16:20:21.687846: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n" + ] + } + ], "source": [ "#import torch\n", "import open3d.ml.torch as ml3d" @@ -51,7 +90,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ @@ -90,7 +129,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 3, "metadata": { "scrolled": true }, @@ -99,7 +138,7 @@ "name": "stderr", "output_type": "stream", "text": [ - "INFO - 2022-03-29 10:31:54,916 - semantickitti - Found 1 pointclouds for training\n" + "INFO - 2022-04-06 16:22:26,972 - semantickitti - Found 1 pointclouds for training\n" ] } ], @@ -117,14 +156,14 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "INFO - 2022-03-29 10:31:54,926 - semantickitti - Found 1 pointclouds for validation\n" + "INFO - 2022-04-06 16:22:31,932 - semantickitti - Found 1 pointclouds for validation\n" ] } ], @@ -142,14 +181,14 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "INFO - 2022-03-29 10:31:54,939 - semantickitti - Found 1 pointclouds for test\n" + "INFO - 2022-04-06 16:22:33,834 - semantickitti - Found 1 pointclouds for test\n" ] } ], @@ -169,7 +208,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 6, "metadata": {}, "outputs": [ { @@ -205,7 +244,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 7, "metadata": {}, "outputs": [ { @@ -239,7 +278,7 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 8, "metadata": {}, "outputs": [ { @@ -274,7 +313,7 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 9, "metadata": {}, "outputs": [ { diff --git a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb index c6fe9515b..7e6e0b9da 100644 --- a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb +++ b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb @@ -55,10 +55,18 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 5, "id": "d8090925", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO - 2022-04-07 10:01:51,953 - semantickitti - Found 1 pointclouds for training\n" + ] + } + ], "source": [ "# Training Semantic Segmentation Model using PyTorch\n", "\n", @@ -105,82 +113,50 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": 6, "id": "989a6579", "metadata": {}, "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Jupyter environment detected. Enabling Open3D WebVisualizer.\n", - "[Open3D INFO] WebRTC GUI backend enabled.\n", - "[Open3D INFO] WebRTCWindowSystem: HTTP handshake server disabled.\n", - "\n", - "--------------------------------------------------------------------------------\n", - "\n", - " Using the Open3D PyTorch ops with CUDA 11 may have stability issues!\n", - "\n", - " We recommend to compile PyTorch from source with compile flags\n", - " '-Xcompiler -fno-gnu-unique'\n", - "\n", - " or use the PyTorch wheels at\n", - " https://github.com/isl-org/open3d_downloads/releases/tag/torch1.8.2\n", - "\n", - "\n", - " Ignore this message if PyTorch has been compiled with the aforementioned\n", - " flags.\n", - "\n", - " See https://github.com/isl-org/Open3D/issues/3324 and\n", - " https://github.com/pytorch/pytorch/issues/52663 for more information on this\n", - " problem.\n", - "\n", - "--------------------------------------------------------------------------------\n", - "\n" - ] - }, { "name": "stderr", "output_type": "stream", "text": [ - "2022-02-28 12:25:57.683214: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", - "2022-02-28 12:25:57.683243: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n", - "INFO - 2022-02-28 12:25:59,912 - semantic_segmentation - DEVICE : cpu\n", - "INFO - 2022-02-28 12:25:59,913 - semantic_segmentation - Logging in file : ./logs/RandLANet_SemanticKITTI_torch/log_train_2022-02-28_12:25:59.txt\n", - "INFO - 2022-02-28 12:25:59,914 - semantickitti - Found 1 pointclouds for train\n", - "INFO - 2022-02-28 12:25:59,915 - semantickitti - Found 1 pointclouds for validation\n", - "INFO - 2022-02-28 12:25:59,917 - semantic_segmentation - ckpt_path not given. Restore from the latest ckpt\n", - "INFO - 2022-02-28 12:25:59,917 - semantic_segmentation - Loading checkpoint ./logs/RandLANet_SemanticKITTI_torch/checkpoint/ckpt_00000.pth\n", - "INFO - 2022-02-28 12:25:59,979 - semantic_segmentation - Loading checkpoint optimizer_state_dict\n", - "INFO - 2022-02-28 12:26:00,004 - semantic_segmentation - Loading checkpoint scheduler_state_dict\n", - "INFO - 2022-02-28 12:26:00,010 - semantic_segmentation - Writing summary in train_log/00003_RandLANet_SemanticKITTI_torch.\n", - "INFO - 2022-02-28 12:26:00,011 - semantic_segmentation - Started training\n", - "INFO - 2022-02-28 12:26:00,013 - semantic_segmentation - === EPOCH 0/3 ===\n", - "training: 0%| | 0/1 [00:00\u001b[0;34m\u001b[0m\n\u001b[1;32m 7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[0;31m#Get pipeline, model, and dataset.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 9\u001b[0;31m \u001b[0mPipeline\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mget_module\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"pipeline\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"SemanticSegmentation\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"torch\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 10\u001b[0m \u001b[0mModel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mget_module\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"model\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"RandLANet\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"torch\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 11\u001b[0m \u001b[0mDataset\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mget_module\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"dataset\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"SemanticKITTI\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", - "\u001b[0;31mNameError\u001b[0m: name 'get_module' is not defined" - ] - } - ], - "source": [ - "# Training Semantic Segmentation Model using PyTorch\n", - "\n", - "# Import torch and the model to use for training\n", - "import open3d.ml.torch as ml3d\n", - "from open3d.ml.torch.models import RandLANet\n", - "from open3d.ml.torch.pipelines import SemanticSegmentation\n", - "\n", - "# Get pipeline, model, and dataset.\n", - "Pipeline = get_module(\"pipeline\", \"SemanticSegmentation\", \"torch\")\n", - "Model = get_module(\"model\", \"RandLANet\", \"torch\")\n", - "Dataset = get_module(\"dataset\", \"SemanticKITTI\")\n", - "\n", - "# Create a checkpoint\n", - "RandLANet = Model(ckpt_path=args.path_ckpt_randlanet)\n", - "SemanticKITTI = Dataset(args.path_semantickitti, use_cache=False)\n", - "pipeline = Pipeline(model=RandLANet, dataset=SemanticKITTI)\n", - "\n", - "# Get data from the SemanticKITTI dataset using the \"train\" split\n", - "train_split = SemanticKITTI.get_split(\"train\")\n", - "data = train_split.get_data(0)\n", - "\n", - "# Run the test\n", - "pipeline.run_test(data)" - ] - }, { "cell_type": "code", "execution_count": null, diff --git a/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb b/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb index 3f51a0f13..09a6871c1 100644 --- a/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb +++ b/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb @@ -19,20 +19,14 @@ "\n", "```sh\n", "pip install -r requirements-tensorflow.txt\n", - "```" - ] - }, - { - "cell_type": "markdown", - "id": "ec60b6a0", - "metadata": {}, - "source": [ + "```\n", + "\n", "At a high-level, we will:\n", "\n", "- Read a dataset and get a training split. For this example, we will use `SemanticKITTI` dataset.\n", "- Run a pre-trained model. For this example, we will use the `RandLANet` model.\n", "- Train a model. We will train a model using the `SemanticKITTI` dataset and `RandLANet` model.\n", - "- Run an inference and run a test. We will run an inference using the 'training' split that use a pointcloud and display a result. However, a test is run on a pre-defined test set rather than a pass pointcloud.\n" + "- Run an inference and run a test. We will run an inference using the 'training' split that use a pointcloud and display a result. However, a test is run on a pre-defined test set rather than a pass pointcloud." ] }, { @@ -55,7 +49,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 1, "id": "b4e344db", "metadata": {}, "outputs": [ @@ -69,15 +63,14 @@ ] }, { - "ename": "ModuleNotFoundError", - "evalue": "No module named 'tensorflow'", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)", - "\u001b[0;32m/tmp/ipykernel_9847/1245153815.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;31m#import tensorflow\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mml3d\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0;31m#Read a dataset by specifying the path. We are also providing the cache directory and training split.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", - "\u001b[0;32m~/miniconda3/envs/open3d/lib/python3.8/site-packages/open3d/ml/tf/__init__.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 27\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 28\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mos\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_os\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 29\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mtensorflow\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m__version__\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_tf_version\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 30\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m_build_config\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 31\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", - "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'tensorflow'" + "name": "stderr", + "output_type": "stream", + "text": [ + "2022-04-07 10:20:31.517091: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", + "2022-04-07 10:20:31.517124: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n", + "2022-04-07 10:20:33.389558: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", + "2022-04-07 10:20:33.389589: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n", + "INFO - 2022-04-07 10:20:34,543 - semantickitti - Found 1 pointclouds for training\n" ] } ], @@ -88,14 +81,14 @@ "import open3d.ml.tf as ml3d\n", "\n", "# Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", - "dataset = ml3d.datasets.SemanticKITTI(dataset_path='../datasets/', cache_dir='./logs/cache',training_split=['00', '01', '02', '03', '04', '05', '06', '07', '09', '10'])\n", + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='./SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'])\n", "\n", "# Split the dataset for 'training'. You can get the other splits by passing 'validation' or 'test'\n", "train_split = dataset.get_split('training')\n", "\n", "# view the first 1000 frames using the visualizer\n", - "MyVis = ml3d.vis.Visualizer()\n", - "vis.visualize_dataset(dataset, 'training',indices=range(100))" + "# MyVis = ml3d.vis.Visualizer()\n", + "# MyVis.visualize_dataset(dataset, 'training',indices=range(1))" ] }, { @@ -124,7 +117,7 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": 2, "id": "c220cfb6", "metadata": {}, "outputs": [ @@ -132,11 +125,9 @@ "name": "stderr", "output_type": "stream", "text": [ - "2022-02-24 12:27:34.379346: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", - "2022-02-24 12:27:34.379376: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n", - "2022-02-24 12:27:35.764206: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory\n", - "2022-02-24 12:27:35.764236: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)\n", - "2022-02-24 12:27:35.764259: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (intel-VirtualBox): /proc/driver/nvidia/version does not exist\n" + "2022-04-06 16:31:11.326658: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory\n", + "2022-04-06 16:31:11.326709: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)\n", + "2022-04-06 16:31:11.326750: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (intel-VirtualBox): /proc/driver/nvidia/version does not exist\n" ] } ], @@ -165,20 +156,53 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 6, "id": "caeea522", "metadata": {}, "outputs": [ { - "ename": "ModuleNotFoundError", - "evalue": "No module named 'tensorflow'", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)", - "\u001b[0;32m/tmp/ipykernel_9847/122240413.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;31m#Import tensorflow and the model to use for training\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mml3d\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 5\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmodels\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mRandLANet\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpipelines\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mSemanticSegmentation\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", - "\u001b[0;32m~/miniconda3/envs/open3d/lib/python3.8/site-packages/open3d/ml/tf/__init__.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 27\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 28\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mos\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_os\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 29\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mtensorflow\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m__version__\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_tf_version\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 30\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m_build_config\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 31\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", - "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'tensorflow'" + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO - 2022-04-07 10:25:19,140 - semantic_segmentation - \n", + "INFO - 2022-04-07 10:25:19,142 - semantic_segmentation - Logging in file : ./logs/RandLANet_SemanticKITTI_tf/log_train_2022-04-07_10:25:19.txt\n", + "INFO - 2022-04-07 10:25:19,144 - semantickitti - Found 1 pointclouds for training\n", + "INFO - 2022-04-07 10:25:19,308 - semantickitti - Found 1 pointclouds for validation\n", + "INFO - 2022-04-07 10:25:19,441 - semantic_segmentation - Writing summary in train_log/00009_RandLANet_SemanticKITTI_tf.\n", + "INFO - 2022-04-07 10:25:19,444 - semantic_segmentation - Initializing from scratch.\n", + "INFO - 2022-04-07 10:25:19,445 - semantic_segmentation - === EPOCH 0/3 ===\n", + "training: 0%| | 0/1 [00:00\u001b[0;34m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;31m#Import tensorflow and the model to use for training\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mml3d\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 5\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmodels\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mRandLANet\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpipelines\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mSemanticSegmentation\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", - "\u001b[0;32m~/miniconda3/envs/open3d/lib/python3.8/site-packages/open3d/ml/tf/__init__.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 27\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 28\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mos\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_os\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 29\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mtensorflow\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m__version__\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_tf_version\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 30\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m_build_config\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 31\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", - "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'tensorflow'" + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO - 2022-04-07 10:27:15,640 - semantic_segmentation - Restored from ./logs/RandLANet_SemanticKITTI_tf/checkpoint/ckpt-1\n", + "INFO - 2022-04-07 10:27:15,642 - semantic_segmentation - Logging in file : ./logs/RandLANet_SemanticKITTI_tf/log_test_2022-04-07_10:27:15.txt\n", + "INFO - 2022-04-07 10:27:15,651 - semantic_segmentation - Started testing\n", + "INFO - 2022-04-07 10:27:15,653 - semantickitti - Found 1 pointclouds for test\n", + "test: 0%| | 0/1 [00:00\u001b[0;34m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;31m#Import tensorflow and the model to use for training\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mml3d\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 5\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmodels\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mRandLANet\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mml\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpipelines\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mSemanticSegmentation\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", - "\u001b[0;32m~/miniconda3/envs/open3d/lib/python3.8/site-packages/open3d/ml/tf/__init__.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 27\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 28\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mos\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_os\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 29\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mtensorflow\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m__version__\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0m_tf_version\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 30\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mopen3d\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m_build_config\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 31\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", - "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'tensorflow'" + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO - 2022-04-07 10:29:05,675 - semantickitti - Found 1 pointclouds for test\n", + "INFO - 2022-04-07 10:29:05,678 - semantic_segmentation - running inference\n", + "100%|███████████████████████████████████| 83500/83500 [00:09<00:00, 9063.39it/s]\n", + "/home/intel/miniconda3/envs/open3d/lib/python3.8/site-packages/open3d/_ml3d/tf/modules/metrics/semseg_metric.py:119: UserWarning: Prediction has fewer classes than ground truth. This may affect accuracy.\n", + " warnings.warn(\n", + "INFO - 2022-04-07 10:29:15,823 - semantic_segmentation - Accuracy : [0.0, nan, nan, nan, nan, nan, nan, 0.0, 0.0, 0.0, 0.0, nan, 0.10099061522419187, 0.0, 0.0, 0.0, 0.0, 0.0, 0.95, 0.08758255126868265]\n", + "INFO - 2022-04-07 10:29:15,825 - semantic_segmentation - IoU : [0.0, nan, nan, nan, nan, nan, nan, 0.0, 0.0, 0.0, 0.0, nan, 0.04242968544642075, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0012920775246514791, 0.0036434802475893524]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'predict_labels': array([14, 14, 14, ..., 8, 8, 8]), 'predict_scores': array([[0.004246 , 0.000257 , 0.000623 , ..., 0.0004656, 0.0004363,\n", + " 0.002386 ],\n", + " [0.00435 , 0.0002494, 0.0006337, ..., 0.0004673, 0.000422 ,\n", + " 0.002426 ],\n", + " [0.004383 , 0.0002475, 0.0006332, ..., 0.000465 , 0.0004222,\n", + " 0.00244 ],\n", + " ...,\n", + " [0.02219 , 0.011086 , 0.01326 , ..., 0.01465 , 0.01413 ,\n", + " 0.01793 ],\n", + " [0.02217 , 0.01109 , 0.01327 , ..., 0.01464 , 0.01414 ,\n", + " 0.01793 ],\n", + " [0.02217 , 0.01109 , 0.01327 , ..., 0.01464 , 0.01414 ,\n", + " 0.01793 ]], dtype=float16)}\n" ] } ], "source": [ - "# Training Semantic Segmentation Model using TensorFlow\n", - "\n", - "# Import tensorflow and the model to use for training\n", - "import open3d.ml.tf as ml3d\n", - "from open3d.ml.tf.models import RandLANet\n", - "from open3d.ml.tf.pipelines import SemanticSegmentation\n", - "\n", - "# Get pipeline, model, and dataset.\n", - "Pipeline = get_module(\"pipeline\", \"SemanticSegmentation\", \"tf\")\n", - "Model = get_module(\"model\", \"RandLANet\", \"tf\")\n", - "Dataset = get_module(\"dataset\", \"SemanticKITTI\")\n", - "\n", - "# Create a checkpoint\n", - "RandLANet = Model(ckpt_path=args.path_ckpt_randlanet)\n", - "SemanticKITTI = Dataset(args.path_semantickitti, use_cache=False)\n", - "pipeline = Pipeline(model=RandLANet, dataset=SemanticKITTI)\n", - "\n", - "# Get data from the SemanticKITTI dataset using the \"train\" split\n", - "train_split = SemanticKITTI.get_split(\"train\")\n", + "# Get data from the SemanticKITTI dataset using the \"test\" split\n", + "train_split = dataset.get_split(\"test\")\n", "data = train_split.get_data(0)\n", "\n", - "# Run the test\n", - "pipeline.run_test(data)" + "# Run the inference\n", + "results = pipeline.run_inference(data)\n", + "\n", + "# Print the results\n", + "print(results)" ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bcf019b3", + "metadata": {}, + "outputs": [], + "source": [] } ], "metadata": { From 8d1a9ef9c5f6cb68c39130a59cb88bb6868da4e2 Mon Sep 17 00:00:00 2001 From: AlexZakIntel Date: Tue, 12 Apr 2022 10:59:16 -0400 Subject: [PATCH 10/13] 4-12-2022 - Address PR Reviews --- .../notebook/Inference_on_a_custom_data.ipynb | 195 ++----- .../notebook/reading_a_config_file.ipynb | 534 ++++++++---------- .../tutorial/notebook/reading_a_dataset.ipynb | 173 ++---- .../train_ss_model_using_pytorch.ipynb | 190 ++----- .../train_ss_model_using_tensorflow.ipynb | 217 +------ 5 files changed, 403 insertions(+), 906 deletions(-) diff --git a/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb b/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb index 34e9257a6..19f0b34b2 100644 --- a/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb +++ b/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb @@ -4,16 +4,32 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Running an Inference on Custom Data" + "# Running Semantic Segmentation inference on custom data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "> **Note:** In this tutorial, we will be using a sample `RandLANet` `SemanticKITTI` weight file which we need to:\n", + "In this tutorial, we will cover how to run an inference on a pointcloud data in Open3D-ML. To accomplish that, we will take these steps:\n", + "\n", + "1. Download the data *weights* file;\n", + "2. Set up `torch` and `numpy` libraries;\n", + "3. Create a variable holding the path to the *weights* file;\n", + "4. Create a `dataset` object and extract a sample from its `'test'` split;\n", + "5. Create and initialize `model` and `pipeline` objects;\n", + "6. Restore the `model` with data from the *weights* file;\n", + "7. Convert the custom pointcloud data into the specified format;\n", + "8. Run an inference on the sample data.\n", + "\n", + "\n", + "> **Note:** We will be using a sample `RandLANet` `SemanticKITTI` weight file which we need to:\n", + ">\n", + "> 1. Download for either *PyTorch* or *TensorFlow* from links below:\n", + "> > a. For *PyTorch*: https://storage.googleapis.com/open3d-releases/model-zoo/randlanet_semantickitti_202201071330utc.pth\n", + "> >\n", + "> > b. For *TensorFlow*: https://storage.googleapis.com/open3d-releases/model-zoo/randlanet_semantickitti_202201071330utc.zip\n", ">\n", - "> 1. Download from: https://storage.googleapis.com/open3d-releases/model-zoo/randlanet_semantickitti_202201071330utc.pth\n", "> 2. Place the downloaded `randlanet_semantickitti_202201071330utc.pth` file into `'Open3D-ML/docs/tutorial/notebook/'` subdirectory, or any other place and change the `ckpt_path` accordingly.\n", ">\n", "> For other model/dataset weight files, please check out https://github.com/isl-org/Open3D-ML#semantic-segmentation-1\n", @@ -21,9 +37,9 @@ "\n", "An inference predicts the results based on the trained model. **Inference always depends on a prior step - _training_**.\n", "\n", - "> **Please see the [Training a Semantic Segmentation Model Using PyTorch](train_ss_model_using_pytorch.ipynb) and [Training a Semantic Segmentation Model Using TensorFlow](train_ss_model_using_tensorflow.ipynb) for training tutorials.**\n", + "> **Please see the [Training a semantic segmentation model using PyTorch](train_ss_model_using_pytorch.ipynb) and [Training a semantic segmentation model using TensorFlow](train_ss_model_using_tensorflow.ipynb) for training tutorials.**\n", "\n", - "To recap the model training process, - in our data model, we:\n", + "To recap the model training process:\n", "\n", "1. define a dataset;\n", "2. create a training split part of the data;\n", @@ -33,53 +49,14 @@ "\n", "While training, the model saves the checkpoint files every few epochs, in the *logs* directory. We use these trained weights to restore the model for inference.\n", "\n", - "Our first step in Inference on a Custom Data implementation is to import *Open3D-ML* and *Numpy* libraries: \n" + "Our first step in inference on a custom data implementation is to import `open3d.ml` and `numpy` libraries: \n" ] }, { "cell_type": "code", - "execution_count": 1, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Jupyter environment detected. Enabling Open3D WebVisualizer.\n", - "[Open3D INFO] WebRTC GUI backend enabled.\n", - "[Open3D INFO] WebRTCWindowSystem: HTTP handshake server disabled.\n", - "\n", - "--------------------------------------------------------------------------------\n", - "\n", - " Using the Open3D PyTorch ops with CUDA 11 may have stability issues!\n", - "\n", - " We recommend to compile PyTorch from source with compile flags\n", - " '-Xcompiler -fno-gnu-unique'\n", - "\n", - " or use the PyTorch wheels at\n", - " https://github.com/isl-org/open3d_downloads/releases/tag/torch1.8.2\n", - "\n", - "\n", - " Ignore this message if PyTorch has been compiled with the aforementioned\n", - " flags.\n", - "\n", - " See https://github.com/isl-org/Open3D/issues/3324 and\n", - " https://github.com/pytorch/pytorch/issues/52663 for more information on this\n", - " problem.\n", - "\n", - "--------------------------------------------------------------------------------\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "2022-04-06 15:53:47.653694: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", - "2022-04-06 15:53:47.653724: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n" - ] - } - ], + "outputs": [], "source": [ "import open3d.ml.torch as ml3d # just switch to open3d.ml.tf for tf usage\n", "import numpy as np" @@ -89,48 +66,56 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We then create a checkpoint path pointing to the data weights file we downloaded (generated at the end of the Training stage):" + "We then create a checkpoint path pointing to the weights file we downloaded (generated at the end of the Training stage):\n", + "\n", + "(You can download any other weights using a link from model zoo (collection of weights for all combinations of model and dataset): https://github.com/isl-org/Open3D-ML#semantic-segmentation-1 )" ] }, { "cell_type": "code", - "execution_count": 2, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ - "# Download weights using link from model zoo (collection of weights for all combinations of model and dataset).\n", - "# Link : https://github.com/isl-org/Open3D-ML#semantic-segmentation-1\n", - "# Randlanet SemanticKITTI[PyTorch] : https://storage.googleapis.com/open3d-releases/model-zoo/randlanet_semantickitti_202201071330utc.pth\n", - "ckpt_path = './randlanet_semantickitti_202201071330utc.pth'" + "weights_url = 'https://storage.googleapis.com/open3d-releases/model-zoo/randlanet_semantickitti_202201071330utc.zip'\n", + "ckpt_path = './randlanet_semantickitti_202201071330utc.pth'\n", + "# from urllib.request import urlretrieve\n", + "# urlretrieve(weights_url, filename=ckpt_path)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Now, we define a `dataset` object identical to how it was done in our previous *Training a Semantic Segmentation Model* tutorials:" + "Now, we define a `dataset` object identical to how it was done in our previous *Training a semantic segmentation model* tutorials:" ] }, { "cell_type": "code", - "execution_count": 3, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# We define dataset (similar to train_ss_using_pytorch tutorial)\n", - "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n" + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/',\n", + " cache_dir='./logs/cache',\n", + " training_split=['00'],\n", + " validation_split=['01'],\n", + " test_split=['01'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Then, initialize `model` and `pipeline` objects:" + "Then, initialize `model` and `pipeline` objects.\n", + "\n", + "For the `model` object, we have to specify the dimension of the features in the input data ( in the `in_channels` parameter):" ] }, { "cell_type": "code", - "execution_count": 4, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -143,22 +128,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Next, we restore the model with our data weights file with `pipeline.load_ckpt()` method:" + "Next, we restore the model with our weights file with `pipeline.load_ckpt()` method:" ] }, { "cell_type": "code", - "execution_count": 5, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO - 2022-04-06 15:54:37,266 - semantic_segmentation - Loading checkpoint ./randlanet_semantickitti_202201071330utc.pth\n" - ] - } - ], + "outputs": [], "source": [ "# Load checkpoint using `load_ckpt` method (restoring weights for inference)\n", "pipeline.load_ckpt(ckpt_path=ckpt_path)" @@ -173,19 +150,11 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": null, "metadata": { "scrolled": true }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO - 2022-04-06 15:54:40,026 - semantickitti - Found 1 pointclouds for test\n" - ] - } - ], + "outputs": [], "source": [ "test_data = dataset.get_split('test')\n", "data = test_data.get_data(0)" @@ -195,28 +164,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's display the content `data` contains:" + "Let's display what `data` contains:" ] }, { "cell_type": "code", - "execution_count": 7, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'point': array([[ 5.2305943e+01, 2.2989707e-02, 1.9779946e+00],\n", - " [ 5.3259735e+01, 1.0695236e-01, 2.0099745e+00],\n", - " [ 5.3284321e+01, 2.7487758e-01, 2.0109341e+00],\n", - " ...,\n", - " [ 3.8249431e+00, -1.4261885e+00, -1.7655631e+00],\n", - " [ 3.8495324e+00, -1.4222100e+00, -1.7755738e+00],\n", - " [ 3.8631279e+00, -1.4142324e+00, -1.7805853e+00]], dtype=float32), 'feat': None, 'label': array([0, 0, 0, ..., 9, 9, 9], dtype=int32)}\n" - ] - } - ], + "outputs": [], "source": [ "print(data)" ] @@ -231,28 +186,14 @@ "\n", "If you already have the *ground truth labels*, you can add them to data to get accuracy and IoU (Intersection over Union). Otherwise, pass labels as `None`.\n", "\n", - "And now - the main topic of our tutorial - running an inference on the test data. You can call the `run_inference()` method with your data, - it will print accuracy per class and Intersection over Union (IoU) metrics. The last entry in the list is *mean accuracy* and *mean IoU*:" + "And now - the main topic of our tutorial - running an inference on the test data. You can call the `run_inference()` method with your data, - it will print *accuracy per class* and *Intersection over Union (IoU)* metrics. The last entry in the list is *mean accuracy* and *mean IoU*:" ] }, { "cell_type": "code", - "execution_count": 9, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\n", - "test 0/1: 100%|█████████████████████████| 83500/83500 [01:14<00:00, 1115.74it/s]\u001b[A\n", - "\n", - "test 0/1: 100%|████████████████████████▉| 83262/83500 [00:09<00:00, 9105.02it/s]\u001b[AINFO - 2022-04-06 15:57:33,626 - semantic_segmentation - Accuracy : [0.9721905630271256, nan, nan, nan, nan, nan, nan, 0.0, 0.9554558517345533, 0.9201252936570086, 0.47223502304147463, nan, 0.9598018769551616, 0.9974093264248705, 0.8371072429561882, 0.8510466988727858, 0.9349005424954792, 0.9367521367521368, 0.67, 0.7922520463263987]\n", - "INFO - 2022-04-06 15:57:33,627 - semantic_segmentation - IoU : [0.9032189750105888, nan, 0.0, nan, 0.0, nan, nan, 0.0, 0.9118182072593961, 0.21114106019766396, 0.464405755504362, 0.0, 0.9425046078230596, 0.11663132384126022, 0.8259888650340235, 0.753922967189729, 0.6790123456790124, 0.8404907975460123, 0.5775862068965517, 0.481781407465444]\n", - "\n", - "test 0/1: 100%|█████████████████████████| 83500/83500 [00:25<00:00, 9105.02it/s]\u001b[A" - ] - } - ], + "outputs": [], "source": [ "# Running inference on test data\n", "results = pipeline.run_inference(data)\n", @@ -269,33 +210,9 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "{'predict_labels': array([12, 12, 12, ..., 8, 8, 8], dtype=int16),\n", - " 'predict_scores': array([[2.044e-05, 1.186e-05, 8.345e-07, ..., 1.608e-04, 2.861e-03,\n", - " 1.152e-03],\n", - " [1.615e-05, 1.079e-05, 8.345e-07, ..., 2.084e-04, 5.005e-03,\n", - " 2.203e-03],\n", - " [1.317e-05, 9.954e-06, 6.557e-07, ..., 1.574e-04, 6.054e-03,\n", - " 2.481e-03],\n", - " ...,\n", - " [0.000e+00, 0.000e+00, 0.000e+00, ..., 0.000e+00, 0.000e+00,\n", - " 0.000e+00],\n", - " [5.960e-08, 0.000e+00, 0.000e+00, ..., 0.000e+00, 0.000e+00,\n", - " 0.000e+00],\n", - " [5.960e-08, 0.000e+00, 0.000e+00, ..., 0.000e+00, 0.000e+00,\n", - " 0.000e+00]], dtype=float16)}" - ] - }, - "execution_count": 10, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "# Dictionary of predicted labels and predicted probabilities per class\n", "results" diff --git a/docs/tutorial/notebook/reading_a_config_file.ipynb b/docs/tutorial/notebook/reading_a_config_file.ipynb index 8efd66efd..aa9219fa1 100644 --- a/docs/tutorial/notebook/reading_a_config_file.ipynb +++ b/docs/tutorial/notebook/reading_a_config_file.ipynb @@ -4,7 +4,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Reading a *config* file" + "# Controlling training and inference with config files" ] }, { @@ -16,7 +16,11 @@ "```py\n", "# Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", "\n", - "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n", + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/',\n", + " cache_dir='./logs/cache',\n", + " training_split=['00'],\n", + " validation_split=['01'],\n", + " test_split=['01'])\n", "```\n", "\n", "In the code above, the `dataset` object is created by explicitly passing dataset-specific parameters into `ml3d.datasets.SemanticKITTI()` method.\n", @@ -77,8 +81,8 @@ "name": "stderr", "output_type": "stream", "text": [ - "2022-04-06 16:18:39.509582: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", - "2022-04-06 16:18:39.509610: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n" + "2022-04-12 10:51:04.294784: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", + "2022-04-12 10:51:04.294817: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n" ] } ], @@ -94,14 +98,14 @@ "Here, we import two modules:\n", " \n", " 1. `utils` - Open3D-ML utilities\n", - " 2. `ml3d` - Open3D-ML Torch library\n", + " 2. `ml3d` - Open3D-ML Torch API library\n", "\n", "Now, we'll create *config*-specific objects:" ] }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 3, "metadata": {}, "outputs": [], "source": [ @@ -116,124 +120,238 @@ "The `cfg_file` holds the full path to the particular *config* file - `randlanet_semantickitti.yml`.\n", "The `cfg` object is initialized by the Open3D-ML `utils.Config.load_from_file()` method to hold parameters that are read from the `cfg_file`.\n", "\n", - "## Examining Dataset Dictionaries in the `cfg` Object\n", + "## Examining dataset dictionaries in the `cfg` object\n", "\n", - "To find out what type the `cfg` object is, we can enter it in the next code cell:" + "Let's examine the contents of the `cfg` object:" ] }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "" + "{'_cfg_dict': {'dataset': {'name': 'SemanticKITTI',\n", + " 'dataset_path': None,\n", + " 'cache_dir': './logs/cache',\n", + " 'class_weights': [55437630,\n", + " 320797,\n", + " 541736,\n", + " 2578735,\n", + " 3274484,\n", + " 552662,\n", + " 184064,\n", + " 78858,\n", + " 240942562,\n", + " 17294618,\n", + " 170599734,\n", + " 6369672,\n", + " 230413074,\n", + " 101130274,\n", + " 476491114,\n", + " 9833174,\n", + " 129609852,\n", + " 4506626,\n", + " 1168181],\n", + " 'test_result_folder': './test',\n", + " 'test_split': ['11',\n", + " '12',\n", + " '13',\n", + " '14',\n", + " '15',\n", + " '16',\n", + " '17',\n", + " '18',\n", + " '19',\n", + " '20',\n", + " '21'],\n", + " 'training_split': ['00',\n", + " '01',\n", + " '02',\n", + " '03',\n", + " '04',\n", + " '05',\n", + " '06',\n", + " '07',\n", + " '09',\n", + " '10'],\n", + " 'all_split': ['00',\n", + " '01',\n", + " '02',\n", + " '03',\n", + " '04',\n", + " '05',\n", + " '06',\n", + " '07',\n", + " '09',\n", + " '08',\n", + " '10',\n", + " '11',\n", + " '12',\n", + " '13',\n", + " '14',\n", + " '15',\n", + " '16',\n", + " '17',\n", + " '18',\n", + " '19',\n", + " '20',\n", + " '21'],\n", + " 'validation_split': ['08'],\n", + " 'use_cache': True,\n", + " 'sampler': {'name': 'SemSegRandomSampler'}},\n", + " 'model': {'name': 'RandLANet',\n", + " 'batcher': 'DefaultBatcher',\n", + " 'ckpt_path': None,\n", + " 'num_neighbors': 16,\n", + " 'num_layers': 4,\n", + " 'num_points': 45056,\n", + " 'num_classes': 19,\n", + " 'ignored_label_inds': [0],\n", + " 'sub_sampling_ratio': [4, 4, 4, 4],\n", + " 'in_channels': 3,\n", + " 'dim_features': 8,\n", + " 'dim_output': [16, 64, 128, 256],\n", + " 'grid_size': 0.06,\n", + " 'augment': {'recenter': {'dim': [0, 1]}}},\n", + " 'pipeline': {'name': 'SemanticSegmentation',\n", + " 'optimizer': {'lr': 0.001},\n", + " 'batch_size': 4,\n", + " 'main_log_dir': './logs',\n", + " 'max_epoch': 100,\n", + " 'save_ckpt_freq': 5,\n", + " 'scheduler_gamma': 0.9886,\n", + " 'test_batch_size': 1,\n", + " 'train_sum_dir': 'train_log',\n", + " 'val_batch_size': 2,\n", + " 'summary': {'record_for': [],\n", + " 'max_pts': None,\n", + " 'use_reference': False,\n", + " 'max_outputs': 1}}},\n", + " 'cfg_dict': {'dataset': {'name': 'SemanticKITTI',\n", + " 'dataset_path': None,\n", + " 'cache_dir': './logs/cache',\n", + " 'class_weights': [55437630,\n", + " 320797,\n", + " 541736,\n", + " 2578735,\n", + " 3274484,\n", + " 552662,\n", + " 184064,\n", + " 78858,\n", + " 240942562,\n", + " 17294618,\n", + " 170599734,\n", + " 6369672,\n", + " 230413074,\n", + " 101130274,\n", + " 476491114,\n", + " 9833174,\n", + " 129609852,\n", + " 4506626,\n", + " 1168181],\n", + " 'test_result_folder': './test',\n", + " 'test_split': ['11',\n", + " '12',\n", + " '13',\n", + " '14',\n", + " '15',\n", + " '16',\n", + " '17',\n", + " '18',\n", + " '19',\n", + " '20',\n", + " '21'],\n", + " 'training_split': ['00',\n", + " '01',\n", + " '02',\n", + " '03',\n", + " '04',\n", + " '05',\n", + " '06',\n", + " '07',\n", + " '09',\n", + " '10'],\n", + " 'all_split': ['00',\n", + " '01',\n", + " '02',\n", + " '03',\n", + " '04',\n", + " '05',\n", + " '06',\n", + " '07',\n", + " '09',\n", + " '08',\n", + " '10',\n", + " '11',\n", + " '12',\n", + " '13',\n", + " '14',\n", + " '15',\n", + " '16',\n", + " '17',\n", + " '18',\n", + " '19',\n", + " '20',\n", + " '21'],\n", + " 'validation_split': ['08'],\n", + " 'use_cache': True,\n", + " 'sampler': {'name': 'SemSegRandomSampler'}},\n", + " 'model': {'name': 'RandLANet',\n", + " 'batcher': 'DefaultBatcher',\n", + " 'ckpt_path': None,\n", + " 'num_neighbors': 16,\n", + " 'num_layers': 4,\n", + " 'num_points': 45056,\n", + " 'num_classes': 19,\n", + " 'ignored_label_inds': [0],\n", + " 'sub_sampling_ratio': [4, 4, 4, 4],\n", + " 'in_channels': 3,\n", + " 'dim_features': 8,\n", + " 'dim_output': [16, 64, 128, 256],\n", + " 'grid_size': 0.06,\n", + " 'augment': {'recenter': {'dim': [0, 1]}}},\n", + " 'pipeline': {'name': 'SemanticSegmentation',\n", + " 'optimizer': {'lr': 0.001},\n", + " 'batch_size': 4,\n", + " 'main_log_dir': './logs',\n", + " 'max_epoch': 100,\n", + " 'save_ckpt_freq': 5,\n", + " 'scheduler_gamma': 0.9886,\n", + " 'test_batch_size': 1,\n", + " 'train_sum_dir': 'train_log',\n", + " 'val_batch_size': 2,\n", + " 'summary': {'record_for': [],\n", + " 'max_pts': None,\n", + " 'use_reference': False,\n", + " 'max_outputs': 1}}}}" ] }, - "execution_count": 3, + "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "cfg" + "vars(cfg)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Open3D-ML confirms the `cfg` is of `ml3d.utils.config.Config` type. The `cfg` object contains three dictionaries: `dataset`, `model`, and `pipeline`.\n", + "`vars(cfg)` returns the three dictionaries: `dataset`, `model`, and `pipeline`.\n", "\n", "Now, let's explore them. The first one to look at is the `cfg.dataset`:" ] }, { "cell_type": "code", - "execution_count": 4, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "{'name': 'SemanticKITTI',\n", - " 'dataset_path': None,\n", - " 'cache_dir': './logs/cache',\n", - " 'class_weights': [55437630,\n", - " 320797,\n", - " 541736,\n", - " 2578735,\n", - " 3274484,\n", - " 552662,\n", - " 184064,\n", - " 78858,\n", - " 240942562,\n", - " 17294618,\n", - " 170599734,\n", - " 6369672,\n", - " 230413074,\n", - " 101130274,\n", - " 476491114,\n", - " 9833174,\n", - " 129609852,\n", - " 4506626,\n", - " 1168181],\n", - " 'test_result_folder': './test',\n", - " 'test_split': ['11',\n", - " '12',\n", - " '13',\n", - " '14',\n", - " '15',\n", - " '16',\n", - " '17',\n", - " '18',\n", - " '19',\n", - " '20',\n", - " '21'],\n", - " 'training_split': ['00',\n", - " '01',\n", - " '02',\n", - " '03',\n", - " '04',\n", - " '05',\n", - " '06',\n", - " '07',\n", - " '09',\n", - " '10'],\n", - " 'all_split': ['00',\n", - " '01',\n", - " '02',\n", - " '03',\n", - " '04',\n", - " '05',\n", - " '06',\n", - " '07',\n", - " '09',\n", - " '08',\n", - " '10',\n", - " '11',\n", - " '12',\n", - " '13',\n", - " '14',\n", - " '15',\n", - " '16',\n", - " '17',\n", - " '18',\n", - " '19',\n", - " '20',\n", - " '21'],\n", - " 'validation_split': ['08'],\n", - " 'use_cache': True,\n", - " 'sampler': {'name': 'SemSegRandomSampler'}}" - ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "cfg.dataset" ] @@ -242,27 +360,16 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Accessing Individual Dictionary Items\n", + "### Accessing individual dictionary items\n", "\n", - "We can access individual items of the `cfg.dataset` dictionary like so: " + "These `cfg` dictionary items can be viewed as well as updated like in a standard Python dictionary. We can access individual items of the `cfg.dataset` dictionary like so: " ] }, { "cell_type": "code", - "execution_count": 5, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "'SemanticKITTI'" - ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "cfg.dataset['name']" ] @@ -271,42 +378,18 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## `cfg.model` and `cfg.pipeline` Dictionaries\n", + "## `cfg.model` and `cfg.pipeline` dictionaries\n", "\n", "We'll later revisit the `cfg.dataset`. Next, let's look at the `cfg.model` dictionary:" ] }, { "cell_type": "code", - "execution_count": 6, + "execution_count": null, "metadata": { "scrolled": true }, - "outputs": [ - { - "data": { - "text/plain": [ - "{'name': 'RandLANet',\n", - " 'batcher': 'DefaultBatcher',\n", - " 'ckpt_path': None,\n", - " 'num_neighbors': 16,\n", - " 'num_layers': 4,\n", - " 'num_points': 45056,\n", - " 'num_classes': 19,\n", - " 'ignored_label_inds': [0],\n", - " 'sub_sampling_ratio': [4, 4, 4, 4],\n", - " 'in_channels': 3,\n", - " 'dim_features': 8,\n", - " 'dim_output': [16, 64, 128, 256],\n", - " 'grid_size': 0.06,\n", - " 'augment': {'recenter': {'dim': [0, 1]}}}" - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "cfg.model" ] @@ -320,20 +403,9 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "[4, 4, 4, 4]" - ] - }, - "execution_count": 7, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "cfg.model['sub_sampling_ratio']" ] @@ -347,33 +419,9 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "{'name': 'SemanticSegmentation',\n", - " 'optimizer': {'lr': 0.001},\n", - " 'batch_size': 4,\n", - " 'main_log_dir': './logs',\n", - " 'max_epoch': 100,\n", - " 'save_ckpt_freq': 5,\n", - " 'scheduler_gamma': 0.9886,\n", - " 'test_batch_size': 1,\n", - " 'train_sum_dir': 'train_log',\n", - " 'val_batch_size': 2,\n", - " 'summary': {'record_for': [],\n", - " 'max_pts': None,\n", - " 'use_reference': False,\n", - " 'max_outputs': 1}}" - ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "cfg.pipeline" ] @@ -387,20 +435,9 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "'SemanticSegmentation'" - ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "cfg.pipeline['name']" ] @@ -409,14 +446,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Initializing Datasets from *Config* Files\n", + "## Initializing datasets from *config* files\n", "\n", "Next, we explicitly create the `dataset` object which will hold all information from the `cfg.dataset` dictionary:" ] }, { "cell_type": "code", - "execution_count": 10, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -432,73 +469,9 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "{'cfg': ,\n", - " 'name': 'SemanticKITTI',\n", - " 'rng': Generator(PCG64) at 0x7F42605AFA90,\n", - " 'label_to_names': {0: 'unlabeled',\n", - " 1: 'car',\n", - " 2: 'bicycle',\n", - " 3: 'motorcycle',\n", - " 4: 'truck',\n", - " 5: 'other-vehicle',\n", - " 6: 'person',\n", - " 7: 'bicyclist',\n", - " 8: 'motorcyclist',\n", - " 9: 'road',\n", - " 10: 'parking',\n", - " 11: 'sidewalk',\n", - " 12: 'other-ground',\n", - " 13: 'building',\n", - " 14: 'fence',\n", - " 15: 'vegetation',\n", - " 16: 'trunk',\n", - " 17: 'terrain',\n", - " 18: 'pole',\n", - " 19: 'traffic-sign'},\n", - " 'num_classes': 20,\n", - " 'remap_lut_val': array([ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 0, 5, 0, 3, 5,\n", - " 0, 4, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 7, 8, 0,\n", - " 0, 0, 0, 0, 0, 0, 9, 0, 0, 0, 10, 0, 0, 0, 11, 12, 13,\n", - " 14, 0, 0, 0, 0, 0, 0, 0, 0, 9, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 15, 16, 17, 0, 0, 0, 0, 0, 0, 0, 18, 19, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 7, 6,\n", - " 8, 5, 5, 4, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0], dtype=int32),\n", - " 'remap_lut': array([ 0, 10, 11, 15, 18, 20, 30, 31, 32, 40, 44, 48, 49, 50, 51, 70, 71,\n", - " 72, 80, 81, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", - " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n", - " dtype=int32)}" - ] - }, - "execution_count": 11, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "vars(dataset)" ] @@ -512,20 +485,9 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "20" - ] - }, - "execution_count": 12, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "dataset.num_classes" ] @@ -539,39 +501,9 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "{0: 'unlabeled',\n", - " 1: 'car',\n", - " 2: 'bicycle',\n", - " 3: 'motorcycle',\n", - " 4: 'truck',\n", - " 5: 'other-vehicle',\n", - " 6: 'person',\n", - " 7: 'bicyclist',\n", - " 8: 'motorcyclist',\n", - " 9: 'road',\n", - " 10: 'parking',\n", - " 11: 'sidewalk',\n", - " 12: 'other-ground',\n", - " 13: 'building',\n", - " 14: 'fence',\n", - " 15: 'vegetation',\n", - " 16: 'trunk',\n", - " 17: 'terrain',\n", - " 18: 'pole',\n", - " 19: 'traffic-sign'}" - ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "dataset.label_to_names" ] diff --git a/docs/tutorial/notebook/reading_a_dataset.ipynb b/docs/tutorial/notebook/reading_a_dataset.ipynb index ccbe3a907..39b1db6ec 100644 --- a/docs/tutorial/notebook/reading_a_dataset.ipynb +++ b/docs/tutorial/notebook/reading_a_dataset.ipynb @@ -4,7 +4,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Reading a Dataset" + "# Reading a dataset" ] }, { @@ -19,7 +19,7 @@ "\n", "- Dataset path (`dataset_path`)\n", "- Cache directory (`cache_dir`)\n", - "- Dataset Splits (for training, validation, and testing)\n", + "- Dataset splits (for training, validation, and testing)\n", "\n", "> **For more theoretical background information on dataset splitting, please refer to these articles:**\n", ">\n", @@ -27,57 +27,18 @@ ">\n", "> https://www.freecodecamp.org/news/key-machine-learning-concepts-explained-dataset-splitting-and-random-forest/\n", "\n", - "## Creating a Global Dataset Object\n", + "## Creating a global dataset object\n", "\n", "First, we declare the `ml3d` object to be of `open3d.ml.torch` library:" ] }, { "cell_type": "code", - "execution_count": 1, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Jupyter environment detected. Enabling Open3D WebVisualizer.\n", - "[Open3D INFO] WebRTC GUI backend enabled.\n", - "[Open3D INFO] WebRTCWindowSystem: HTTP handshake server disabled.\n", - "\n", - "--------------------------------------------------------------------------------\n", - "\n", - " Using the Open3D PyTorch ops with CUDA 11 may have stability issues!\n", - "\n", - " We recommend to compile PyTorch from source with compile flags\n", - " '-Xcompiler -fno-gnu-unique'\n", - "\n", - " or use the PyTorch wheels at\n", - " https://github.com/isl-org/open3d_downloads/releases/tag/torch1.8.2\n", - "\n", - "\n", - " Ignore this message if PyTorch has been compiled with the aforementioned\n", - " flags.\n", - "\n", - " See https://github.com/isl-org/Open3D/issues/3324 and\n", - " https://github.com/pytorch/pytorch/issues/52663 for more information on this\n", - " problem.\n", - "\n", - "--------------------------------------------------------------------------------\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "2022-04-06 16:20:21.687818: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", - "2022-04-06 16:20:21.687846: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n" - ] - } - ], + "outputs": [], "source": [ - "#import torch\n", + "# import torch\n", "import open3d.ml.torch as ml3d" ] }, @@ -90,12 +51,16 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Read a dataset by specifying the path. We are also providing the cache directory and splits.\n", - "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache', training_split=['00'], validation_split=['01'], test_split=['01'])" + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/',\n", + " cache_dir='./logs/cache',\n", + " training_split=['00'],\n", + " validation_split=['01'],\n", + " test_split=['01'])" ] }, { @@ -110,7 +75,7 @@ "\n", "Note the `SemanticKITTI` dataset folder structure:\n", "\n", - "![dataset_splits](https://user-images.githubusercontent.com/93158890/160633509-e190d8ea-7d3c-4eea-a00f-29ba215b7e69.jpg)\n", + "![dataset_structure](https://user-images.githubusercontent.com/93158890/162548755-28c541d3-3557-4903-a9a1-cc685d16dfc2.jpg)\n", "\n", "The three different *split* parameter variables instruct Open3D-ML subsystem to reference the following folder locations:\n", "\n", @@ -118,9 +83,9 @@ "- `validation_split=['01']` points to `'SemanticKITTI/dataset/sequences/01/'`\n", "- `test_split=['01']` points to `'SemanticKITTI/dataset/sequences/01/'`\n", "\n", - "> Note: **Dataset split directories usually contain numerous pointcloud files.** In our example we included only one pointcloud file for extra speed and convenience.\n", + "> Note: **dataset split directories usually contain numerous pointcloud files.** In our example we included only one pointcloud file for extra speed and convenience.\n", "\n", - "## Creating Dataset Split Objects to Query the Data\n", + "## Creating dataset split objects to query the data\n", "\n", "Next, we will create **dedicated** dataset split objects for specifying which split portion we would like to query.\n", "\n", @@ -129,19 +94,11 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": null, "metadata": { "scrolled": true }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO - 2022-04-06 16:22:26,972 - semantickitti - Found 1 pointclouds for training\n" - ] - } - ], + "outputs": [], "source": [ "# Split the dataset for 'training'. You can get the other splits by passing 'validation' or 'test'\n", "train_split = dataset.get_split('training')" @@ -156,17 +113,9 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO - 2022-04-06 16:22:31,932 - semantickitti - Found 1 pointclouds for validation\n" - ] - } - ], + "outputs": [], "source": [ "# Similarly, get validataion split.\n", "val_split = dataset.get_split('validation')" @@ -181,17 +130,9 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO - 2022-04-06 16:22:33,834 - semantickitti - Found 1 pointclouds for test\n" - ] - } - ], + "outputs": [], "source": [ "# Get test split\n", "test_split = dataset.get_split('test')" @@ -201,26 +142,16 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Determining the Size of Dataset Splits\n", + "## Determining the size of dataset splits\n", "\n", "Let's see how large out *split* portions are:" ] }, { "cell_type": "code", - "execution_count": 6, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "1\n", - "1\n", - "1\n" - ] - } - ], + "outputs": [], "source": [ "# Get length of splits\n", "print(len(train_split))\n", @@ -234,7 +165,7 @@ "source": [ "Above, Open3D-ML prints out the number of pointcloud files it found in `'SemanticKITTI/dataset/sequences/'`' `'/00/'` and `'/01/'` subdirectories we have specified earlier in `training_split=['00'], validation_split=['01'], test_split=['01']` varables for the `dataset`.\n", "\n", - "## Querying Dataset Splits for Data\n", + "## Querying dataset splits for data\n", "\n", "In this section, we are using the `train_split` dataset split object as an example. The procedure would be identical for all other dataset splits - `val_split` and `test_split`.\n", "\n", @@ -244,23 +175,9 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'point': array([[ 5.2305943e+01, 2.2989707e-02, 1.9779946e+00],\n", - " [ 5.3259735e+01, 1.0695236e-01, 2.0099745e+00],\n", - " [ 5.3284321e+01, 2.7487758e-01, 2.0109341e+00],\n", - " ...,\n", - " [ 3.8249431e+00, -1.4261885e+00, -1.7655631e+00],\n", - " [ 3.8495324e+00, -1.4222100e+00, -1.7755738e+00],\n", - " [ 3.8631279e+00, -1.4142324e+00, -1.7805853e+00]], dtype=float32), 'feat': None, 'label': array([0, 0, 0, ..., 9, 9, 9], dtype=int32)}\n" - ] - } - ], + "outputs": [], "source": [ "# Query splits for data, index should be from `0` to `len(split) - 1`\n", "for i in range(len(train_split)):\n", @@ -278,19 +195,11 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "dict_keys(['point', 'feat', 'label'])\n" - ] - } - ], + "outputs": [], "source": [ - "data = train_split.get_data(0) # Dictionary of `point`, `feat`, and `label`\n", + "data = train_split.get_data(0) # Dictionary of `point`, `feat`, and `label`\n", "print(data.keys())" ] }, @@ -300,33 +209,27 @@ "source": [ "- The **`'point'`** key value contains a set of 3D points / coordinates - X, Y, and Z:\n", "\n", - "![dataset_coordinates](https://user-images.githubusercontent.com/93158890/160503607-76e77f7a-56be-4fba-91c5-7e35019f68e8.jpg)\n", + "![dataset_coordinates](https://user-images.githubusercontent.com/93158890/162549410-6369cbd0-b835-4216-ba54-945e3f591395.jpg)\n", "\n", "- The **`'feat'`** (*features*) key value contains RGB color information for each of the above points.\n", "\n", "- The **`'label'`** key value represents which class the dataset content belongs to, i.e.: *pedestrian, vehicle, traffic light*, etc.\n", "\n", - "### Querying Dataset Splits for Attributes\n", + "### Querying dataset splits for attributes\n", "\n", "We can also extract corresponding pointcloud information:" ] }, { "cell_type": "code", - "execution_count": 9, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'idx': 0, 'name': '00_000001', 'path': 'SemanticKITTI/dataset/sequences/00/velodyne/000001.bin', 'split': 'training'}\n" - ] - } - ], + "outputs": [], "source": [ "attr = train_split.get_attr(0)\n", - "print(attr) # Dictionary containing information about the data e.g. name, path, split, etc." + "print(\n", + " attr\n", + ") # Dictionary containing information about the data e.g. name, path, split, etc." ] }, { @@ -338,7 +241,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ diff --git a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb index 7e6e0b9da..92e5e12ad 100644 --- a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb +++ b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb @@ -5,7 +5,7 @@ "id": "48589547", "metadata": {}, "source": [ - "# Training a Semantic Segmentation Model Using PyTorch" + "# Training a semantic segmentation model using PyTorch" ] }, { @@ -13,26 +13,20 @@ "id": "49b68041", "metadata": {}, "source": [ - "In this tutorial, we will learn how to train a semantic segmentation model using PyTorch in a Jupyter Notebook. We assume that you are familiar with Jupyter Notebook and have created a folder notebooks in a folder that is relative to ml3d.\n", + "In this tutorial, we will learn how to train a semantic segmentation model using PyTorch.\n", "\n", "Before you begin, ensure that you have *PyTorch* installed. To install a compatible version of PyTorch, use the requirement file:\n", "\n", "```sh\n", "pip install -r requirements-torch-cuda.txt\n", - "```" - ] - }, - { - "cell_type": "markdown", - "id": "2c10c2d2", - "metadata": {}, - "source": [ + "```\n", + "\n", "At a high-level, we will:\n", "\n", "- Read a dataset and get a training split. For this example, we will use `SemanticKITTI` dataset.\n", "- Run a pre-trained model. For this example, we will use the `RandLANet` model.\n", "- Train a model. We will train a model using the `SemanticKITTI` dataset and `RandLANet` model.\n", - "- Run an inference and run a test. We will run an inference using the 'training' split that use a pointcloud and display a result. However, a test is run on a pre-defined test set rather than a pass pointcloud.\n" + "- Run an inference and run a test. We will run an inference using the *'training'* split that use a pointcloud and display a result. However, a test is run on a pre-defined test set rather than a pass pointcloud.\n" ] }, { @@ -55,18 +49,10 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": null, "id": "d8090925", "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO - 2022-04-07 10:01:51,953 - semantickitti - Found 1 pointclouds for training\n" - ] - } - ], + "outputs": [], "source": [ "# Training Semantic Segmentation Model using PyTorch\n", "\n", @@ -74,7 +60,11 @@ "import open3d.ml.torch as ml3d\n", "\n", "# Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", - "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n", + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/',\n", + " cache_dir='./logs/cache',\n", + " training_split=['00'],\n", + " validation_split=['01'],\n", + " test_split=['01'])\n", "\n", "# Split the dataset for 'training'. You can get the other splits by passing 'validation' or 'test'\n", "train_split = dataset.get_split('training')\n", @@ -93,73 +83,24 @@ "Now that you have visualized the dataset for training, let us train the model." ] }, - { - "cell_type": "markdown", - "id": "d077a5a2", - "metadata": {}, - "source": [ - "## Training a model" - ] - }, { "cell_type": "markdown", "id": "bb442bcd", "metadata": {}, "source": [ - "Before you train a model, you must decide which model you want to use. For this example, we will use the `RandLANet` model. To use models, you must import the model from `open3d.ml.torch.models`.\n", + "## Training a model\n", + "\n", + "Before you train a model, you must decide which model you want to use. To use models, you must import the model from `open3d.ml.torch.models`.\n", "\n", "After you load a dataset, you can initialize any model and then train the model. The following example shows how you can train a model:" ] }, { "cell_type": "code", - "execution_count": 6, + "execution_count": null, "id": "989a6579", "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "test 0/1: 100%|████████████████████████| 83500/83500 [17:36:53<00:00, 1.32it/s]\n", - "INFO - 2022-04-07 10:02:40,533 - semantic_segmentation - DEVICE : cpu\n", - "INFO - 2022-04-07 10:02:40,545 - semantic_segmentation - Logging in file : ./logs/RandLANet_SemanticKITTI_torch/log_train_2022-04-07_10:02:40.txt\n", - "INFO - 2022-04-07 10:02:40,610 - semantickitti - Found 1 pointclouds for train\n", - "INFO - 2022-04-07 10:02:40,613 - semantickitti - Found 1 pointclouds for validation\n", - "INFO - 2022-04-07 10:02:40,657 - semantic_segmentation - ckpt_path not given. Restore from the latest ckpt\n", - "INFO - 2022-04-07 10:02:40,658 - semantic_segmentation - Loading checkpoint ./logs/RandLANet_SemanticKITTI_torch/checkpoint/ckpt_00000.pth\n", - "INFO - 2022-04-07 10:02:41,056 - semantic_segmentation - Loading checkpoint optimizer_state_dict\n", - "INFO - 2022-04-07 10:02:41,082 - semantic_segmentation - Loading checkpoint scheduler_state_dict\n", - "INFO - 2022-04-07 10:02:41,100 - semantic_segmentation - Writing summary in train_log/00005_RandLANet_SemanticKITTI_torch.\n", - "INFO - 2022-04-07 10:02:41,104 - semantic_segmentation - Started training\n", - "INFO - 2022-04-07 10:02:41,105 - semantic_segmentation - === EPOCH 0/3 ===\n", - "training: 100%|███████████████████████████████████| 1/1 [00:05<00:00, 5.33s/it]\n", - "validation: 100%|█████████████████████████████████| 1/1 [00:02<00:00, 2.04s/it]\n", - "INFO - 2022-04-07 10:02:48,503 - semantic_segmentation - Loss train: 3.181 eval: 2.993\n", - "INFO - 2022-04-07 10:02:48,505 - semantic_segmentation - Mean acc train: 0.063 eval: 0.000\n", - "INFO - 2022-04-07 10:02:48,506 - semantic_segmentation - Mean IoU train: 0.009 eval: 0.000\n", - "INFO - 2022-04-07 10:02:48,607 - semantic_segmentation - Epoch 0: save ckpt to ./logs/RandLANet_SemanticKITTI_torch/checkpoint\n", - "INFO - 2022-04-07 10:02:48,609 - semantic_segmentation - === EPOCH 1/3 ===\n", - "training: 100%|███████████████████████████████████| 1/1 [00:04<00:00, 4.57s/it]\n", - "validation: 100%|█████████████████████████████████| 1/1 [00:01<00:00, 1.99s/it]\n", - "INFO - 2022-04-07 10:02:55,186 - semantic_segmentation - Loss train: 3.007 eval: 3.019\n", - "INFO - 2022-04-07 10:02:55,187 - semantic_segmentation - Mean acc train: 0.077 eval: 0.096\n", - "INFO - 2022-04-07 10:02:55,192 - semantic_segmentation - Mean IoU train: 0.020 eval: 0.014\n", - "INFO - 2022-04-07 10:02:55,194 - semantic_segmentation - === EPOCH 2/3 ===\n", - "training: 100%|███████████████████████████████████| 1/1 [00:03<00:00, 3.69s/it]\n", - "validation: 100%|█████████████████████████████████| 1/1 [00:02<00:00, 2.07s/it]\n", - "INFO - 2022-04-07 10:03:00,958 - semantic_segmentation - Loss train: 2.939 eval: 3.001\n", - "INFO - 2022-04-07 10:03:00,960 - semantic_segmentation - Mean acc train: 0.112 eval: 0.094\n", - "INFO - 2022-04-07 10:03:00,961 - semantic_segmentation - Mean IoU train: 0.024 eval: 0.017\n", - "INFO - 2022-04-07 10:03:00,962 - semantic_segmentation - === EPOCH 3/3 ===\n", - "training: 100%|███████████████████████████████████| 1/1 [00:03<00:00, 3.88s/it]\n", - "validation: 100%|█████████████████████████████████| 1/1 [00:01<00:00, 1.98s/it]\n", - "INFO - 2022-04-07 10:03:06,841 - semantic_segmentation - Loss train: 2.942 eval: 3.001\n", - "INFO - 2022-04-07 10:03:06,843 - semantic_segmentation - Mean acc train: 0.099 eval: 0.111\n", - "INFO - 2022-04-07 10:03:06,844 - semantic_segmentation - Mean IoU train: 0.025 eval: 0.019\n" - ] - } - ], + "outputs": [], "source": [ "# Training Semantic Segmentation Model using PyTorch\n", "\n", @@ -170,58 +111,42 @@ "\n", "# Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", "# dataset = ml3d.datasets.SemanticKITTI(dataset_path='/Users/sanskara/data/SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n", - "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n", + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/',\n", + " cache_dir='./logs/cache',\n", + " training_split=['00'],\n", + " validation_split=['01'],\n", + " test_split=['01'])\n", "\n", "# Initialize the RandLANet model with three layers.\n", "model = RandLANet(in_channels=3)\n", - "pipeline = SemanticSegmentation(model=model, dataset=dataset, max_epoch=3, optimizer={'lr': 0.001}, num_workers=0)\n", + "pipeline = SemanticSegmentation(model=model,\n", + " dataset=dataset,\n", + " max_epoch=3,\n", + " optimizer={'lr': 0.001},\n", + " num_workers=0)\n", "\n", "# Run the training\n", "pipeline.run_train()" ] }, - { - "cell_type": "markdown", - "id": "deb61bc0", - "metadata": {}, - "source": [ - "## Running a Test" - ] - }, { "cell_type": "markdown", "id": "ad40d3ea", "metadata": {}, "source": [ - "Running a test is very similar to training the model.\n", + "## Running a test\n", + "\n", + "Next, we will evaluate the trained model on the test split.\n", "\n", "We can call the `run_test()` method, and it will run inference on the test split." ] }, { "cell_type": "code", - "execution_count": 10, + "execution_count": null, "id": "9269a2df", "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO - 2022-04-07 10:11:46,055 - semantic_segmentation - DEVICE : cpu\n", - "INFO - 2022-04-07 10:11:46,056 - semantic_segmentation - Logging in file : ./logs/RandLANet_SemanticKITTI_torch/log_test_2022-04-07_10:11:46.txt\n", - "INFO - 2022-04-07 10:11:46,057 - semantickitti - Found 1 pointclouds for test\n", - "INFO - 2022-04-07 10:11:46,742 - semantic_segmentation - ckpt_path not given. Restore from the latest ckpt\n", - "INFO - 2022-04-07 10:11:46,743 - semantic_segmentation - Loading checkpoint ./logs/RandLANet_SemanticKITTI_torch/checkpoint/ckpt_00000.pth\n", - "INFO - 2022-04-07 10:11:46,823 - semantic_segmentation - Started testing\n", - "test 0/1: 100%|██████████████████████████| 83500/83500 [04:38<00:00, 300.24it/s]\n", - "test 0/1: 100%|█████████████████████████| 83500/83500 [00:19<00:00, 3141.54it/s]INFO - 2022-04-07 10:12:13,619 - semantic_segmentation - Accuracy : [0.0, nan, nan, nan, nan, nan, nan, 0.0, 0.0, 0.0, 0.0, nan, 0.0, 0.0, 0.0, 0.0, 0.9023508137432188, 0.0, 0.0, 0.07519590114526824]\n", - "INFO - 2022-04-07 10:12:13,621 - semantic_segmentation - IoU : [0.0, nan, nan, nan, nan, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, nan, 0.0, 0.0, 0.0, 0.0, 0.12928800912011607, 0.0, 0.0, 0.009234857794294005]\n", - "INFO - 2022-04-07 10:12:13,624 - semantic_segmentation - Overall Testing Accuracy : 0.07519590114526824, mIoU : 0.009234857794294005\n", - "INFO - 2022-04-07 10:12:13,625 - semantic_segmentation - Finished testing\n" - ] - } - ], + "outputs": [], "source": [ "pipeline.run_test()" ] @@ -231,7 +156,7 @@ "id": "5404cae2", "metadata": {}, "source": [ - "## Running an Inference\n", + "## Running an inference\n", "\n", "An inference processes point cloud and displays the results based on the trained model. For this example, we will use a trained `RandLANet` model.\n", "\n", @@ -240,43 +165,10 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": null, "id": "2e6aad28", "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "test 0/1: 100%|█████████████████████████| 83500/83500 [00:24<00:00, 3389.87it/s]\n", - "INFO - 2022-04-07 10:12:17,679 - semantickitti - Found 1 pointclouds for test\n", - "test 0/1: 100%|████████████████████████▉| 83406/83500 [00:09<00:00, 9129.51it/s]INFO - 2022-04-07 10:12:46,190 - semantic_segmentation - Accuracy : [0.0, nan, nan, nan, nan, nan, nan, 0.0, 0.0, 0.0, 0.0, nan, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]\n", - "INFO - 2022-04-07 10:12:46,191 - semantic_segmentation - IoU : [0.0, nan, nan, 0.0, nan, nan, nan, 0.0, 0.0, 0.0, 0.0, nan, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'predict_labels': array([3, 3, 3, ..., 3, 3, 3], dtype=int16), 'predict_scores': array([[0.00556 , 0.005356, 0.004414, ..., 0.00475 , 0.005623, 0.004128],\n", - " [0.005558, 0.00535 , 0.004414, ..., 0.004745, 0.005623, 0.004124],\n", - " [0.005558, 0.00535 , 0.004417, ..., 0.004745, 0.00562 , 0.004124],\n", - " ...,\n", - " [0.02107 , 0.01907 , 0.0178 , ..., 0.01909 , 0.01985 , 0.01637 ],\n", - " [0.02107 , 0.01907 , 0.0178 , ..., 0.01909 , 0.01985 , 0.01637 ],\n", - " [0.02107 , 0.01907 , 0.0178 , ..., 0.01909 , 0.01985 , 0.01637 ]],\n", - " dtype=float16)}\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\r", - "test 0/1: 100%|█████████████████████████| 83500/83500 [00:20<00:00, 9129.51it/s]" - ] - } - ], + "outputs": [], "source": [ "# Training Semantic Segmentation Model using PyTorch\n", "\n", @@ -285,9 +177,17 @@ "from open3d.ml.torch.models import RandLANet\n", "from open3d.ml.torch.pipelines import SemanticSegmentation\n", "\n", - "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'], validation_split=['01'], test_split=['01'])\n", + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/',\n", + " cache_dir='./logs/cache',\n", + " training_split=['00'],\n", + " validation_split=['01'],\n", + " test_split=['01'])\n", "model = RandLANet(in_channels=3)\n", - "pipeline = SemanticSegmentation(model=model, dataset=dataset, max_epoch=3, optimizer={'lr': 0.001}, num_workers=0)\n", + "pipeline = SemanticSegmentation(model=model,\n", + " dataset=dataset,\n", + " max_epoch=3,\n", + " optimizer={'lr': 0.001},\n", + " num_workers=0)\n", "\n", "# Get data from the SemanticKITTI dataset using the \"train\" split\n", "train_split = dataset.get_split(\"test\")\n", diff --git a/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb b/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb index 09a6871c1..632a491ab 100644 --- a/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb +++ b/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb @@ -5,7 +5,7 @@ "id": "c8d52012", "metadata": {}, "source": [ - "# Training a Semantic Segmentation Model Using TensorFlow" + "# Training a semantic segmentation model using TensorFlow" ] }, { @@ -29,19 +29,13 @@ "- Run an inference and run a test. We will run an inference using the 'training' split that use a pointcloud and display a result. However, a test is run on a pre-defined test set rather than a pass pointcloud." ] }, - { - "cell_type": "markdown", - "id": "8055b308", - "metadata": {}, - "source": [ - "## Reading a dataset" - ] - }, { "cell_type": "markdown", "id": "5e33bf9d", "metadata": {}, "source": [ + "## Reading a dataset\n", + "\n", "You can use any dataset available in the `ml3d.datasets` dataset namespace. For this example, we will use the `SemanticKITTI` dataset and visualize it. You can use any of the other dataset to load data. However, you must understand that the parameters may vary for each dataset.\n", "\n", "We will read the dataset by specifying its path and then get all splits." @@ -49,31 +43,10 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": null, "id": "b4e344db", "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Jupyter environment detected. Enabling Open3D WebVisualizer.\n", - "[Open3D INFO] WebRTC GUI backend enabled.\n", - "[Open3D INFO] WebRTCWindowSystem: HTTP handshake server disabled.\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "2022-04-07 10:20:31.517091: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", - "2022-04-07 10:20:31.517124: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n", - "2022-04-07 10:20:33.389558: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", - "2022-04-07 10:20:33.389589: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n", - "INFO - 2022-04-07 10:20:34,543 - semantickitti - Found 1 pointclouds for training\n" - ] - } - ], + "outputs": [], "source": [ "# Training Semantic Segmentation Model using TensorFlow\n", "\n", @@ -81,7 +54,9 @@ "import open3d.ml.tf as ml3d\n", "\n", "# Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", - "dataset = ml3d.datasets.SemanticKITTI(dataset_path='./SemanticKITTI/', cache_dir='./logs/cache',training_split=['00'])\n", + "dataset = ml3d.datasets.SemanticKITTI(dataset_path='./SemanticKITTI/',\n", + " cache_dir='./logs/cache',\n", + " training_split=['00'])\n", "\n", "# Split the dataset for 'training'. You can get the other splits by passing 'validation' or 'test'\n", "train_split = dataset.get_split('training')\n", @@ -99,38 +74,22 @@ "Now that you have visualized the dataset for training, let us train the model." ] }, - { - "cell_type": "markdown", - "id": "258186a2", - "metadata": {}, - "source": [ - "## Training a model" - ] - }, { "cell_type": "markdown", "id": "9653bd32", "metadata": {}, "source": [ + "## Training a model\n", + "\n", "`TensorFlow` maps nearly all of GPU memory by default. This may result in out_of_memory error if some of the ops allocate memory independent to tensorflow. You may want to limit memory usage as and when needed by the process. Use following code right after importing tensorflow:" ] }, { "cell_type": "code", - "execution_count": 2, + "execution_count": null, "id": "c220cfb6", "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "2022-04-06 16:31:11.326658: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory\n", - "2022-04-06 16:31:11.326709: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)\n", - "2022-04-06 16:31:11.326750: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (intel-VirtualBox): /proc/driver/nvidia/version does not exist\n" - ] - } - ], + "outputs": [], "source": [ "import tensorflow as tf\n", "gpus = tf.config.experimental.list_physical_devices('GPU')\n", @@ -156,56 +115,10 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": null, "id": "caeea522", "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO - 2022-04-07 10:25:19,140 - semantic_segmentation - \n", - "INFO - 2022-04-07 10:25:19,142 - semantic_segmentation - Logging in file : ./logs/RandLANet_SemanticKITTI_tf/log_train_2022-04-07_10:25:19.txt\n", - "INFO - 2022-04-07 10:25:19,144 - semantickitti - Found 1 pointclouds for training\n", - "INFO - 2022-04-07 10:25:19,308 - semantickitti - Found 1 pointclouds for validation\n", - "INFO - 2022-04-07 10:25:19,441 - semantic_segmentation - Writing summary in train_log/00009_RandLANet_SemanticKITTI_tf.\n", - "INFO - 2022-04-07 10:25:19,444 - semantic_segmentation - Initializing from scratch.\n", - "INFO - 2022-04-07 10:25:19,445 - semantic_segmentation - === EPOCH 0/3 ===\n", - "training: 0%| | 0/1 [00:00 Date: Tue, 19 Apr 2022 10:51:28 -0400 Subject: [PATCH 11/13] 4-19-2022 - More editing passes for * Reading a Config File * Training Semantic Segmentation Model Using PyTorch --- .../notebook/reading_a_config_file.ipynb | 253 +----------------- .../train_ss_model_using_pytorch.ipynb | 22 +- 2 files changed, 20 insertions(+), 255 deletions(-) diff --git a/docs/tutorial/notebook/reading_a_config_file.ipynb b/docs/tutorial/notebook/reading_a_config_file.ipynb index aa9219fa1..958d6df32 100644 --- a/docs/tutorial/notebook/reading_a_config_file.ipynb +++ b/docs/tutorial/notebook/reading_a_config_file.ipynb @@ -44,48 +44,9 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Jupyter environment detected. Enabling Open3D WebVisualizer.\n", - "[Open3D INFO] WebRTC GUI backend enabled.\n", - "[Open3D INFO] WebRTCWindowSystem: HTTP handshake server disabled.\n", - "\n", - "--------------------------------------------------------------------------------\n", - "\n", - " Using the Open3D PyTorch ops with CUDA 11 may have stability issues!\n", - "\n", - " We recommend to compile PyTorch from source with compile flags\n", - " '-Xcompiler -fno-gnu-unique'\n", - "\n", - " or use the PyTorch wheels at\n", - " https://github.com/isl-org/open3d_downloads/releases/tag/torch1.8.2\n", - "\n", - "\n", - " Ignore this message if PyTorch has been compiled with the aforementioned\n", - " flags.\n", - "\n", - " See https://github.com/isl-org/Open3D/issues/3324 and\n", - " https://github.com/pytorch/pytorch/issues/52663 for more information on this\n", - " problem.\n", - "\n", - "--------------------------------------------------------------------------------\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "2022-04-12 10:51:04.294784: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", - "2022-04-12 10:51:04.294817: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n" - ] - } - ], + "outputs": [], "source": [ "from open3d.ml import utils\n", "import open3d.ml.torch as ml3d" @@ -105,7 +66,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -127,213 +88,9 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "{'_cfg_dict': {'dataset': {'name': 'SemanticKITTI',\n", - " 'dataset_path': None,\n", - " 'cache_dir': './logs/cache',\n", - " 'class_weights': [55437630,\n", - " 320797,\n", - " 541736,\n", - " 2578735,\n", - " 3274484,\n", - " 552662,\n", - " 184064,\n", - " 78858,\n", - " 240942562,\n", - " 17294618,\n", - " 170599734,\n", - " 6369672,\n", - " 230413074,\n", - " 101130274,\n", - " 476491114,\n", - " 9833174,\n", - " 129609852,\n", - " 4506626,\n", - " 1168181],\n", - " 'test_result_folder': './test',\n", - " 'test_split': ['11',\n", - " '12',\n", - " '13',\n", - " '14',\n", - " '15',\n", - " '16',\n", - " '17',\n", - " '18',\n", - " '19',\n", - " '20',\n", - " '21'],\n", - " 'training_split': ['00',\n", - " '01',\n", - " '02',\n", - " '03',\n", - " '04',\n", - " '05',\n", - " '06',\n", - " '07',\n", - " '09',\n", - " '10'],\n", - " 'all_split': ['00',\n", - " '01',\n", - " '02',\n", - " '03',\n", - " '04',\n", - " '05',\n", - " '06',\n", - " '07',\n", - " '09',\n", - " '08',\n", - " '10',\n", - " '11',\n", - " '12',\n", - " '13',\n", - " '14',\n", - " '15',\n", - " '16',\n", - " '17',\n", - " '18',\n", - " '19',\n", - " '20',\n", - " '21'],\n", - " 'validation_split': ['08'],\n", - " 'use_cache': True,\n", - " 'sampler': {'name': 'SemSegRandomSampler'}},\n", - " 'model': {'name': 'RandLANet',\n", - " 'batcher': 'DefaultBatcher',\n", - " 'ckpt_path': None,\n", - " 'num_neighbors': 16,\n", - " 'num_layers': 4,\n", - " 'num_points': 45056,\n", - " 'num_classes': 19,\n", - " 'ignored_label_inds': [0],\n", - " 'sub_sampling_ratio': [4, 4, 4, 4],\n", - " 'in_channels': 3,\n", - " 'dim_features': 8,\n", - " 'dim_output': [16, 64, 128, 256],\n", - " 'grid_size': 0.06,\n", - " 'augment': {'recenter': {'dim': [0, 1]}}},\n", - " 'pipeline': {'name': 'SemanticSegmentation',\n", - " 'optimizer': {'lr': 0.001},\n", - " 'batch_size': 4,\n", - " 'main_log_dir': './logs',\n", - " 'max_epoch': 100,\n", - " 'save_ckpt_freq': 5,\n", - " 'scheduler_gamma': 0.9886,\n", - " 'test_batch_size': 1,\n", - " 'train_sum_dir': 'train_log',\n", - " 'val_batch_size': 2,\n", - " 'summary': {'record_for': [],\n", - " 'max_pts': None,\n", - " 'use_reference': False,\n", - " 'max_outputs': 1}}},\n", - " 'cfg_dict': {'dataset': {'name': 'SemanticKITTI',\n", - " 'dataset_path': None,\n", - " 'cache_dir': './logs/cache',\n", - " 'class_weights': [55437630,\n", - " 320797,\n", - " 541736,\n", - " 2578735,\n", - " 3274484,\n", - " 552662,\n", - " 184064,\n", - " 78858,\n", - " 240942562,\n", - " 17294618,\n", - " 170599734,\n", - " 6369672,\n", - " 230413074,\n", - " 101130274,\n", - " 476491114,\n", - " 9833174,\n", - " 129609852,\n", - " 4506626,\n", - " 1168181],\n", - " 'test_result_folder': './test',\n", - " 'test_split': ['11',\n", - " '12',\n", - " '13',\n", - " '14',\n", - " '15',\n", - " '16',\n", - " '17',\n", - " '18',\n", - " '19',\n", - " '20',\n", - " '21'],\n", - " 'training_split': ['00',\n", - " '01',\n", - " '02',\n", - " '03',\n", - " '04',\n", - " '05',\n", - " '06',\n", - " '07',\n", - " '09',\n", - " '10'],\n", - " 'all_split': ['00',\n", - " '01',\n", - " '02',\n", - " '03',\n", - " '04',\n", - " '05',\n", - " '06',\n", - " '07',\n", - " '09',\n", - " '08',\n", - " '10',\n", - " '11',\n", - " '12',\n", - " '13',\n", - " '14',\n", - " '15',\n", - " '16',\n", - " '17',\n", - " '18',\n", - " '19',\n", - " '20',\n", - " '21'],\n", - " 'validation_split': ['08'],\n", - " 'use_cache': True,\n", - " 'sampler': {'name': 'SemSegRandomSampler'}},\n", - " 'model': {'name': 'RandLANet',\n", - " 'batcher': 'DefaultBatcher',\n", - " 'ckpt_path': None,\n", - " 'num_neighbors': 16,\n", - " 'num_layers': 4,\n", - " 'num_points': 45056,\n", - " 'num_classes': 19,\n", - " 'ignored_label_inds': [0],\n", - " 'sub_sampling_ratio': [4, 4, 4, 4],\n", - " 'in_channels': 3,\n", - " 'dim_features': 8,\n", - " 'dim_output': [16, 64, 128, 256],\n", - " 'grid_size': 0.06,\n", - " 'augment': {'recenter': {'dim': [0, 1]}}},\n", - " 'pipeline': {'name': 'SemanticSegmentation',\n", - " 'optimizer': {'lr': 0.001},\n", - " 'batch_size': 4,\n", - " 'main_log_dir': './logs',\n", - " 'max_epoch': 100,\n", - " 'save_ckpt_freq': 5,\n", - " 'scheduler_gamma': 0.9886,\n", - " 'test_batch_size': 1,\n", - " 'train_sum_dir': 'train_log',\n", - " 'val_batch_size': 2,\n", - " 'summary': {'record_for': [],\n", - " 'max_pts': None,\n", - " 'use_reference': False,\n", - " 'max_outputs': 1}}}}" - ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "vars(cfg)" ] diff --git a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb index 92e5e12ad..34ea519f2 100644 --- a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb +++ b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb @@ -23,10 +23,10 @@ "\n", "At a high-level, we will:\n", "\n", - "- Read a dataset and get a training split. For this example, we will use `SemanticKITTI` dataset.\n", - "- Run a pre-trained model. For this example, we will use the `RandLANet` model.\n", - "- Train a model. We will train a model using the `SemanticKITTI` dataset and `RandLANet` model.\n", - "- Run an inference and run a test. We will run an inference using the *'training'* split that use a pointcloud and display a result. However, a test is run on a pre-defined test set rather than a pass pointcloud.\n" + "- Read a dataset and create a *'training'* split. For this example, we will use `SemanticKITTI` dataset.\n", + "- Train a model. We will train a `RandLANet` model using the *'training'* split.\n", + "- Run a test on a *'test'* split to evaluate the model.\n", + "- Run an inference on a custom pointcloud.\n" ] }, { @@ -42,6 +42,8 @@ "id": "b69972ae", "metadata": {}, "source": [ + "Downloading scripts are available in: `Open3D-ML/scripts/download_datasets`\n", + "\n", "You can use any dataset available in the `ml3d.datasets` dataset namespace. For this example, we will use the `SemanticKITTI` dataset and visualize it. You can use any of the other dataset to load data. However, you must understand that the parameters may vary for each dataset.\n", "\n", "We will read the dataset by specifying its path and then get all splits." @@ -129,6 +131,14 @@ "pipeline.run_train()" ] }, + { + "cell_type": "markdown", + "id": "bc87e93f", + "metadata": {}, + "source": [ + "The training checkpoints are saved in: `pipeline.main_log_dir` (default path is: “./logs/Model_Dataset/“). You can use them for testing and inference." + ] + }, { "cell_type": "markdown", "id": "ad40d3ea", @@ -136,9 +146,7 @@ "source": [ "## Running a test\n", "\n", - "Next, we will evaluate the trained model on the test split.\n", - "\n", - "We can call the `run_test()` method, and it will run inference on the test split." + "Next, we will evaluate the trained model on the test split by calling the `run_test()` method:" ] }, { From 06ca7ecb5ee805147dbb5cb657e0be778e50da61 Mon Sep 17 00:00:00 2001 From: Sanskar Agrawal Date: Tue, 3 May 2022 21:07:47 +0530 Subject: [PATCH 12/13] small changes --- .../notebook/Inference_on_a_custom_data.ipynb | 47 +++++-------------- .../notebook/reading_a_config_file.ipynb | 23 ++++----- .../tutorial/notebook/reading_a_dataset.ipynb | 10 ++-- .../train_ss_model_using_pytorch.ipynb | 47 +++---------------- .../train_ss_model_using_tensorflow.ipynb | 39 +++++++-------- 5 files changed, 48 insertions(+), 118 deletions(-) diff --git a/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb b/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb index 19f0b34b2..5b4838eb1 100644 --- a/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb +++ b/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb @@ -15,12 +15,11 @@ "\n", "1. Download the data *weights* file;\n", "2. Set up `torch` and `numpy` libraries;\n", - "3. Create a variable holding the path to the *weights* file;\n", - "4. Create a `dataset` object and extract a sample from its `'test'` split;\n", - "5. Create and initialize `model` and `pipeline` objects;\n", - "6. Restore the `model` with data from the *weights* file;\n", - "7. Convert the custom pointcloud data into the specified format;\n", - "8. Run an inference on the sample data.\n", + "3. Create a `dataset` object and extract a sample from its `'test'` split;\n", + "4. Create and initialize `model` and `pipeline` objects;\n", + "5. Restore the `model` with data from the *weights* file;\n", + "6. Convert the custom pointcloud data into the specified format;\n", + "7. Run an inference on the sample data.\n", "\n", "\n", "> **Note:** We will be using a sample `RandLANet` `SemanticKITTI` weight file which we need to:\n", @@ -35,21 +34,13 @@ "> For other model/dataset weight files, please check out https://github.com/isl-org/Open3D-ML#semantic-segmentation-1\n", "\n", "\n", - "An inference predicts the results based on the trained model. **Inference always depends on a prior step - _training_**.\n", + "An inference predicts the results based on the trained model.\n", "\n", "> **Please see the [Training a semantic segmentation model using PyTorch](train_ss_model_using_pytorch.ipynb) and [Training a semantic segmentation model using TensorFlow](train_ss_model_using_tensorflow.ipynb) for training tutorials.**\n", "\n", - "To recap the model training process:\n", - "\n", - "1. define a dataset;\n", - "2. create a training split part of the data;\n", - "3. define a model;\n", - "4. create a `SemanticSegmentation` `pipeline` object;\n", - "5. execute the `pipeline.run_train()` method which runs the training process.\n", - "\n", "While training, the model saves the checkpoint files every few epochs, in the *logs* directory. We use these trained weights to restore the model for inference.\n", "\n", - "Our first step in inference on a custom data implementation is to import `open3d.ml` and `numpy` libraries: \n" + "Our first step in inference on a custom data implementation is to import `open3d.ml` and `numpy` libraries:\n" ] }, { @@ -87,7 +78,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now, we define a `dataset` object identical to how it was done in our previous *Training a semantic segmentation model* tutorials:" + "Now, we define a `dataset`, `model`, and `pipeline` objects identical to how it was done in our previous *Training a semantic segmentation model* tutorials:" ] }, { @@ -101,24 +92,8 @@ " cache_dir='./logs/cache',\n", " training_split=['00'],\n", " validation_split=['01'],\n", - " test_split=['01'])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Then, initialize `model` and `pipeline` objects.\n", + " test_split=['01'])\n", "\n", - "For the `model` object, we have to specify the dimension of the features in the input data ( in the `in_channels` parameter):" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ "# Initializing the model and pipeline\n", "model = ml3d.models.RandLANet(in_channels=3)\n", "pipeline = ml3d.pipelines.SemanticSegmentation(model)" @@ -228,7 +203,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "Python 3", "language": "python", "name": "python3" }, @@ -242,7 +217,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.12" + "version": "3.7.6" } }, "nbformat": 4, diff --git a/docs/tutorial/notebook/reading_a_config_file.ipynb b/docs/tutorial/notebook/reading_a_config_file.ipynb index 958d6df32..45724c3b3 100644 --- a/docs/tutorial/notebook/reading_a_config_file.ipynb +++ b/docs/tutorial/notebook/reading_a_config_file.ipynb @@ -11,10 +11,10 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In other examples, we examine the following snippet of code which creates a dataset:\n", + "Look at the following snippet of code which creates a dataset:\n", "\n", "```py\n", - "# Read a dataset by specifying the path. We are also providing the cache directory and training split.\n", + "# Read a dataset by specifying the path. We can pass other arguments like cache directory and training split.\n", "\n", "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/',\n", " cache_dir='./logs/cache',\n", @@ -25,7 +25,7 @@ "\n", "In the code above, the `dataset` object is created by explicitly passing dataset-specific parameters into `ml3d.datasets.SemanticKITTI()` method.\n", "\n", - "Instead of passing a bunch of parameters to a function call as part of `dataset` object creation, we can supply `dataset` information from a specific *config* file. Each *config* file contains parameters for for a dataset, model and pipeline.\n", + "Instead of passing a bunch of parameters to a function call, we can supply `dataset` information from a specific *config* file. Each *config* file contains parameters for for a dataset, model and pipeline.\n", "\n", "\n", ">***Config* files pass information into Open3D-ML in YAML format.**\n", @@ -59,9 +59,9 @@ "Here, we import two modules:\n", " \n", " 1. `utils` - Open3D-ML utilities\n", - " 2. `ml3d` - Open3D-ML Torch API library\n", + " 2. `ml3d` - Open3D-ML PyTorch API library\n", "\n", - "Now, we'll create *config*-specific objects:" + "Now, we'll create *config* object:" ] }, { @@ -101,7 +101,7 @@ "source": [ "`vars(cfg)` returns the three dictionaries: `dataset`, `model`, and `pipeline`.\n", "\n", - "Now, let's explore them. The first one to look at is the `cfg.dataset`:" + "Now, let's explore them. The first one is the `cfg.dataset`:" ] }, { @@ -167,13 +167,6 @@ "cfg.model['sub_sampling_ratio']" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`cfg.pipeline` is the last dictionary item of the `cfg` object:" - ] - }, { "cell_type": "code", "execution_count": null, @@ -275,7 +268,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "Python 3", "language": "python", "name": "python3" }, @@ -289,7 +282,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.12" + "version": "3.7.6" } }, "nbformat": 4, diff --git a/docs/tutorial/notebook/reading_a_dataset.ipynb b/docs/tutorial/notebook/reading_a_dataset.ipynb index 39b1db6ec..60b80bce1 100644 --- a/docs/tutorial/notebook/reading_a_dataset.ipynb +++ b/docs/tutorial/notebook/reading_a_dataset.ipynb @@ -13,7 +13,7 @@ "source": [ "In this tutorial, we are going to learn how to read Open3D-ML datasets.\n", "\n", - "You can use any dataset available in the `ml3d.datasets` dataset namespace. For this example, we will use the `SemanticKITTI` dataset. You can use any of the other datasets to load data. However, you must understand that the parameters may vary for each dataset.\n", + "You can use any dataset available in the `ml3d.datasets` namespace. For this example, we will use the `SemanticKITTI` dataset. You can use any of the other datasets to load data. However, you must understand that the parameters may vary for each dataset.\n", "\n", "In order to read a dataset in this example, we will supply the following parameter variables:\n", "\n", @@ -29,7 +29,7 @@ "\n", "## Creating a global dataset object\n", "\n", - "First, we declare the `ml3d` object to be of `open3d.ml.torch` library:" + "First, we import the Open3D-ML PyTorch library:" ] }, { @@ -46,7 +46,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We then create a global `dataset` object, initializing it with *path, cache directory*, and *splits*. This `dataset` consists of all files specified in the `dataset_path=''` variable:" + "We then create a `dataset` object, initializing it with *path, cache directory*, and *splits*. This `dataset` can read all the files inside `dataset_path` directory:" ] }, { @@ -254,7 +254,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "Python 3", "language": "python", "name": "python3" }, @@ -268,7 +268,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.12" + "version": "3.7.6" } }, "nbformat": 4, diff --git a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb index 34ea519f2..327887f77 100644 --- a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb +++ b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb @@ -2,7 +2,6 @@ "cells": [ { "cell_type": "markdown", - "id": "48589547", "metadata": {}, "source": [ "# Training a semantic segmentation model using PyTorch" @@ -10,7 +9,6 @@ }, { "cell_type": "markdown", - "id": "49b68041", "metadata": {}, "source": [ "In this tutorial, we will learn how to train a semantic segmentation model using PyTorch.\n", @@ -24,14 +22,13 @@ "At a high-level, we will:\n", "\n", "- Read a dataset and create a *'training'* split. For this example, we will use `SemanticKITTI` dataset.\n", - "- Train a model. We will train a `RandLANet` model using the *'training'* split.\n", + "- Train a model. We will train a `RandLANet` model on the *'training'* split.\n", "- Run a test on a *'test'* split to evaluate the model.\n", "- Run an inference on a custom pointcloud.\n" ] }, { "cell_type": "markdown", - "id": "0f51bcdd", "metadata": {}, "source": [ "## Reading a dataset" @@ -39,7 +36,6 @@ }, { "cell_type": "markdown", - "id": "b69972ae", "metadata": {}, "source": [ "Downloading scripts are available in: `Open3D-ML/scripts/download_datasets`\n", @@ -52,7 +48,6 @@ { "cell_type": "code", "execution_count": null, - "id": "d8090925", "metadata": {}, "outputs": [], "source": [ @@ -79,7 +74,6 @@ }, { "cell_type": "markdown", - "id": "b965946b", "metadata": {}, "source": [ "Now that you have visualized the dataset for training, let us train the model." @@ -87,20 +81,18 @@ }, { "cell_type": "markdown", - "id": "bb442bcd", "metadata": {}, "source": [ "## Training a model\n", "\n", - "Before you train a model, you must decide which model you want to use. To use models, you must import the model from `open3d.ml.torch.models`.\n", + "First, import the desired model from `open3d.ml.torch.models`.\n", "\n", - "After you load a dataset, you can initialize any model and then train the model. The following example shows how you can train a model:" + "After you load a dataset, you can initialize any model and then train the model. The following example shows how you can train RandLANet:" ] }, { "cell_type": "code", "execution_count": null, - "id": "989a6579", "metadata": {}, "outputs": [], "source": [ @@ -119,7 +111,7 @@ " validation_split=['01'],\n", " test_split=['01'])\n", "\n", - "# Initialize the RandLANet model with three layers.\n", + "# Initialize the RandLANet model.\n", "model = RandLANet(in_channels=3)\n", "pipeline = SemanticSegmentation(model=model,\n", " dataset=dataset,\n", @@ -133,7 +125,6 @@ }, { "cell_type": "markdown", - "id": "bc87e93f", "metadata": {}, "source": [ "The training checkpoints are saved in: `pipeline.main_log_dir` (default path is: “./logs/Model_Dataset/“). You can use them for testing and inference." @@ -141,7 +132,6 @@ }, { "cell_type": "markdown", - "id": "ad40d3ea", "metadata": {}, "source": [ "## Running a test\n", @@ -152,7 +142,6 @@ { "cell_type": "code", "execution_count": null, - "id": "9269a2df", "metadata": {}, "outputs": [], "source": [ @@ -161,7 +150,6 @@ }, { "cell_type": "markdown", - "id": "5404cae2", "metadata": {}, "source": [ "## Running an inference\n", @@ -174,30 +162,10 @@ { "cell_type": "code", "execution_count": null, - "id": "2e6aad28", "metadata": {}, "outputs": [], "source": [ - "# Training Semantic Segmentation Model using PyTorch\n", - "\n", - "# Import torch and the model to use for training\n", - "import open3d.ml.torch as ml3d\n", - "from open3d.ml.torch.models import RandLANet\n", - "from open3d.ml.torch.pipelines import SemanticSegmentation\n", - "\n", - "dataset = ml3d.datasets.SemanticKITTI(dataset_path='SemanticKITTI/',\n", - " cache_dir='./logs/cache',\n", - " training_split=['00'],\n", - " validation_split=['01'],\n", - " test_split=['01'])\n", - "model = RandLANet(in_channels=3)\n", - "pipeline = SemanticSegmentation(model=model,\n", - " dataset=dataset,\n", - " max_epoch=3,\n", - " optimizer={'lr': 0.001},\n", - " num_workers=0)\n", - "\n", - "# Get data from the SemanticKITTI dataset using the \"train\" split\n", + "# Get data from the SemanticKITTI dataset using the \"test\" split\n", "train_split = dataset.get_split(\"test\")\n", "data = train_split.get_data(0)\n", "\n", @@ -211,7 +179,6 @@ { "cell_type": "code", "execution_count": null, - "id": "b85be785", "metadata": {}, "outputs": [], "source": [] @@ -219,7 +186,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "Python 3", "language": "python", "name": "python3" }, @@ -233,7 +200,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.12" + "version": "3.7.6" } }, "nbformat": 4, diff --git a/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb b/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb index 632a491ab..937b9c9b3 100644 --- a/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb +++ b/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb @@ -2,7 +2,6 @@ "cells": [ { "cell_type": "markdown", - "id": "c8d52012", "metadata": {}, "source": [ "# Training a semantic segmentation model using TensorFlow" @@ -10,7 +9,6 @@ }, { "cell_type": "markdown", - "id": "04ed6952", "metadata": {}, "source": [ "In this tutorial, we will learn how to train a semantic segmentation model using `TensorFlow` in a Jupyter Notebook. We assume that you are familiar with Jupyter Notebook and have created a folder *notebooks* in a folder that is relative to *ml3d*.\n", @@ -23,19 +21,20 @@ "\n", "At a high-level, we will:\n", "\n", - "- Read a dataset and get a training split. For this example, we will use `SemanticKITTI` dataset.\n", - "- Run a pre-trained model. For this example, we will use the `RandLANet` model.\n", - "- Train a model. We will train a model using the `SemanticKITTI` dataset and `RandLANet` model.\n", - "- Run an inference and run a test. We will run an inference using the 'training' split that use a pointcloud and display a result. However, a test is run on a pre-defined test set rather than a pass pointcloud." + "- Read a dataset and create a training split. For this example, we will use `SemanticKITTI` dataset.\n", + "- Train a model. We will train a `RandLANet` model on the training split.\n", + "- Run a test on a *'test'* split to evaluate the model.\n", + "- Run an inference on a custom pointcloud." ] }, { "cell_type": "markdown", - "id": "5e33bf9d", "metadata": {}, "source": [ "## Reading a dataset\n", "\n", + "Downloading scripts are available in: `Open3D-ML/scripts/download_datasets`\n", + "\n", "You can use any dataset available in the `ml3d.datasets` dataset namespace. For this example, we will use the `SemanticKITTI` dataset and visualize it. You can use any of the other dataset to load data. However, you must understand that the parameters may vary for each dataset.\n", "\n", "We will read the dataset by specifying its path and then get all splits." @@ -44,7 +43,6 @@ { "cell_type": "code", "execution_count": null, - "id": "b4e344db", "metadata": {}, "outputs": [], "source": [ @@ -68,7 +66,6 @@ }, { "cell_type": "markdown", - "id": "1138db9e", "metadata": {}, "source": [ "Now that you have visualized the dataset for training, let us train the model." @@ -76,7 +73,6 @@ }, { "cell_type": "markdown", - "id": "9653bd32", "metadata": {}, "source": [ "## Training a model\n", @@ -87,7 +83,6 @@ { "cell_type": "code", "execution_count": null, - "id": "c220cfb6", "metadata": {}, "outputs": [], "source": [ @@ -103,20 +98,18 @@ }, { "cell_type": "markdown", - "id": "12d8271a", "metadata": {}, "source": [ "Refer to [this link](https://www.tensorflow.org/guide/gpu#limiting_gpu_memory_growth) for more details.\n", "\n", - "Before you train a model, you must decide which model you want to use. For this example, we will use `RandLANet` model. To use models, you must import the model from `open3d.ml.tf.models`.\n", + "First, import the desired model from `open3d.ml.torch.models`.\n", "\n", - "After you load a dataset, you can initialize any model and then train the model. The following example shows how you can train a model:" + "After you load a dataset, you can initialize any model and then train the model. The following example shows how you can train RandLANet:" ] }, { "cell_type": "code", "execution_count": null, - "id": "caeea522", "metadata": {}, "outputs": [], "source": [ @@ -147,7 +140,13 @@ }, { "cell_type": "markdown", - "id": "9648d5da", + "metadata": {}, + "source": [ + "The training checkpoints are saved in: `pipeline.main_log_dir` (default path is: “./logs/Model_Dataset/“). You can use them for testing and inference." + ] + }, + { + "cell_type": "markdown", "metadata": {}, "source": [ "## Running a test\n", @@ -160,7 +159,6 @@ { "cell_type": "code", "execution_count": null, - "id": "7a81f7f2", "metadata": {}, "outputs": [], "source": [ @@ -170,7 +168,6 @@ }, { "cell_type": "markdown", - "id": "515a986a", "metadata": {}, "source": [ "## Running an inference\n", @@ -183,7 +180,6 @@ { "cell_type": "code", "execution_count": null, - "id": "3ec86a88", "metadata": {}, "outputs": [], "source": [ @@ -201,7 +197,6 @@ { "cell_type": "code", "execution_count": null, - "id": "bcf019b3", "metadata": {}, "outputs": [], "source": [] @@ -209,7 +204,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "Python 3", "language": "python", "name": "python3" }, @@ -223,7 +218,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.12" + "version": "3.7.6" } }, "nbformat": 4, From b248113a9558674efd4fe8e46089cc5a9f6b9000 Mon Sep 17 00:00:00 2001 From: Asapanna Rakesh <45640029+INF800@users.noreply.github.com> Date: Tue, 31 May 2022 21:26:41 +0530 Subject: [PATCH 13/13] pass through gramarly (#539) --- .gitignore | 1 + .../notebook/Inference_on_a_custom_data.ipynb | 14 +++++----- .../notebook/reading_a_config_file.ipynb | 8 +++--- .../tutorial/notebook/reading_a_dataset.ipynb | 14 +++++----- .../train_ss_model_using_pytorch.ipynb | 26 +++++++++++++----- .../train_ss_model_using_tensorflow.ipynb | 27 ++++++++++++++----- 6 files changed, 60 insertions(+), 30 deletions(-) diff --git a/.gitignore b/.gitignore index fa8dd8468..c51478ef6 100644 --- a/.gitignore +++ b/.gitignore @@ -26,3 +26,4 @@ test_kpconv/ kernels/ **/.fuse* train_log/ +*.ipynb_checkpoints \ No newline at end of file diff --git a/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb b/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb index 5b4838eb1..11793198b 100644 --- a/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb +++ b/docs/tutorial/notebook/Inference_on_a_custom_data.ipynb @@ -59,7 +59,7 @@ "source": [ "We then create a checkpoint path pointing to the weights file we downloaded (generated at the end of the Training stage):\n", "\n", - "(You can download any other weights using a link from model zoo (collection of weights for all combinations of model and dataset): https://github.com/isl-org/Open3D-ML#semantic-segmentation-1 )" + "(You can download any other weights using a link from the model zoo (collection of weights for all combinations of model and dataset): https://github.com/isl-org/Open3D-ML#semantic-segmentation-1 )" ] }, { @@ -120,7 +120,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now, lets query the first pointcloud from the `test` split." + "Now, let us query the first pointcloud from the `test` split." ] }, { @@ -155,13 +155,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "For inference on custom data, you can convert your pointcloud into this format:\n", + "For inference on custom data, you can convert your point cloud into this format:\n", "\n", "**Dictionary with keys {'point', 'feat', 'label'}**\n", "\n", "If you already have the *ground truth labels*, you can add them to data to get accuracy and IoU (Intersection over Union). Otherwise, pass labels as `None`.\n", "\n", - "And now - the main topic of our tutorial - running an inference on the test data. You can call the `run_inference()` method with your data, - it will print *accuracy per class* and *Intersection over Union (IoU)* metrics. The last entry in the list is *mean accuracy* and *mean IoU*:" + "And now - the main topic of our tutorial - running inference on the test data. You can call the `run_inference()` method with your data, - it will print *accuracy per class* and *Intersection over Union (IoU)* metrics. The last entry in the list is *mean accuracy* and *mean IoU*:" ] }, { @@ -180,7 +180,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The `results` object will return dictionary of predicted labels and predicted probabilities per point:" + "The `results` object will return a dictionary of predicted labels and predicted probabilities per point:" ] }, { @@ -203,7 +203,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -217,7 +217,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.6" + "version": "3.10.4" } }, "nbformat": 4, diff --git a/docs/tutorial/notebook/reading_a_config_file.ipynb b/docs/tutorial/notebook/reading_a_config_file.ipynb index 45724c3b3..7789b0d34 100644 --- a/docs/tutorial/notebook/reading_a_config_file.ipynb +++ b/docs/tutorial/notebook/reading_a_config_file.ipynb @@ -25,7 +25,7 @@ "\n", "In the code above, the `dataset` object is created by explicitly passing dataset-specific parameters into `ml3d.datasets.SemanticKITTI()` method.\n", "\n", - "Instead of passing a bunch of parameters to a function call, we can supply `dataset` information from a specific *config* file. Each *config* file contains parameters for for a dataset, model and pipeline.\n", + "Instead of passing a bunch of parameters to a function call, we can supply `dataset` information from a specific *config* file. Each *config* file contains parameters for a dataset, model and pipeline.\n", "\n", "\n", ">***Config* files pass information into Open3D-ML in YAML format.**\n", @@ -246,7 +246,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Likewise, to extract information from a `label_to_names` property which maps labels to the objects names, we call:" + "Likewise, to extract information from a `label_to_names` property which maps labels to the object names, we call:" ] }, { @@ -268,7 +268,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -282,7 +282,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.6" + "version": "3.10.4" } }, "nbformat": 4, diff --git a/docs/tutorial/notebook/reading_a_dataset.ipynb b/docs/tutorial/notebook/reading_a_dataset.ipynb index 60b80bce1..21f0ada90 100644 --- a/docs/tutorial/notebook/reading_a_dataset.ipynb +++ b/docs/tutorial/notebook/reading_a_dataset.ipynb @@ -11,11 +11,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In this tutorial, we are going to learn how to read Open3D-ML datasets.\n", + "In this tutorial, we will learn how to read Open3D-ML datasets.\n", "\n", "You can use any dataset available in the `ml3d.datasets` namespace. For this example, we will use the `SemanticKITTI` dataset. You can use any of the other datasets to load data. However, you must understand that the parameters may vary for each dataset.\n", "\n", - "In order to read a dataset in this example, we will supply the following parameter variables:\n", + "To read a dataset in this example, we will supply the following parameter variables:\n", "\n", "- Dataset path (`dataset_path`)\n", "- Cache directory (`cache_dir`)\n", @@ -83,7 +83,7 @@ "- `validation_split=['01']` points to `'SemanticKITTI/dataset/sequences/01/'`\n", "- `test_split=['01']` points to `'SemanticKITTI/dataset/sequences/01/'`\n", "\n", - "> Note: **dataset split directories usually contain numerous pointcloud files.** In our example we included only one pointcloud file for extra speed and convenience.\n", + "> Note: **dataset split directories usually contain numerous point cloud files.** In our example we included only one point cloud file for extra speed and convenience.\n", "\n", "## Creating dataset split objects to query the data\n", "\n", @@ -207,7 +207,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "- The **`'point'`** key value contains a set of 3D points / coordinates - X, Y, and Z:\n", + "- The **`'point'`** key value contains a set of 3D points/coordinates - X, Y, and Z:\n", "\n", "![dataset_coordinates](https://user-images.githubusercontent.com/93158890/162549410-6369cbd0-b835-4216-ba54-945e3f591395.jpg)\n", "\n", @@ -217,7 +217,7 @@ "\n", "### Querying dataset splits for attributes\n", "\n", - "We can also extract corresponding pointcloud information:" + "We can also extract corresponding point cloud information:" ] }, { @@ -254,7 +254,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -268,7 +268,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.6" + "version": "3.10.4" } }, "nbformat": 4, diff --git a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb index 327887f77..28ce55ca8 100644 --- a/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb +++ b/docs/tutorial/notebook/train_ss_model_using_pytorch.ipynb @@ -2,6 +2,7 @@ "cells": [ { "cell_type": "markdown", + "id": "9bc05fec", "metadata": {}, "source": [ "# Training a semantic segmentation model using PyTorch" @@ -9,6 +10,7 @@ }, { "cell_type": "markdown", + "id": "f25922b0", "metadata": {}, "source": [ "In this tutorial, we will learn how to train a semantic segmentation model using PyTorch.\n", @@ -19,16 +21,17 @@ "pip install -r requirements-torch-cuda.txt\n", "```\n", "\n", - "At a high-level, we will:\n", + "At a high level, we will:\n", "\n", - "- Read a dataset and create a *'training'* split. For this example, we will use `SemanticKITTI` dataset.\n", + "- Read a dataset and create a *'training'* split. For this example, we will use the `SemanticKITTI` dataset.\n", "- Train a model. We will train a `RandLANet` model on the *'training'* split.\n", "- Run a test on a *'test'* split to evaluate the model.\n", - "- Run an inference on a custom pointcloud.\n" + "- Run an inference on a custom point cloud.\n" ] }, { "cell_type": "markdown", + "id": "76e89bac", "metadata": {}, "source": [ "## Reading a dataset" @@ -36,11 +39,12 @@ }, { "cell_type": "markdown", + "id": "fdff079f", "metadata": {}, "source": [ "Downloading scripts are available in: `Open3D-ML/scripts/download_datasets`\n", "\n", - "You can use any dataset available in the `ml3d.datasets` dataset namespace. For this example, we will use the `SemanticKITTI` dataset and visualize it. You can use any of the other dataset to load data. However, you must understand that the parameters may vary for each dataset.\n", + "You can use any dataset available in the `ml3d.datasets` dataset namespace. Here, we will use the `SemanticKITTI` dataset and visualize it. You can use any of the other datasets to load data. However, you must understand that the parameters may vary for each dataset.\n", "\n", "We will read the dataset by specifying its path and then get all splits." ] @@ -48,6 +52,7 @@ { "cell_type": "code", "execution_count": null, + "id": "508129cd", "metadata": {}, "outputs": [], "source": [ @@ -74,6 +79,7 @@ }, { "cell_type": "markdown", + "id": "f34dca8f", "metadata": {}, "source": [ "Now that you have visualized the dataset for training, let us train the model." @@ -81,6 +87,7 @@ }, { "cell_type": "markdown", + "id": "e2dc77ea", "metadata": {}, "source": [ "## Training a model\n", @@ -93,6 +100,7 @@ { "cell_type": "code", "execution_count": null, + "id": "b7c1ff9a", "metadata": {}, "outputs": [], "source": [ @@ -125,6 +133,7 @@ }, { "cell_type": "markdown", + "id": "1a0b0bc9", "metadata": {}, "source": [ "The training checkpoints are saved in: `pipeline.main_log_dir` (default path is: “./logs/Model_Dataset/“). You can use them for testing and inference." @@ -132,6 +141,7 @@ }, { "cell_type": "markdown", + "id": "6f4fa94c", "metadata": {}, "source": [ "## Running a test\n", @@ -142,6 +152,7 @@ { "cell_type": "code", "execution_count": null, + "id": "80f461dd", "metadata": {}, "outputs": [], "source": [ @@ -150,6 +161,7 @@ }, { "cell_type": "markdown", + "id": "3a385892", "metadata": {}, "source": [ "## Running an inference\n", @@ -162,6 +174,7 @@ { "cell_type": "code", "execution_count": null, + "id": "d39c5381", "metadata": {}, "outputs": [], "source": [ @@ -179,6 +192,7 @@ { "cell_type": "code", "execution_count": null, + "id": "d4039f9d", "metadata": {}, "outputs": [], "source": [] @@ -186,7 +200,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -200,7 +214,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.6" + "version": "3.10.4" } }, "nbformat": 4, diff --git a/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb b/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb index 937b9c9b3..b8070ba80 100644 --- a/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb +++ b/docs/tutorial/notebook/train_ss_model_using_tensorflow.ipynb @@ -2,6 +2,7 @@ "cells": [ { "cell_type": "markdown", + "id": "91802780", "metadata": {}, "source": [ "# Training a semantic segmentation model using TensorFlow" @@ -9,6 +10,7 @@ }, { "cell_type": "markdown", + "id": "6c42d107", "metadata": {}, "source": [ "In this tutorial, we will learn how to train a semantic segmentation model using `TensorFlow` in a Jupyter Notebook. We assume that you are familiar with Jupyter Notebook and have created a folder *notebooks* in a folder that is relative to *ml3d*.\n", @@ -19,23 +21,24 @@ "pip install -r requirements-tensorflow.txt\n", "```\n", "\n", - "At a high-level, we will:\n", + "At a high level, we will:\n", "\n", - "- Read a dataset and create a training split. For this example, we will use `SemanticKITTI` dataset.\n", + "- Read a dataset and create a training split. Here, we will use the `SemanticKITTI` dataset.\n", "- Train a model. We will train a `RandLANet` model on the training split.\n", "- Run a test on a *'test'* split to evaluate the model.\n", - "- Run an inference on a custom pointcloud." + "- Run an inference on a custom point cloud." ] }, { "cell_type": "markdown", + "id": "f788f8d5", "metadata": {}, "source": [ "## Reading a dataset\n", "\n", "Downloading scripts are available in: `Open3D-ML/scripts/download_datasets`\n", "\n", - "You can use any dataset available in the `ml3d.datasets` dataset namespace. For this example, we will use the `SemanticKITTI` dataset and visualize it. You can use any of the other dataset to load data. However, you must understand that the parameters may vary for each dataset.\n", + "You can use any dataset available in the `ml3d.datasets` dataset namespace. For this example, we will use the `SemanticKITTI` dataset and visualize it. You can use any of the other datasets to load data. However, you must understand that the parameters may vary for each dataset.\n", "\n", "We will read the dataset by specifying its path and then get all splits." ] @@ -43,6 +46,7 @@ { "cell_type": "code", "execution_count": null, + "id": "8714f23e", "metadata": {}, "outputs": [], "source": [ @@ -66,6 +70,7 @@ }, { "cell_type": "markdown", + "id": "9d46e91b", "metadata": {}, "source": [ "Now that you have visualized the dataset for training, let us train the model." @@ -73,6 +78,7 @@ }, { "cell_type": "markdown", + "id": "cb4d73cd", "metadata": {}, "source": [ "## Training a model\n", @@ -83,6 +89,7 @@ { "cell_type": "code", "execution_count": null, + "id": "90c55d8d", "metadata": {}, "outputs": [], "source": [ @@ -98,6 +105,7 @@ }, { "cell_type": "markdown", + "id": "9ef4c76d", "metadata": {}, "source": [ "Refer to [this link](https://www.tensorflow.org/guide/gpu#limiting_gpu_memory_growth) for more details.\n", @@ -110,6 +118,7 @@ { "cell_type": "code", "execution_count": null, + "id": "66b92b2d", "metadata": {}, "outputs": [], "source": [ @@ -140,6 +149,7 @@ }, { "cell_type": "markdown", + "id": "0e99379b", "metadata": {}, "source": [ "The training checkpoints are saved in: `pipeline.main_log_dir` (default path is: “./logs/Model_Dataset/“). You can use them for testing and inference." @@ -147,6 +157,7 @@ }, { "cell_type": "markdown", + "id": "001c40dc", "metadata": {}, "source": [ "## Running a test\n", @@ -159,6 +170,7 @@ { "cell_type": "code", "execution_count": null, + "id": "9b936b96", "metadata": {}, "outputs": [], "source": [ @@ -168,6 +180,7 @@ }, { "cell_type": "markdown", + "id": "d0b482d8", "metadata": {}, "source": [ "## Running an inference\n", @@ -180,6 +193,7 @@ { "cell_type": "code", "execution_count": null, + "id": "c3c7dbc9", "metadata": {}, "outputs": [], "source": [ @@ -197,6 +211,7 @@ { "cell_type": "code", "execution_count": null, + "id": "633dab8c", "metadata": {}, "outputs": [], "source": [] @@ -204,7 +219,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -218,7 +233,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.6" + "version": "3.10.4" } }, "nbformat": 4,