diff --git a/Text-Summarizer-Browser-Plugin/.gitignore b/Text-Summarizer-Browser-Plugin/.gitignore
index 6e22444..3daffc1 100644
--- a/Text-Summarizer-Browser-Plugin/.gitignore
+++ b/Text-Summarizer-Browser-Plugin/.gitignore
@@ -35,6 +35,7 @@ MANIFEST
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
+*.txt
# Unit test / coverage reports
htmlcov/
@@ -78,6 +79,7 @@ target/
# Jupyter Notebook
.ipynb_checkpoints
**/*.ipynb_checkpoints/
+models
# IPython
profile_default/
diff --git a/Text-Summarizer-Browser-Plugin/.python-version b/Text-Summarizer-Browser-Plugin/.python-version
new file mode 100644
index 0000000..e4fba21
--- /dev/null
+++ b/Text-Summarizer-Browser-Plugin/.python-version
@@ -0,0 +1 @@
+3.12
diff --git a/Text-Summarizer-Browser-Plugin/README.md b/Text-Summarizer-Browser-Plugin/README.md
index 5476bb6..00b257f 100644
--- a/Text-Summarizer-Browser-Plugin/README.md
+++ b/Text-Summarizer-Browser-Plugin/README.md
@@ -1,90 +1,127 @@
# Text summarizer browser Plugin Sample
-A plug-and-play Chrome extension seamlessly integrates with Flask and leverages an OpenVINO backend for fast and efficient summarization of webpages (via URL) and PDFs (via upload). Powered by LangChain tools, it handles advanced tasks like text splitting and vectorstore management to deliver accurate and meaningful summaries.
-
-## How it Works
-
+A plug-and -play Chrome extension seamlessly integrates with Flask and leverages an OpenVINO backend for fast and efficient summarization of webpages(via URL) and PDFs(via upload). Powered by LangChain tools, it handles advanced tasks like text splitting and vectorstore management to deliver accurate and meaningful summaries.
+
+This Chrome extension acts as the immediate user gateway, offering a minimalist interface right in the browser's toolbar that lets users instantly trigger summarization. It's designed to efficiently capture the required input, whether it's a webpage's URL or content from an uploaded PDF, and securely send this data to the Flask server for further processing.
+
+# Contents
+- [Sample Workflow](./Readme.md#sample-workflow)
+- [Prerequisites](./Readme.md#prerequisites)
+- [Sample Structure](./Readme.md#sample-structure)
+- [Run the `Text Summarizer Browser Plugin` Sample](./Readme.md#run-the-text-summarizer-browser-plugin-sample)
+ - [Setup the environment with uv](./Readme.md#setup-the-environment-with-uv)
+ - [Load this `Text Summarizer` extension](./Readme.md#load-this-text=summarizer-extension)
+ - [Contens of `extension` directory](./Readme.md#contents-of-extension-directory)
+ - [Load an unpacked extension](./Readme.md#load-an-unpacked-extension)
+ - [Pin the extension](./Readme.md#pin-the-extension)
+ - [Run the sample](./Readme.md#run-the-sample)
+ - [Steps to follow via Jupyter Notebook](./Readme.md#steps-to-follow-via-jupyter-notebook)
+ - [Steps to follow via Terminal](./Readme.md#steps-to-follow-via-terminal)
+ - [Interact with `Text Summarizer Browser Plugin`](./Readme.md#interact-with-text-summarizer-browser-plugin)
+
+
+# Sample Workflow
+
+- This architecture is split into two main components: a lightweight **Chrome extension** and a powerful **Flask server**.
+- The **Chrome extension** serves as the user's direct interface, enabling them to simply click a button to initiate summarization of the current webpage or to upload a PDF directly from their browser.
+- Once activated, it securely sends the webpage URL or the uploaded PDF content to the Flask server.
+- This **Flask server**, acting as the intelligent backend, processes the request, orchestrates the summarization using the OpenVINO backend and LangChain tools, and then sends the concise summary back to the extension for display.
+- This clear separation of concerns ensures a responsive user experience within the browser while offloading heavy computational tasks to a dedicated and scalable server.
+
-## Sample Structure
-
-The directory contains:
-- **backend:** Includes `code.py` and `server.py` for processing text from webpages or PDFs and managing Flask-related operations.
-- **extension:** Contains `manifest.json` for the Chrome extension along with `popup.html`, `popup.js`, and `style.css` for the user interface.
-
-## Prerequisites
+## Pre-requisites
| Optimized for | Description |
| :------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| OS | Windows 11 64-bit (22H2, 23H2) and newer or Ubuntu* 22.04 64-bit (with Linux kernel 6.6+) and newer |
-| Hardware | Intel® Core™ Ultra Processors |
-| Software | 1. [Intel® GPU drivers from Intel® Arc™ & Iris® Xe Graphics for Windows](https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html) or [Linux GPU drivers](https://dgpu-docs.intel.com/driver/client/overview.html)
2. NPU(Optional): [Intel® NPU Driver for Windows](https://www.intel.com/content/www/us/en/download/794734/intel-npu-driver-windows.html) or [Linux NPU Driver](https://github.com/intel/linux-npu-driver/releases) |
+| Hardware | 1. [Intel® GPU drivers from Intel® Arc™ & Iris® Xe Graphics for Windows](https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html) or [Linux GPU drivers](https://dgpu-docs.intel.com/driver/client/overview.html)
2. NPU(Optional): [Intel® NPU Driver for Windows](https://www.intel.com/content/www/us/en/download/794734/intel-npu-driver-windows.html) or [Linux NPU Driver](https://github.com/intel/linux-npu-driver/releases)
3. Intel® Core™ Ultra Processors |
+| Software | 1. [Intel® GPU drivers from Intel® Arc™ & Iris® Xe Graphics for Windows](https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html),
2. [uv](https://docs.astral.sh/uv/) |
| Browsers | [Google Chrome](https://www.google.com/chrome/dr/download/?brand=MRUS&ds_kid=43700079286123654&gad_source=1&gclid=EAIaIQobChMI0J3fybvSigMV5dXCBB1TDARCEAAYASAAEgL36_D_BwE&gclsrc=aw.ds) & [Microsoft Edge](https://www.microsoft.com/en-us/edge/download?form=MA13FJ)
-1. **Install the below necessary tools/packages:**
- - Git
- - [Git for Windows](https://git-scm.com/downloads)
- - Git for Linux
- - For Debian/Ubuntu-based systems:
- ```bash
- sudo apt update && sudo apt -y install git
- ```
- - For RHEL/CentOS-based systems:
- ```bash
- sudo dnf update && sudo dnf -y install git
- ```
- - Miniforge
- - [Miniforge for Windows](https://conda-forge.org/download/)
- - Miniforge for Linux
- Download, install the Miniconda using the below commands.
- ```bash
- wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
- bash Miniforge3-$(uname)-$(uname -m).sh
- ```
- Replace with your actual Miniforge bin folder path and run the cd command to go there. Initialize the conda environment and restart the terminal.
- ```bash
- cd
- ```
- ```bash
- ./conda init
- ```
-
-2. **Create a Conda Environment:**
- - Run the command:
- ```bash
- conda create -n summarizer_plugin python=3.11 libuv
- ```
- ```bash
- conda activate summarizer_plugin
- ```
+## Sample Structure
-3. **Install Dependencies:**
- - Execute:
- ```bash
- pip install -r requirements.txt
- ```
+The directory contains:
+- **backend:** Includes `code.py` and `server.py` for processing text from webpages or PDFs and managing Flask-related operations.
+- **extension:** Contains `manifest.json` for the Chrome extension along with `popup.html`, `popup.js`, and `style.css` for the user interface. These files are used to load an unapcked extension.
+- **`TextSummarizerPlugin.ipynb`:** Notebook which processes text from webpages or PDFs and manages Flask-related operations.
+
+
+## Run the `Text Summarizer Browser Plugin` Sample:
+
+### Setup Environment with `uv`:
+The sample uses [uv](https://docs.astral.sh/uv/) for environment management.
+> Steps to install `uv` are as follows. Refer [this documentation](https://docs.astral.sh/uv/getting-started/installation/) this for more information.
+> **Windows:**
+> ```bash
+> powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
+> ```
+>
+> **Linux:**
+> Use curl to download the script and execute it with sh:
+> ```bash
+> curl -LsSf https://astral.sh/uv/install.sh | sh
+> ```
+>
+> If your system doesn't have curl, you can use wget:
+> ```bash
+> wget -qO- https://astral.sh/uv/install.sh | sh
+> ```
+
+1. In a terminal, navigate to `Text Summarizer Browser Plugin Sample` folder:
+ ```bash
+ cd
+ ```
+2. Sync project dependencies:
+ ```bash
+ uv sync
+ ```
+> **NOTE:** Run the below command if you face any dependency issues:
+> ```bash
+> uv clean
+> ```
+### Load this `Text Summarizer` extension
+
+1. **Contents of `extension` directory**
+ - [`extension` directory](./extension) consists of files, i.e. `manifest.json`, `popup.html`, `popup.js` and `style.css`, used to load an unpacked extension.
+
+2. **Load an unpacked Extension:**
+ - To load an unpacked extension in developer mode:
+ - Go to the Extensions page by entering **chrome://extensions** in a new tab. (By design chrome:// URLs are not linkable.)
+ - Alternatively, **click the Extensions menu puzzle button and select Manage Extensions** at the bottom of the menu.
+ - Or, click the Chrome menu, hover over More Tools, then select Extensions.
+ - Enable **Developer Mode** by clicking the toggle switch next to Developer mode.
+ - Click the **Load unpacked** button and select the `extension directory`.
+ - Refer to [Chrome’s development documentation](https://developer.chrome.com/docs/extensions/get-started/tutorial/hello-world#load-unpacked) for further details.
-4. **Install an ipykernel to select the "summarizer_plugin" environment:**
- ```bash
- python -m ipykernel install --user --name=summarizer_plugin
- ```
+
- >**Note**: Run your terminal as admin to avoid any permission issues.
-
-## Next Steps
+3. **Pin the Extension:**
+
+ Pin your extension to the toolbar for a quick access to your extension.
+
+
+
+
+### Run the Sample
Below are the steps to run the plugin from Jupyter Notebook **OR** Terminal.
-### **Steps to follow via Jupyter Notebook:**
+#### **Steps to follow via Jupyter Notebook:**
Once the environment is created, we can run the plugin via [TextSummarizerPlugin.ipynb](./TextSummarizerPlugin.ipynb). Please follow the below steps to open the jupyter notebook:
-1. Open Jupyter Notebook & run the cells:
+1. Launch Jupyter Notebook:
```
- jupyter notebook
+ uv run jupyter lab
```
-2. Select the **summarizer_plugin** kernel.
+2. Select the default **ipykernel** kernel.
+3. Run all the cells one by one.
+4. The last cell at the end of the notebook launches the flask server.
+5. Once the flask server is running, open Chrome Browser & click on the pinned text-summarizer
+
+OR
-### **Steps to follow via Terminal:**
+#### **Steps to follow via Terminal:**
1. **Download and Convert the Huggingface Model to OpenVINO IR Format:**
- Log in to Huggingface:
```
@@ -95,59 +132,36 @@ Once the environment is created, we can run the plugin via [TextSummarizerPlugin
```
mkdir models
cd models
- optimum-cli export openvino --model meta-llama/Llama-2-7b-chat-hf --weight-format int4 ov_llama_2
- optimum-cli export openvino --model Qwen/Qwen2-7B-Instruct --weight-format int4 ov_qwen7b
+ uv run optimum-cli export openvino --model meta-llama/Llama-2-7b-chat-hf --weight-format int4 ov_llama_2
+ uv run optimum-cli export openvino --model Qwen/Qwen2-7B-Instruct --weight-format int4 ov_qwen7b
```
>**Note**: [Raise access request](https://www.llama.com/llama-downloads) for Llama models as it is a gated repository.
+
-
-
-2. **Load the Extension:**
- - To load an unpacked extension in developer mode:
- - Go to the Extensions page by entering **chrome://extensions** in a new tab. (By design chrome:// URLs are not linkable.)
- - Alternatively, **click the Extensions menu puzzle button and select Manage Extensions** at the bottom of the menu.
- - Or, click the Chrome menu, hover over More Tools, then select Extensions.
- - Enable **Developer Mode** by clicking the toggle switch next to Developer mode.
- - Click the **Load unpacked** button and select the extension directory.
- - Refer to [Chrome’s development documentation](https://developer.chrome.com/docs/extensions/get-started/tutorial/hello-world#load-unpacked) for further details.
-
-
-
-
-
-3. **Pin the Extension:**
-
- Pin your extension to the toolbar to quickly access your extension.
-
-
-
-
-
-
-#### Steps to Run the Plugin
-
-1. **Start the Flask Server:**
- - Navigate to the backend folder:
+2. **Start the Flask Server:**
+ - In one terminal, navigate to the backend folder:
```
- cd ../backend
- python server.py
+ cd ./backend
+ uv run server.py
```
+ This step would launch a flask server in browser extension
-2. **Open the Chrome Browser:**
- - Activate & Pin the loaded extension.
+### Interact with `Text Summarizer Browser Plugin`
+1. **Open Chrome Extension in Browser:**
+ - Click on the pinned extension of this text summarizer plugin.
- Plugin UI looks as follows:
-3. **Select an OpenVINO Model:**
+2. **Select an OpenVINO Model:**
- Choose an OpenVINO IR format model previously converted from Huggingface.
-4. **Interact with the UI:**
+3. **Interact with the UI:**
- Choose either **Web Page** or **PDF** post selecting one of the converted OV models:
@@ -174,5 +188,5 @@ Once the environment is created, we can run the plugin via [TextSummarizerPlugin
-5. **Reload the Page:**
+4. **Reload the Page:**
- Refresh the webpage or re-open the plugin to restart.
diff --git a/Text-Summarizer-Browser-Plugin/TextSummarizerPlugin.ipynb b/Text-Summarizer-Browser-Plugin/TextSummarizerPlugin.ipynb
index b66572e..a76cfc8 100644
--- a/Text-Summarizer-Browser-Plugin/TextSummarizerPlugin.ipynb
+++ b/Text-Summarizer-Browser-Plugin/TextSummarizerPlugin.ipynb
@@ -36,17 +36,6 @@
"## Pre-requisites"
]
},
- {
- "cell_type": "markdown",
- "id": "ee13e251-d11d-4709-8068-29bdf38ed67d",
- "metadata": {},
- "source": [
- "### Install the below necessary tools/packages:\n",
- " - [Git on Windows](https://git-scm.com/downloads)\n",
- " - [Miniforge](https://conda-forge.org/download/)\n",
- " - [Google Chrome for Windows](https://www.google.com/chrome/?brand=OZZY&ds_kid=43700080794581137&gad_source=1&gclid=Cj0KCQiAoae5BhCNARIsADVLzZdwNNB5nIyjZ8OyCzg6h_cCig1eoaYquUSEd7BAigJhTzps1Kxuop8aArE6EALw_wcB&gclsrc=aw.ds)\n"
- ]
- },
{
"cell_type": "markdown",
"id": "c07f0ea0-c374-4048-b907-bf8a9fd43a9b",
@@ -60,46 +49,20 @@
"id": "479ef7d9-b2d0-4b38-9829-f92224ee9998",
"metadata": {},
"source": [
- "Before converting the models & running the plugin, make sure you have followed all the below listed [steps to prepare the environment](./README.md/#prerequisites)\n",
- "- Setup conda environment\n",
- " ```bash\n",
- " conda create -n summarizer_plugin python=3.11 libuv\n",
- " ```\n",
- " ```bash\n",
- " conda activate summarizer_plugin\n",
- " ```\n",
- " ```bash\n",
- " python -m pip install ipykernel tqdm ipywidgets\n",
- " ```\n",
- " ```bash\n",
- " python -m ipykernel install --user --name=summarizer_plugin\n",
- " ```\n",
- " Now choose the `summarizer_plugin` kernel in the notebook.\n",
- "- Installing necessary packages"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "787b4825-ad2d-4b1f-9ae2-d03da418ac29",
- "metadata": {},
- "source": [
- "#### Installing necessary packages"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "25425663-cca9-47d7-a93f-5ec6239de66b",
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "import sys\n",
- "import subprocess\n",
- "import warnings\n",
- "warnings.filterwarnings(\"ignore\")\n",
- "\n",
- "os.system(f\"{sys.executable} -m pip install -r requirements.txt\")"
+ "Before converting the models & running the plugin, make sure you have followed all the below listed [steps to prepare the environment](./Readme.md/#setup-environment-with-uv)\n",
+ "- Install uv by refering to [this documentation](https://docs.astral.sh/uv/getting-started/installation/)\n",
+ "- Sync dependencies\n",
+ " ```bash\n",
+ " uv sync\n",
+ " ```\n",
+ "Load and pin the unpacked extension as mentioned [here](./Readme.md/):\n",
+ "- Navigate to the Extensions page by entering chrome://extensions in a new tab. (By design chrome:// URLs are not linkable.)\n",
+ " - Alternatively, click the Extensions menu puzzle button and select Manage Extensions at the bottom of the menu.\n",
+ " - Or, click the Chrome menu, hover over More Tools, then select Extensions.\n",
+ "- Enable Developer Mode by clicking the toggle switch next to Developer mode.\n",
+ "- Click the Load unpacked button and select the extension directory.\n",
+ "- Refer to Chrome’s development documentation for further details.\n",
+ "- Pin your extension to the toolbar for a quick access."
]
},
{
@@ -110,17 +73,33 @@
"### Download and Convert the Huggingface Model to OpenVINO IR Format:\n",
"\n",
"#### Login to Huggingface:\n",
- "Generate a token from Huggingface for private/gated models like Meta Llama, etc. To access such private/gated models, refer to [Huggingface documentation](https://huggingface.co/docs/hub/en/models-gated)."
+ "- Generate a token from Huggingface for private/gated models like Meta Llama, etc. To access such private/gated models, refer to [Huggingface documentation](https://huggingface.co/docs/hub/en/models-gated).\n",
+ "- If you already have access to such gated models, know how to generate an access token by referring [this](https://huggingface.co/docs/hub/en/security-tokens)."
]
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 1,
"id": "0c9347f7-09dc-4324-b1e2-253c4d5a9f2f",
"metadata": {
"scrolled": true
},
- "outputs": [],
+ "outputs": [
+ {
+ "data": {
+ "application/vnd.jupyter.widget-view+json": {
+ "model_id": "33774886e55e49729038a78264b350de",
+ "version_major": 2,
+ "version_minor": 0
+ },
+ "text/plain": [
+ "VBox(children=(HTML(value='
Converting LLMs to OpenVINO IR Format could take sometime.. Meanwhile, have a milkshake!🧋 "
+ ]
+ },
{
"cell_type": "code",
"execution_count": null,
@@ -147,8 +134,8 @@
"ROOT_DIR = Path.cwd()\n",
"MODEL_DIR = ROOT_DIR / 'models'\n",
"MODEL_DIR.mkdir(parents=True, exist_ok=True)\n",
- "os.system(f\"optimum-cli export openvino --model Qwen/Qwen2-7B-Instruct --weight-format int4 {MODEL_DIR}/ov_qwen7b\")\n",
- "os.system(f\"optimum-cli export openvino --model meta-llama/Llama-2-7b-chat-hf --weight-format int4 {MODEL_DIR}/ov_llama_2\")"
+ "os.system(f\"uv run optimum-cli export openvino --model Qwen/Qwen2-7B-Instruct --weight-format int4 {MODEL_DIR}/ov_qwen7b\")\n",
+ "os.system(f\"uv run optimum-cli export openvino --model meta-llama/Llama-2-7b-chat-hf --weight-format int4 {MODEL_DIR}/ov_llama_2\")"
]
},
{
@@ -159,56 +146,6 @@
">**Note**: [Raise access request](https://www.llama.com/llama-downloads) for Llama models as it is a gated repository.\n"
]
},
- {
- "attachments": {},
- "cell_type": "markdown",
- "id": "4b2ed7c2-e406-4423-b7a6-0a377bb2a4b5",
- "metadata": {},
- "source": [
- "### Load the extension\n",
- "\n",
- "To load an unpacked extension in developer mode:\n",
- "- Go to the Extensions page by entering **chrome://extensions** in a new tab. (By design chrome:// URLs are not linkable.)\n",
- " - Alternatively, **click the Extensions menu puzzle button and select Manage Extensions** at the bottom of the menu.\n",
- " - Or, click the Chrome menu, hover over More Tools, then select Extensions.\n",
- "- Enable **Developer Mode** by clicking the toggle switch next to Developer mode.\n",
- "- Click the **Load unpacked** button and select the extension directory.\n",
- "- Refer to [Chrome’s development documentation](https://developer.chrome.com/docs/extensions/get-started/tutorial/hello-world#load-unpacked) for further details.\n",
- "\n",
- "
\n",
- "\n",
- "\n",
- "\n",
- "\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "5e4c9750-8fcc-474b-87c5-ba28448b632d",
- "metadata": {},
- "source": [
- "### Pin the extension\n",
- "Pin your extension to the toolbar to quickly access your extension.\n",
- "\n",
- "
\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "1191f358-ebcb-4ca9-b0a5-c80f96ac93fd",
- "metadata": {
- "jp-MarkdownHeadingCollapsed": true
- },
- "source": [
- "### Code Sample Structure\n",
- "Browser plugin code has two parts, one is backend folder & the other is extension folder.\n",
- "- **Backend** - In the backend folder, we have two python files `code.py` and `server.py`\n",
- " - `code.py` manages data pre-processing tasks\n",
- " - `server.py` manages flask server-side operations\n",
- "- **Extension** - In the extension we have the front end code required for the browser plugin (popup.html, popup.js, style.css, manifest.json)\n"
- ]
- },
{
"cell_type": "markdown",
"id": "0d060a9e-0355-41e5-a33c-c8baf35d7481",
@@ -227,7 +164,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 3,
"id": "9d1333e1-1785-4a4b-b426-a7d32640236c",
"metadata": {},
"outputs": [],
@@ -240,7 +177,13 @@
"from langchain_text_splitters import RecursiveCharacterTextSplitter\n",
"from langchain_chroma import Chroma\n",
"from langchain_community.embeddings import HuggingFaceEmbeddings\n",
- "from langchain_community.document_loaders import WebBaseLoader, PyPDFLoader"
+ "from langchain_community.document_loaders import WebBaseLoader, PyPDFLoader\n",
+ "import os\n",
+ "import openvino as ov\n",
+ "import logging\n",
+ "os.environ['USER_AGENT']='myagent'\n",
+ "import warnings\n",
+ "warnings.filterwarnings(\"ignore\")"
]
},
{
@@ -254,7 +197,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 4,
"id": "de8469f9-5b72-4a2c-bc09-a4406a34a1c8",
"metadata": {},
"outputs": [],
@@ -290,7 +233,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 5,
"id": "dd83e322-d97f-420d-88cf-21ceba6261fe",
"metadata": {},
"outputs": [],
@@ -331,7 +274,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 6,
"id": "6750172c-0d22-4e70-98d5-7a5ec29f4c36",
"metadata": {},
"outputs": [],
@@ -348,7 +291,19 @@
" model_path='models/ov_llama_2'\n",
" elif model_id==\"Qwen 7B Instruct\":\n",
" model_path='models/ov_qwen7b'\n",
- " model = OVModelForCausalLM.from_pretrained(model_path , device='GPU')\n",
+ " core = ov.Core()\n",
+ " available_devices = core.available_devices\n",
+ " logging.info(f\"Detected OpenVINO devices: {available_devices}\")\n",
+ " device = 'CPU' \n",
+ " for device_name in available_devices:\n",
+ " if device_name.startswith('GPU'):\n",
+ " device = device_name\n",
+ " logging.info(f\"Intel GPU found: '{device}'. Attempting to load model on it first.\")\n",
+ " break \n",
+ " else:\n",
+ " logging.info(\"No Intel GPU detected by OpenVINO. Proceeding with CPU.\")\n",
+ "\n",
+ " model = OVModelForCausalLM.from_pretrained(model_path , device=device)\n",
" tokenizer = AutoTokenizer.from_pretrained(model_path)\n",
" pipe=pipeline(\n",
" \"text-generation\",\n",
@@ -383,7 +338,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 7,
"id": "edebde7e-f6df-4123-88e8-13f9f072d296",
"metadata": {},
"outputs": [],
@@ -437,7 +392,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 8,
"id": "c6ad4d92-c524-450b-b8af-e5630ed0b5bb",
"metadata": {},
"outputs": [],
@@ -490,7 +445,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 9,
"id": "83fc83ad-3e4a-4d4b-906e-01bbebd10ae3",
"metadata": {},
"outputs": [],
@@ -544,7 +499,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 10,
"id": "2fd5d11d-0e12-4e1e-9669-31692af50159",
"metadata": {},
"outputs": [],
@@ -595,7 +550,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 11,
"id": "1434779c-1d0b-49ec-bbb9-dbbd827f61bf",
"metadata": {},
"outputs": [],
@@ -618,7 +573,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 12,
"id": "54773857-d296-44c4-847e-d26da86b01f9",
"metadata": {},
"outputs": [],
@@ -641,7 +596,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 13,
"id": "354d0e26-8719-4b54-888d-0264c0954416",
"metadata": {},
"outputs": [],
@@ -673,7 +628,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 14,
"id": "8f664170-bf7c-41b5-b41b-45217a5c1724",
"metadata": {},
"outputs": [],
@@ -701,7 +656,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 15,
"id": "36575ff6-f3ee-4ddc-8c56-689c8b6844bd",
"metadata": {},
"outputs": [],
@@ -734,7 +689,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 16,
"id": "6ea9ce78-c713-4204-ad68-9b9a9e9a79f2",
"metadata": {},
"outputs": [],
@@ -775,7 +730,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 17,
"id": "7b4f6d9e-9007-461c-88ed-13260d5cf559",
"metadata": {},
"outputs": [],
@@ -807,7 +762,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 18,
"id": "873be699-2bf8-4238-88cd-f3c36431fd85",
"metadata": {},
"outputs": [],
@@ -833,8 +788,23 @@
"id": "40f1be65-2d3b-4fce-bb31-baa7a6573e25",
"metadata": {},
"source": [
- "### Calling the main function\n",
- "This code snippet ensures that the Flask development server starts only when the application is run directly"
+ "### Launch Plugin via Flask Server\n",
+ "\n",
+ "#### 🎉 Flask server is Ready! 🎉\n",
+ "\n",
+ "Your application is now live and waiting for interaction!\n",
+ "\n",
+ "**🚀 Essential Step: Activate Your Browser Plugin!**\n",
+ "\n",
+ "- This application operates through its dedicated browser extension.\n",
+ "- To begin, please open your web browser and locate the plugin's icon, which looks like `T`, in your toolbar (it's often in the top-right corner).\n",
+ "- Click on the `T` icon to access the browser extension\n",
+ " > **NOTE**: Do not access the flask server API Endpoint as the application is running on **browser extension**. \n",
+ "\n",
+ "**Having trouble?**\n",
+ "- Is the plugin loaded? If you haven't already, please load it by following the Readme.md file.\n",
+ "- Is it enabled? Double-check your browser's extension settings to ensure the plugin isn't disabled.\n",
+ "- Have you pinned the extension? Pin the extension.\n"
]
},
{
@@ -842,17 +812,43 @@
"execution_count": null,
"id": "4138fa1d-5f11-4629-a636-f17cc114644d",
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " * Serving Flask app '__main__'\n",
+ " * Debug mode: off\n"
+ ]
+ },
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.\n",
+ " * Running on http://127.0.0.1:5000\n",
+ "Press CTRL+C to quit\n"
+ ]
+ }
+ ],
"source": [
"app.run(port=5000)"
]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "1a5379a7-3e4e-4dbb-acbc-145f731318e1",
+ "metadata": {},
+ "outputs": [],
+ "source": []
}
],
"metadata": {
"kernelspec": {
- "display_name": "summarizer_plugin",
+ "display_name": "Python 3 (ipykernel)",
"language": "python",
- "name": "summarizer_plugin"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
@@ -864,7 +860,14 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.11.10"
+ "version": "3.12.7"
+ },
+ "widgets": {
+ "application/vnd.jupyter.widget-state+json": {
+ "state": {},
+ "version_major": 2,
+ "version_minor": 0
+ }
}
},
"nbformat": 4,
diff --git a/Text-Summarizer-Browser-Plugin/backend/code.py b/Text-Summarizer-Browser-Plugin/backend/code.py
index 9e3a7e6..57bf492 100644
--- a/Text-Summarizer-Browser-Plugin/backend/code.py
+++ b/Text-Summarizer-Browser-Plugin/backend/code.py
@@ -1,4 +1,5 @@
# Importing necessary libraries
+import warnings
from transformers import AutoTokenizer, pipeline
from optimum.intel import OVModelForCausalLM
from langchain_community.llms import HuggingFacePipeline
@@ -8,6 +9,10 @@
from langchain_chroma import Chroma
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_community.document_loaders import WebBaseLoader, PyPDFLoader
+import openvino as ov
+import logging
+os.environ['USER_AGENT'] = 'myagent'
+warnings.filterwarnings("ignore")
# Prompt Templates for Summarization & QA Bot
summary_template = """Write a concise summary of the following: "{context}" CONCISE SUMMARY: """
@@ -57,8 +62,23 @@ def load_llm(model_id):
model_path = '../models/ov_llama_2'
elif model_id == "Qwen 7B Instruct":
model_path = '../models/ov_qwen7b'
- model = OVModelForCausalLM.from_pretrained(
- model_path, device='GPU')
+ core = ov.Core()
+ available_devices = core.available_devices
+ logging.info(f"Detected OpenVINO devices: {available_devices}")
+ device = 'CPU'
+ for device_name in available_devices:
+ if device_name.startswith('GPU'):
+ device = device_name
+ logging.info(
+ f"Intel GPU found: '{device}'. Attempting to load model on it first.")
+ break
+ else:
+ logging.info(
+ "No Intel GPU detected by OpenVINO. Proceeding with CPU.")
+
+ model = OVModelForCausalLM.from_pretrained(model_path,
+ device=device
+ )
tokenizer = AutoTokenizer.from_pretrained(model_path)
pipe = pipeline(
"text-generation",
diff --git a/Text-Summarizer-Browser-Plugin/backend/server.py b/Text-Summarizer-Browser-Plugin/backend/server.py
index 29cbd66..a65a2da 100644
--- a/Text-Summarizer-Browser-Plugin/backend/server.py
+++ b/Text-Summarizer-Browser-Plugin/backend/server.py
@@ -1,10 +1,14 @@
# Importing necessary Libraries
+import warnings
import time
from flask import Flask, Response, request, jsonify
from flask_cors import CORS
from code import load_llm, pre_process_url_data, qa_on_url_summarized_text, pre_process_pdf_data, qa_on_pdf_summarized_text
import tempfile
import chromadb
+import os
+os.environ['USER_AGENT'] = 'myagent'
+warnings.filterwarnings("ignore")
# Initializing the flask app and enabling CORS
app = Flask(__name__)
@@ -114,4 +118,20 @@ def url_process_query():
if __name__ == '__main__':
+ print("""
+ 🎉 Flask server is Ready! 🎉
+ Your application is now live and waiting for interaction!
+
+ **🚀 Essential Step: Activate Your Browser Plugin!**
+
+ - This application operates through its dedicated browser extension.
+ - To begin, please open your web browser and locate the plugin's icon, which looks like `T`, in your toolbar (it's often in the top-right corner).
+ - Click on the `T` icon to access the browser extension
+
+ **Having trouble?**
+ - Is the plugin loaded? If you haven't already, please load it by following the Readme.md file.
+ - Is it enabled? Double-check your browser's extension settings to ensure the plugin isn't disabled.
+ - Have you pinned the extension? Pin the extension.
+ """
+ )
app.run(port=5000)
diff --git a/Text-Summarizer-Browser-Plugin/extension/manifest.json b/Text-Summarizer-Browser-Plugin/extension/manifest.json
index 049083a..5c0850e 100644
--- a/Text-Summarizer-Browser-Plugin/extension/manifest.json
+++ b/Text-Summarizer-Browser-Plugin/extension/manifest.json
@@ -8,4 +8,4 @@
"action": {
"default_popup": "popup.html"
}
- }
+ }
\ No newline at end of file
diff --git a/Text-Summarizer-Browser-Plugin/extension/popup.html b/Text-Summarizer-Browser-Plugin/extension/popup.html
index 59258b9..3b59231 100644
--- a/Text-Summarizer-Browser-Plugin/extension/popup.html
+++ b/Text-Summarizer-Browser-Plugin/extension/popup.html
@@ -81,5 +81,4 @@ Summarizer