From 1a7a0b7615b2f75232a1a32bf4736d2b4e83517d Mon Sep 17 00:00:00 2001 From: Rui Vieira Date: Mon, 9 Sep 2024 10:02:12 +0100 Subject: [PATCH 1/2] Add notes on using local models --- examples/Detoxify.ipynb | 104 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) diff --git a/examples/Detoxify.ipynb b/examples/Detoxify.ipynb index 09bbe2f..2438a21 100644 --- a/examples/Detoxify.ipynb +++ b/examples/Detoxify.ipynb @@ -188,6 +188,12 @@ "text" ] }, + { + "metadata": {}, + "cell_type": "markdown", + "source": "## Initializing TMaRCo", + "id": "1eb7719e30054304" + }, { "cell_type": "code", "execution_count": 5, @@ -198,6 +204,62 @@ "tmarco = TMaRCo()" ] }, + { + "metadata": {}, + "cell_type": "markdown", + "source": [ + "This will initialize `TMaRCo` using the default models, taken from HuggingFace.\n", + "
\n", + "To use local models with TMaRCo, we need to have them in a local storage, accessible to TMaRCo, initialize separately, and pass them to TMaRCo's constructor.\n", + "
\n", + "For instance, to use the default `facebook/bart-large` model, but locally. First, we would need to retrieve the model:" + ], + "id": "3e16ee305f4983d9" + }, + { + "metadata": {}, + "cell_type": "code", + "outputs": [], + "execution_count": null, + "source": [ + "from huggingface_hub import snapshot_download\n", + "\n", + "snapshot_download(repo_id=\"facebook/bart-large\", local_dir=\"models/bart\")" + ], + "id": "614c9ff6f46a0ea9" + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": "We now initialize the base model and tokenizer from local files and pass them to `TMaRCo`:", + "id": "95bd792e757205d6" + }, + { + "metadata": {}, + "cell_type": "code", + "source": [ + "from transformers import BartForConditionalGeneration, BartTokenizer\n", + "\n", + "tokenizer = BartTokenizer.from_pretrained(\n", + " \"models/bart\", # Or directory where the local model is stored \n", + " is_split_into_words=True, add_prefix_space=True\n", + ")\n", + "\n", + "tokenizer.pad_token_id = tokenizer.eos_token_id\n", + "\n", + "base = BartForConditionalGeneration.from_pretrained(\n", + " \"models/bart\", # Or directory where the local model is stored\n", + " max_length=150,\n", + " forced_bos_token_id=tokenizer.bos_token_id,\n", + ")\n", + "\n", + "# Initialize TMaRCo with local models\n", + "tmarco = TMaRCo(tokenizer=tokenizer, base_model=base)" + ], + "id": "f0f24485822a7c3f", + "outputs": [], + "execution_count": null + }, { "cell_type": "code", "execution_count": 7, @@ -223,6 +285,31 @@ "tmarco.load_models([\"trustyai/gminus\", \"trustyai/gplus\"])" ] }, + { + "metadata": {}, + "cell_type": "markdown", + "source": [ + "
\n", + "To use local expert/anti-expert models with TMaRCo, we need to have them in a local storage, accessible to TMaRCo, as previously.\n", + "However, we don't need to initialize them separately, and can pass the directory directly.\n", + "
\n", + "If we want to use local models with `TMaRCo` (in this case the same default `gminus`/`gplus`):\n" + ], + "id": "c113208c527c342e" + }, + { + "metadata": {}, + "cell_type": "code", + "outputs": [], + "execution_count": null, + "source": [ + "snapshot_download(repo_id=\"trustyai/gminus\", local_dir=\"models/gminus\")\n", + "snapshot_download(repo_id=\"trustyai/gplus\", local_dir=\"models/gplus\")\n", + "\n", + "tmarco.load_models([\"models/gminus\", \"models/gplus\"])" + ], + "id": "dfa288dcb60102c" + }, { "cell_type": "code", "execution_count": 13, @@ -362,6 +449,23 @@ "tmarco.load_models([\"trustyai/gminus\", \"trustyai/gplus\"])" ] }, + { + "metadata": {}, + "cell_type": "markdown", + "source": "As noted previously, to use local models, simply pass the initialized tokenizer and base model to the constructor, and the local path as the expert/anti-expert:", + "id": "b0738c324227f57" + }, + { + "metadata": {}, + "cell_type": "code", + "outputs": [], + "execution_count": null, + "source": [ + "tmarco = TMaRCo(tokenizer=tokenizer, base_model=base)\n", + "tmarco.load_models([\"models/gminus\", \"models/gplus\"])" + ], + "id": "b929e21a97ea914e" + }, { "cell_type": "markdown", "id": "5303f56b-85ff-40da-99bf-6962cf2f3395", From b35978a7349b6a11e36e8018d214c1b0c02a1ed1 Mon Sep 17 00:00:00 2001 From: Rui Vieira Date: Mon, 7 Oct 2024 15:40:59 +0100 Subject: [PATCH 2/2] Rewrite local model notes --- examples/Detoxify.ipynb | 77 ++++++++++++++++++++++------------------- 1 file changed, 42 insertions(+), 35 deletions(-) diff --git a/examples/Detoxify.ipynb b/examples/Detoxify.ipynb index 2438a21..f23d8a8 100644 --- a/examples/Detoxify.ipynb +++ b/examples/Detoxify.ipynb @@ -189,10 +189,12 @@ ] }, { - "metadata": {}, "cell_type": "markdown", - "source": "## Initializing TMaRCo", - "id": "1eb7719e30054304" + "id": "1eb7719e30054304", + "metadata": {}, + "source": [ + "## Initializing TMaRCo" + ] }, { "cell_type": "code", @@ -205,43 +207,48 @@ ] }, { - "metadata": {}, "cell_type": "markdown", + "id": "3e16ee305f4983d9", + "metadata": {}, "source": [ "This will initialize `TMaRCo` using the default models, taken from HuggingFace.\n", "
\n", - "To use local models with TMaRCo, we need to have them in a local storage, accessible to TMaRCo, initialize separately, and pass them to TMaRCo's constructor.\n", + "To use local models with TMaRCo, we need to have the pre-initialized models in a local storage that is accessible to TMaRCo.\n", "
\n", "For instance, to use the default `facebook/bart-large` model, but locally. First, we would need to retrieve the model:" - ], - "id": "3e16ee305f4983d9" + ] }, { - "metadata": {}, "cell_type": "code", - "outputs": [], "execution_count": null, + "id": "614c9ff6f46a0ea9", + "metadata": {}, + "outputs": [], "source": [ "from huggingface_hub import snapshot_download\n", "\n", "snapshot_download(repo_id=\"facebook/bart-large\", local_dir=\"models/bart\")" - ], - "id": "614c9ff6f46a0ea9" + ] }, { - "metadata": {}, "cell_type": "markdown", - "source": "We now initialize the base model and tokenizer from local files and pass them to `TMaRCo`:", - "id": "95bd792e757205d6" + "id": "95bd792e757205d6", + "metadata": {}, + "source": [ + "We now initialize the base model and tokenizer from local files and pass them to `TMaRCo`:" + ] }, { - "metadata": {}, "cell_type": "code", + "execution_count": null, + "id": "f0f24485822a7c3f", + "metadata": {}, + "outputs": [], "source": [ "from transformers import BartForConditionalGeneration, BartTokenizer\n", "\n", "tokenizer = BartTokenizer.from_pretrained(\n", - " \"models/bart\", # Or directory where the local model is stored \n", + " \"models/bart\", # Or directory where the local model is stored\n", " is_split_into_words=True, add_prefix_space=True\n", ")\n", "\n", @@ -255,10 +262,7 @@ "\n", "# Initialize TMaRCo with local models\n", "tmarco = TMaRCo(tokenizer=tokenizer, base_model=base)" - ], - "id": "f0f24485822a7c3f", - "outputs": [], - "execution_count": null + ] }, { "cell_type": "code", @@ -286,29 +290,30 @@ ] }, { - "metadata": {}, "cell_type": "markdown", + "id": "c113208c527c342e", + "metadata": {}, "source": [ "
\n", - "To use local expert/anti-expert models with TMaRCo, we need to have them in a local storage, accessible to TMaRCo, as previously.\n", + "To use local expert/anti-expert models with TMaRCo, we need to have them in a local storage that is accessible to TMaRCo, as previously.\n", + "\n", "However, we don't need to initialize them separately, and can pass the directory directly.\n", "
\n", "If we want to use local models with `TMaRCo` (in this case the same default `gminus`/`gplus`):\n" - ], - "id": "c113208c527c342e" + ] }, { - "metadata": {}, "cell_type": "code", - "outputs": [], "execution_count": null, + "id": "dfa288dcb60102c", + "metadata": {}, + "outputs": [], "source": [ "snapshot_download(repo_id=\"trustyai/gminus\", local_dir=\"models/gminus\")\n", "snapshot_download(repo_id=\"trustyai/gplus\", local_dir=\"models/gplus\")\n", "\n", "tmarco.load_models([\"models/gminus\", \"models/gplus\"])" - ], - "id": "dfa288dcb60102c" + ] }, { "cell_type": "code", @@ -450,21 +455,23 @@ ] }, { - "metadata": {}, "cell_type": "markdown", - "source": "As noted previously, to use local models, simply pass the initialized tokenizer and base model to the constructor, and the local path as the expert/anti-expert:", - "id": "b0738c324227f57" + "id": "b0738c324227f57", + "metadata": {}, + "source": [ + "As noted previously, to use local models, simply pass the initialized tokenizer and base model to the constructor, and the local path as the expert/anti-expert:" + ] }, { - "metadata": {}, "cell_type": "code", - "outputs": [], "execution_count": null, + "id": "b929e21a97ea914e", + "metadata": {}, + "outputs": [], "source": [ "tmarco = TMaRCo(tokenizer=tokenizer, base_model=base)\n", "tmarco.load_models([\"models/gminus\", \"models/gplus\"])" - ], - "id": "b929e21a97ea914e" + ] }, { "cell_type": "markdown",