Merge pull request #1184 from JohnSnowLabs/release/2.6.0

Release/2.6.0

Author: chakravarthik27

demo/tutorials/llm_notebooks/Med_Halt_Tests.ipynb

@@ -42,4 +42,6 @@ The following table gives an overview of the different tutorial notebooks to tes
 | [**Question Answering Benchmarking**](question_answering_benchmarking): This notebook provides a demo on benchmarking Language Models (LLMs) for Question-Answering tasks.   |  Hugging Face Inference API         | Question-Answering                          | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/benchmarks/Question-Answering.ipynb)  |
 | **Fewshot Model Evaluation**: This notebook provides a demo on Optimize and evaluate your models using few-shot prompt techniques  |  OpenAI         | Question-Answering                          | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/Fewshot_QA_Notebook.ipynb) |
 | **Evaluating NER in LLMs**:In this tutorial, we assess the support for Named Entity Recognition (NER) tasks specifically for Large Language Models (LLMs) |  OpenAI         | Question-Answering                           | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/NER%20Casual%20LLM.ipynb) |
-| **Swapping Drug Names Test**:In this notebook, we discussed implementing tests that facilitate the swapping of generic drug names with brand names and vice versa. This feature ensures accurate evaluations in medical and pharmaceutical contexts. |  OpenAI         | Question-Answering                           | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/Swapping_Drug_Names_Test.ipynb) |
+| **Swapping Drug Names Test**:In this notebook, we discussed implementing tests that facilitate the swapping of generic drug names with brand names and vice versa. This feature ensures accurate evaluations in medical and pharmaceutical contexts. |  OpenAI         | Question-Answering                           | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/Swapping_Drug_Names_Test.ipynb) |
+| **Evaluation with Structured Outputs**:In this notebook, we discussed implementing evalution with structured output APIs for OpenAI, Ollama, and Azure-OpenAI, offering greater flexibility and precision when processing model responses. |  OpenAI         | Question-Answering                           | https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/misc/Evaluation_with_Structured_Outputs.ipynb |
+| **Med Halt Tests**:In this notebook, we discussed about insights into your LLMs’ robustness and reliability under diverse conditions with our upgraded Med Halt tests.  |  OpenAI         | Question-Answering                           | https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/Med_Halt_Tests.ipynb |

demo/tutorials/llm_notebooks/dataset-notebooks/JSL_Medical_LLM.ipynb

@@ -46,4 +46,5 @@ The following table gives an overview of the different tutorial notebooks. In th
 | **Misuse_Test_with_Prometheus_evaluation**: In this Notebook, we discussed about new safety testing features to identify and mitigate potential misuse and safety issues in your models   | OpenAI                    |Question-Answering                                  | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/misc/Misuse_Test_with_Prometheus_evaluation.ipynb) |
 | **Visual_QA**: In this Notebook, we discussed about the visual question answering tests to evaluate how models handle both visual and textual inputs, offering a deeper understanding of their versatility.   | OpenAI                    | Visual-Question-Answering (visualqa)                                | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/misc/Misuse_Test_with_Prometheus_evaluation.ipynb) |
 | **Add_New_Lines_and_Tabs_Tests**: In this Notebook, we discussed about new tests like inserting new lines and tab characters into text inputs, challenging your models to handle structural changes without compromising accuracy.   | Hugging Face/John Snow Labs/Spacy	                    |Text-Classification/Question-Answering/Summarization                            | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/misc/Add_New_Lines_and_Tabs_Tests.ipynb) |
-| **Safety_Tests_With_PromptGuard**: In this Notebook, we discussed about evaluating prompts before they are sent to large language models (LLMs), ensuring harmful or unethical outputs are avoided with PromptGuard.   | Hugging Face/John Snow Labs/Spacy                    | Text-Classification/Question-Answering/Summarization                               | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/misc/Safety_Tests_With_PromptGuard.ipynb) |
+| **Safety_Tests_With_PromptGuard**: In this Notebook, we discussed about evaluating prompts before they are sent to large language models (LLMs), ensuring harmful or unethical outputs are avoided with PromptGuard.   | Hugging Face/John Snow Labs/Spacy                    | Text-Classification/Question-Answering/Summarization                               | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/misc/Safety_Tests_With_PromptGuard.ipynb) |
+| **De-biasing Data Augmentation**: In this Notebook, We’ve integrated de-biasing techniques into our data augmentation process, ensuring more equitable and representative model assessments.   | Hugging Face/John Snow Labs/Spacy                    | Text-Classification/Question-Answering                               | https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/misc/Dataset_Debiasing.ipynb |

demo/tutorials/misc/Dataset_Debiasing.ipynb

@@ -1,4 +1,11 @@
 from .base import BaseAugmentaion, AugmentRobustness, TemplaticAugment
 from .augmenter import DataAugmenter
+from .debias import DebiasTextProcessing
 
-__all__ = ["BaseAugmentaion", "AugmentRobustness", "TemplaticAugment", "DataAugmenter"]
+__all__ = [
+    "DebiasTextProcessing",
+    "BaseAugmentaion",
+    "AugmentRobustness",
+    "TemplaticAugment",
+    "DataAugmenter",
+]

demo/tutorials/misc/Evaluation_with_Structured_Outputs.ipynb

@@ -361,7 +361,8 @@ def __init__(
                 raise ImportError(Errors.E097())
 
             except Exception as e_msg:
-                raise Errors.E095(e=e_msg)
+                error_message = str(e_msg)
+                raise Exception(Errors.E095(e=error_message))
 
         if show_templates:
             [print(template) for template in self.__templates]
@@ -610,6 +611,7 @@ def __generate_templates(
         from langtest.augmentation.utils import (
             generate_templates_azoi,  # azoi means Azure OpenAI
             generate_templates_openai,
+            generate_templates_ollama,
         )
 
         params = model_config.copy() if model_config else {}
@@ -620,5 +622,8 @@ def __generate_templates(
         elif model_config and model_config.get("provider") == "azure":
             return generate_templates_azoi(template, num_extra_templates, params)
 
+        elif model_config and model_config.get("provider") == "ollama":
+            return generate_templates_ollama(template, num_extra_templates, params)
+
         else:
             return generate_templates_openai(template, num_extra_templates)