Merge branch 'develop' into noun-phrase-features

Author: strangetom

CHANGELOG.md

@@ -1,6 +1,16 @@
 # Changelog
 
-## 2.10
+## develop
+
+* Foundation food improvements:
+  * Bias foundation food matching to prefer "raw" FDC ingredients, but only if the ingredient name does not include any verbs that indicate the ingredient is not raw (e.g. "cooked").
+  * Normalise spelling of tokens in ingredient names to align with spelling used in FDC ingredient descriptions.
+
+## 2.1.1
+
+* Pin Pint version to 0.24.4, as future versions intend to drop support for Python 3.10.
+
+## 2.1.0
 
 > [!WARNING]
 >

README.md

@@ -48,13 +48,13 @@ The model has the following accuracy on a test data set of 20% of the total data
 
 ```
 Sentence-level results:
-	Accuracy: 94.66%
+	Accuracy: 94.65%
 
 Word-level results:
 	Accuracy 97.82%
-	Precision (micro) 97.81%
+	Precision (micro) 97.80%
 	Recall (micro) 97.82%
-	F1 score (micro) 97.81%
+	F1 score (micro) 97.80%
 ```
 
 ## Development

benchmark.py

@@ -1,10 +1,9 @@
 #!/usr/bin/env python3
+import argparse
 import time
 
 from ingredient_parser import parse_ingredient
 
-ITERATIONS = 500
-
 if __name__ == "__main__":
     sentences = [
         ("½ cup warm water (105°F)", "0.5 cup warm water (105°F)"),
@@ -44,12 +43,23 @@
         ),
     ]
 
+    parser = argparse.ArgumentParser(description="Ingredient Parser benchmark")
+    parser.add_argument(
+        "--iterations", "-i", type=int, help="Number of iterations to run.", default=500
+    )
+    parser.add_argument(
+        "--foundationfoods", "-ff", action="store_true", help="Enable foundation foods."
+    )
+    args = parser.parse_args()
+
     start = time.time()
-    for i in range(ITERATIONS):
+    for i in range(args.iterations):
         for sent, _ in sentences:
-            parse_ingredient(sent, expect_name_in_output=True)
+            parse_ingredient(
+                sent, expect_name_in_output=True, foundation_foods=args.foundationfoods
+            )
 
-    total_sentences = ITERATIONS * len(sentences)
+    total_sentences = args.iterations * len(sentences)
     duration = time.time() - start
     print(f"Elapsed time: {duration:.2f} s")
     print(f"{1e6 * duration / total_sentences:.2f} us/sentence")

docs/source/how-to/index.rst

@@ -5,4 +5,5 @@ How to guides
    :maxdepth: 1
 
    Convert units <convert-units>
+   Logging <logging>
    Extend to other languages <extending>

docs/source/how-to/logging.rst

@@ -0,0 +1,47 @@
+Logging
+=======
+
+Python’s standard `logging <https://docs.python.org/3/library/logging.html>`_ module is used to implement debug log output for ingredient-parser.
+This allows ingredient-parser's logging to integrate in a standard way with other application and libraries.
+
+All logging for ingredient-parser is within ``ingredient-parser`` namespace.
+
+* The ``ingredient-parser`` namespace contains general logging for parsing of ingredient sentences.
+* The ``ingredient-parser.foundation-foods`` namespace contains logging related to the :doc:`Foundation Foods </explanation/foundation>` functionality.
+
+For example, to output debug logs to stdout:
+
+.. code:: python
+
+  >>> import logging, sys
+  >>> from ingredient_parser import parse_ingredient
+  >>>
+  >>> logging.basicConfig(stream=sys.stdout)
+  >>> logging.getLogger("ingredient-parser").setLevel(logging.DEBUG)
+  >>>
+  >>> parsed = parse_ingredient("24 fresh basil leaves or dried basil")
+  DEBUG:ingredient-parser:Parsing sentence "24 fresh basil leaves or dried basil" using "en" parser.
+  DEBUG:ingredient-parser:Normalised sentence: "24 fresh basil leaves or dried basil".
+  DEBUG:ingredient-parser:Tokenized sentence: ['24', 'fresh', 'basil', 'leaf', 'or', 'dried', 'basil'].
+  DEBUG:ingredient-parser:Singularised tokens at indices: [3].
+  DEBUG:ingredient-parser:Generating features for tokens.
+  DEBUG:ingredient-parser:Sentence token labels: ['QTY', 'B_NAME_TOK', 'I_NAME_TOK', 'I_NAME_TOK', 'NAME_SEP', 'B_NAME_TOK', 'I_NAME_TOK'].
+
+Only enabling logging for foundation foods:
+
+.. code:: python
+
+  >>> import logging, sys
+  >>> from ingredient_parser import parse_ingredient
+  >>>
+  >>> logging.basicConfig(stream=sys.stdout)
+  >>> logging.getLogger("ingredient-parser.foundation-foods").setLevel(logging.DEBUG)
+  >>>
+  >>> parsed = parse_ingredient("24 fresh basil leaves or dried basil", foundation_foods=True)
+  DEBUG:ingredient-parser.foundation-foods:Matching FDC ingredient for ingredient name tokens: ['fresh', 'basil', 'leaves']
+  DEBUG:ingredient-parser.foundation-foods:Prepared tokens: ['fresh', 'basil', 'leav'].
+  DEBUG:ingredient-parser.foundation-foods:Loaded 13318 FDC ingredients.
+  DEBUG:ingredient-parser.foundation-foods:Selecting best match from 1 candidates based on preferred FDC datatype.
+  DEBUG:ingredient-parser.foundation-foods:Matching FDC ingredient for ingredient name tokens: ['dried', 'basil']
+  DEBUG:ingredient-parser.foundation-foods:Prepared tokens: ['dri', 'basil'].
+  DEBUG:ingredient-parser.foundation-foods:Selecting best match from 1 candidates based on preferred FDC datatype.

ingredient_parser/__init__.py

@@ -9,4 +9,4 @@
     "show_model_card",
 ]
 
-__version__ = "2.1.0"
+__version__ = "2.1.1"

ingredient_parser/_common.py

@@ -1,6 +1,7 @@
 #!/usr/bin/env python3
 
 import collections
+import logging
 import os
 import platform
 import re
@@ -20,6 +21,10 @@
 
 SUPPORTED_LANGUAGES = ["en"]
 
+# Logging
+logger = logging.getLogger("ingredient-parser")
+logger.addHandler(logging.NullHandler())
+
 # Regex pattern for matching a numeric range e.g. 1-2, 2-3, #1$2-1#3$4.
 RANGE_PATTERN = re.compile(r"^[\d\#\$]+\s*[\-][\d\#\$]+$")
 

ingredient_parser/en/ModelCard.en.md

@@ -8,7 +8,7 @@
 
 ### Model Date and Version
 
-Date: April 2025
+Date: May 2025
 
 Version: The model version is the same has the `ingredient_parser_nlp` package version.
 
@@ -124,7 +124,7 @@ The model has the following performance metrics:
 
 | Word level accuracy | Sentence level accuracy |
 | ------------------- | ----------------------- |
-| 97.82 ± 0.18%       | 94.62 ± 0.44%           |
+| 97.82 ± 0.18%       | 94.65 ± 0.44%           |
 
 These metrics were determined by executing 20 training/evaluation cycles and calculating the mean and standard deviation for the two metrics across all cycles. The uncertainty values provided represent the 99.7% confidence bounds (i.e. 3x standard deviation). The uncertainty is due to the randomisation of the selection of training and evaluation data whenever the model is trained.