Merge pull request '2.2.0' (#234) from develop into master

Author: strangetom

.github/workflows/tests.yml

@@ -1,10 +1,15 @@
 name: Tests
 
-on: [push]
+on:
+  push:
+    branches: [master, develop]
+  pull_request:
+    branches: [master, develop]
+    types: [ opened, synchronize, reopened ]
 
 jobs:
   build:
-    runs-on: ubuntu-22.04
+    runs-on: ubuntu-latest
     strategy:
       max-parallel: 4
       matrix:

.pre-commit-config.yaml

@@ -28,12 +28,3 @@ repos:
     hooks:
       - id: sphinx-lint
         stages: [pre-commit]
-  - repo: local
-    hooks:
-      - id: pytest-check
-        name: pytest-check
-        entry: coverage run -m pytest
-        stages: [pre-push]
-        language: system
-        pass_filenames: false
-        always_run: true

CHANGELOG.md

@@ -1,5 +1,25 @@
 # Changelog
 
+## 2.2.0
+
+### Foundation foods:
+
+* Bias foundation food matching to prefer "raw" FDC ingredients, but only if the ingredient name does not include any verbs that indicate the ingredient is not raw (e.g. "cooked").
+* Normalise spelling of tokens in ingredient names to align with spelling used in FDC ingredient descriptions.
+* Fix a bug where foundation foods were never calculated if `separate_names=False`.
+
+### General
+
+* Add logging to library, under the `ingredient-parser` namespace.
+
+### Model
+
+* Improve parser model performance with new features related to sentence structure, such as whether a token is part of an example phrase, a multi-ingredient phrase, or after the split in a compound sentence. See the [Feature Generation](https://ingredient-parser.readthedocs.io/en/latest/explanation/features.html) of the docs for more details.
+
+### Processing
+
+* Improve post processing of names to avoid returning multiple names if the name is split by a non-name token. For example, in the sentence "*8 fresh large basil leaves*", the name should be returned as "*fresh basil leaves*" and not as two separate names: "*fresh*", "*basil leaves*".
+
 ## 2.1.1
 
 * Pin Pint version to 0.24.4, as future versions intend to drop support for Python 3.10.

MANIFEST.in

@@ -1,7 +1,7 @@
 include ingredient_parser/density_context.txt
-include ingredient_parser/en/model.en.crfsuite
-include ingredient_parser/en/ModelCard.en.md
-include ingredient_parser/en/ingredient_embeddings.25d.glove.txt.gz
-include ingredient_parser/en/fdc_ingredients.csv.gz
+include ingredient_parser/en/data/model.en.crfsuite
+include ingredient_parser/en/data/ModelCard.en.md
+include ingredient_parser/en/data/ingredient_embeddings.25d.glove.txt.gz
+include ingredient_parser/en/data/fdc_ingredients.csv.gz
 global-exclude test*
 prune */__pycache__

README.md

@@ -48,13 +48,13 @@ The model has the following accuracy on a test data set of 20% of the total data
 
 ```
 Sentence-level results:
-	Accuracy: 94.66%
+	Accuracy: 94.94%
 
 Word-level results:
-	Accuracy 97.82%
-	Precision (micro) 97.81%
-	Recall (micro) 97.82%
-	F1 score (micro) 97.81%
+	Accuracy 97.90%
+	Precision (micro) 97.88%
+	Recall (micro) 97.90%
+	F1 score (micro) 97.88%
 ```
 
 ## Development
@@ -68,6 +68,8 @@ pre-commit install
 
 to install the pre-commit hooks.
 
+Please target the **develop** branch for pull requests. The main branch is used for stable releases and hotfixes only.
+
 There is a simple web app for testing the parser with ingredient sentences and showing the parsed output. To run the web app, run the command
 
 ```bash

benchmark.py

@@ -1,10 +1,9 @@
 #!/usr/bin/env python3
+import argparse
 import time
 
 from ingredient_parser import parse_ingredient
 
-ITERATIONS = 500
-
 if __name__ == "__main__":
     sentences = [
         ("½ cup warm water (105°F)", "0.5 cup warm water (105°F)"),
@@ -44,12 +43,23 @@
         ),
     ]
 
+    parser = argparse.ArgumentParser(description="Ingredient Parser benchmark")
+    parser.add_argument(
+        "--iterations", "-i", type=int, help="Number of iterations to run.", default=500
+    )
+    parser.add_argument(
+        "--foundationfoods", "-ff", action="store_true", help="Enable foundation foods."
+    )
+    args = parser.parse_args()
+
     start = time.time()
-    for i in range(ITERATIONS):
+    for i in range(args.iterations):
         for sent, _ in sentences:
-            parse_ingredient(sent, expect_name_in_output=True)
+            parse_ingredient(
+                sent, expect_name_in_output=True, foundation_foods=args.foundationfoods
+            )
 
-    total_sentences = ITERATIONS * len(sentences)
+    total_sentences = args.iterations * len(sentences)
     duration = time.time() - start
     print(f"Elapsed time: {duration:.2f} s")
     print(f"{1e6 * duration / total_sentences:.2f} us/sentence")