You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/explanation/foundation.rst
+13-8Lines changed: 13 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -71,7 +71,7 @@ These vectors are used to compute a fuzzy distance score between the ingredient
71
71
72
72
The full process is as follows:
73
73
74
-
#. Load the :abbr:`FDC(Food Data Central)` data. Tokenize the description for each entry and remove tokens that don't provide useful semantic information*.
74
+
#. Load the :abbr:`FDC(Food Data Central)` data. Tokenize the description for each entry and remove tokens that don't provide useful semantic information\*.
75
75
76
76
#. Prepare the ingredient name tokens in the same way.
77
77
@@ -82,26 +82,31 @@ The full process is as follows:
82
82
83
83
#. Compute the fuzzy distance score between each :abbr:`FDC(Food Data Central)` entry and the ingredient name tokens.
84
84
85
-
#. Sort the :abbr:`FDC(Food Data Central)` by their score.
85
+
#. Sort the :abbr:`FDC(Food Data Central)` entries by the fuzzy distance score.
86
86
87
-
#. If there is a match with a score below the threshold, return the best match.
87
+
#. If the lowest (best) score is below the threshold, return the :class:`FoundationFood <ingredient_parser.dataclasses.FoundationFood>` object for the corresponding :abbr:`FDC(Food Data Central)` entry.
88
88
89
-
#. If there are not any matches with a good enough score, store the best match for fallback matching.
89
+
#. If best score is not below the threshold, store the best entry and it's score for fallback matching.
90
90
91
91
#. If none of the :abbr:`FDC(Food Data Central)` datasets contained a good enough match, attempt fallback matching.
92
92
93
-
#. Sort the best match from each :abbr:`FDC(Food Data Central)` data set.
93
+
#. Sort the best matches from each :abbr:`FDC(Food Data Central)` data set.
94
94
95
95
#. If the score for the best of these matches is below a threshold, return this match.
96
96
97
97
#. If no match is good enough, return ``None``.
98
98
99
99
.. note::
100
100
101
-
Tokens that do not provide useful semantic information are as follows: numbers, white space, punctuation, stop words, single character words.
101
+
\*Tokens that do not provide useful semantic information are as follows: numbers, white space, punctuation, stop words, single character words.
102
102
103
+
Limitations
104
+
^^^^^^^^^^^
103
105
106
+
The current implementation has a some limitations.
104
107
108
+
#. The fuzzy distance scoring will sometimes result in returning an :abbr:`FDC(Food Data Central)` entry that has a good score but is not a good match.
109
+
Work is ongoing to improve this, and suggestions and contributions are welcome.
105
110
106
-
Limitations
107
-
^^^^^^^^^^^
111
+
#. This functionality can be very slow.
112
+
The more datasets that need to be checked to find a good match, the slower it will be.
0 commit comments