WFST help needed for special cases

[ti3x]

Have three questions about developing WFST grammar

1. Is there a way to see output without the grammar applied? When something is not working properly, we need to see what's output from the ASR model
2. How to support singular unit in measurement?
   Example:
   The Ground truth for the millimeter transcription is
   "A 7 mm right"
   The default Riva Conformer shows
   "a seven millimeter right"
3. We have to implement certain unique speech-based transform, like "eight eight twenty eighteen" need to be translated to 8/8/2018. I am not sure how to handle this kind of cases
   Example:

five five twenty twenty comma dated three three two thousand nine comma

[titu1994]

@ekmb @yzhang123

[ekmb]

1. There should be a way to disable ITN in Riva, @anand-nv could you help with that, please?
2. You can change this line to graph_unit_plural = graph_unit_singular | get_singulars(graph_unit_singular)
3. New date formats could be added here.
   This tutorial might be helpful - https://github.com/NVIDIA/NeMo-text-processing/blob/main/tutorials/WFST_Tutorial.ipynb

This function might be useful for grammar development/debugging

from pynini.lib.rewrite import top_rewrite
print(top_rewrite(input_string, graph))

[anand-nv]

1. Is there a way to see output without the grammar applied? When something is not working properly, we need to see what's output from the ASR model

How are you using Riva here? If via NGC, you should be able to skip ITN in the ASR pipeline (when creating the rmir) via riva-build https://docs.nvidia.com/deeplearning/riva/user-guide/docs/model-overview.html#riva-build

[anand-nv]

You can also disable ITN per request by passing verbatim_transcripts=true from the client as part of the ASR config

[ti3x]

1. Is there a way to see output without the grammar applied? When something is not working properly, we need to see what's output from the ASR model
   How are you using Riva here? If via NGC, you should be able to skip ITN in the ASR pipeline (when creating the rmir) via riva-build https://docs.nvidia.com/deeplearning/riva/user-guide/docs/model-overview.html#riva-build

Got what you are saying, the verbatim_transcripts runtime flag is what I was looking for. I know there are different ways to tweak the riva_build and I mostly have been just re-using what's found on https://docs.nvidia.com/deeplearning/riva/user-guide/docs/asr/asr-pipeline-configuration.html to avoid any unexpected issues. The verbatim_transcripts should help us iterate on the grammar a bit better, otherwise I'd have to go through riva-build + riva-init + riva start each time.

[ti3x]

@ekmb Thank you for the answer on the change. I think the instruction for updating + exporting grammar steps somehow does not work for me.

After making changes

python NeMo/nemo_text_processing/inverse_text_normalization/inverse_normalize.py --text="seven millimeter one millimeter"
[NeMo W 2022-12-08 16:29:54 optimizers:77] Could not import distributed_fused_adam optimizer from Apex
[NeMo I 2022-12-08 16:29:54 tokenize_and_classify:64] Creating ClassifyFst grammars.
Time to generate graph: 7.37 sec
seven millimeter 1 mm

cd NeMo/tools/text_processing_deployment
bash export_grammars.sh --GRAMMARS=itn_grammars --LANGUAGE=en
echo "ITN result: seven millimeter and one millimeter. TN result: $2.50" | ../../src/bin/normalizer_main --config=sparrowhawk_configuration.ascii_proto
[DEBUG:normalizer.cc:162] Verbalizing: [!/"one millimeter.(�	1"mm]
[DEBUG:normalizer.cc:195] Verbalize output: Words
ITN result: seven millimeter and 1 mm . TN result: .50

ITN result: seven millimeter and 1 mm . TN result: .50

I double-checked the file change to make sure it is actually changed through git output below

diff --git a/nemo_text_processing/inverse_text_normalization/en/taggers/measure.py b/nemo_text_processing/inverse_text_normalization/en/taggers/measure.py
index c050bc759..497a4250b 100644
--- a/nemo_text_processing/inverse_text_normalization/en/taggers/measure.py
+++ b/nemo_text_processing/inverse_text_normalization/en/taggers/measure.py
@@ -43,8 +43,8 @@ class MeasureFst(GraphFst):

         graph_unit = pynini.string_file(get_abs_path("data/measurements.tsv"))
         graph_unit_singular = pynini.invert(graph_unit)  # singular -> abbr
-        graph_unit_plural = get_singulars(graph_unit_singular)  # plural -> abbr
-
+        # graph_unit_plural = get_singulars(graph_unit_singular)  # plural -> abbr
+        graph_unit_plural = graph_unit_singular | get_singulars(graph_unit_singular)
         optional_graph_negative = pynini.closure(
             pynutil.insert("negative: ") + pynini.cross("minus", "\"true\"") + delete_extra_space, 0, 1
         )

[anand-nv]

graph_unit_plural = get_singulars(graph_unit_singular)|graph_unit_singular # plural -> abbr works for me. Likely you already have a verion of NeMo installed and so any changes to these files are being ignored. Probably good to install NeMo using ./reinstall.sh dev in your python environment.

python nemo_text_processing/inverse_text_normalization/inverse_normalize.py --text="seven millimeter one millimeter"
[NeMo W 2022-12-08 22:20:08 optimizers:55] Apex was not found. Using the lamb or fused_adam optimizer will error out.
[NeMo I 2022-12-08 22:20:08 tokenize_and_classify:64] Creating ClassifyFst grammars.
Time to generate graph: 5.44 sec
7 mm 1 mm```

[ti3x]

graph_unit_plural = get_singulars(graph_unit_singular)|graph_unit_singular # plural -> abbr works for me. Likely you already have a verion of NeMo installed and so any changes to these files are being ignored. Probably good to install NeMo using ./reinstall.sh dev in your python environment.

python nemo_text_processing/inverse_text_normalization/inverse_normalize.py --text="seven millimeter one millimeter"
[NeMo W 2022-12-08 22:20:08 optimizers:55] Apex was not found. Using the lamb or fused_adam optimizer will error out.
[NeMo I 2022-12-08 22:20:08 tokenize_and_classify:64] Creating ClassifyFst grammars.
Time to generate graph: 5.44 sec
7 mm 1 mm```

That explains why no matter what changes I tried at WFST, the changes just have no effect.
Do you run re-install.sh dev every time you make a change to the grammar files? Does this also affect the export-grammars.sh script?

After doing /reinsall.sh dev, it says

e>
    from nemo_text_processing.text_normalization.data_loader_utils import load_file, write_file
ModuleNotFoundError: No module named 'nemo_text_processing'

[anand-nv]

It's done once - refer https://pip-python3.readthedocs.io/en/latest/reference/pip_install.html#editable-installs. Likely some of the dependencies failed to install.
You could probably try this in a python virtual environment.

[ti3x]

@anand-nv alright, I will try again with a clean virtualenv and see what happens later and report back.

It indeed failed with

PackagesNotFoundError: The following packages are missing from the target environment:
  - torch-model-archiver==0.5
  - torch-workflow-archiver==0.2
  - magma-cuda116==2.6
  - fastai==2.1
  - cmake==3.22