Not getting response in near real time

[NitinKhandelwal1947]

As your Document say , The primary goal of MetaMapLite to provide a near real-time named-entity recognizer which is not a rigorous as MetaMap but much faster while allowing users to customize and augment its behavior for specific purposes.

But when i tried with running metamaplite.sh as below, i am not getting response in real time. On an average it take 5-6 seconds.

./metamaplite.sh example.txt

As per below screen shot there should be significant improvement in term of processing time between metamap and metamaplite. But when i tried to run metamaplite its not giving response in near real time.

Please let me know if i need to do any set of configuration.

[willjrogers]

There are number of factors determining the length of time MetaMapLite uses to process documents: MetaMapLite is faster when given a large number of documents to process at once versus invoking MetaMapLite once per document due to the startup time of the Java VM.

Also, using MMI output format (the default) requires more processing time than other output format such as Brat due to the MMI format requiring use of a phrase chunker which the other format do not use. Note that, the runs mentioned in the MetaMapLite paper use MetaMapLite with Brat format (the default in 2016) which does not use the chunker.

Time to process Bio_Cits, Clin_Cits, and NCBI Disease Corpora (892 citations):

MetaMapLite 3.6.2rc3 (MMI output format default format for MetaMapLite):

$ time ./metamaplite.sh --mmi ./data/input/Bio_Cits/*.txt \
    ./data/input/Clin_Cits/*.txt \
    ./data/input/NCBI_Disease_Corpus_Training/*.txt

real    21m21.646s
user   15m53.023s
sys      4m14.880s

MetaMapLite 3.6.2rc3 (brat standoff output format used in the MetaMapLite paper):

$ time ./metamaplite.sh --brat ./data/input/Bio_Cits/*.txt \
    ./data/input/Clin_Cits/*.txt \
    ./data/input/NCBI_Disease_Corpus_Training/*.txt

real    7m27.984s
user   7m24.805s
sys      0m14.184s

Also note, that when using MetaMapLite as Java program library that the phrase chunker is not used unless requested explicitly.