You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello! There seem to be two paths currently to hit the system. Use processed grail qa file with evaluate.py under the parser directory. There is a demo pipeline that can be setup using the demo section in the read me.
Current I'm getting different results for the same questions. I have set all the flags mentioned in
For the best possible results, please enable the complete checker (use_beam_check, use_virtual_forward, use_type_checking, and use_entity_anchor. in the demo overrides.
The Redis cache seems up and running. For a sample of 100 questions, nearly 2-5% loss is there in F1 score, EM .
Can you please help with this issue?
Please let me know if you need any further information