benchmarks: latency, precision, recall

[amn41]

Hey! this looks very promising and I'd like to compare it to just using a GliNER model for zero-shot entity extraction.

Any numbers you can share on latency, plus precision and recall on a couple of datasets?

[nicobistolfi]

Some questions to help me understand what you're looking for:

• What's driving the evaluation? Are you processing real-time streams, batch jobs, or something else entirely?
• How do you think about the deterministic vs model trade-off? For example:
  • Do you need the same input to always produce identical output?
  • Is it important that you can audit exactly why something was flagged?
  • Or do you prefer the flexibility to detect novel PII patterns the model learned?
• What's your accuracy bar? Are you more concerned about false positives (flagging non-PII) or false negatives (missing actual PII)?
• Data characteristics? Structured tables, free-form text, specific domains (medical, legal, etc.)?