Extracting Regex Patterns from Strings - Trying to Think of Techniques to Improve

[jwstanwick]

Recently I've been working on a project that requires me to generate a ton of regex patterns from a large amount of strings. These strings can be in any form and may or may not have a pattern in them. An example of my use case would be trying to extract all of the names of people from a sentence. I need to generate both the name and the reusable regex pattern required to extract the name in future strings. For example, in the string "John Doe went to the store", the goal of the system would be to extract "John Doe" with a regex pattern of "^\s*John\s+Doe". The regex pattern just needs to be able to match to another sentence like "I went to dinner with John Doe". Both of those sentences would be able to be matched from the regex pattern generated from the first pattern.

There is a hidden complexity in that the sentence could be something like "George walked his dog Max". In this case, "George" would be the desired extraction, rather than "Max".

Right now, I am using two different LangChain functions to extract these patterns. One of them extracts the name with some simple prompt engineering as well as a couple few-shot examples of names and sentences. The other generates the regex pattern with a similar approach of using some prompt engineering and few-shot examples.

The problem that I am having right now is that my accuracy has hit a ceiling. I am currently sitting at around 60% accuracy on the strings. Most of the strings are incredibly complex and either have a ton of noise, or they have multiple names and determining which one is correct is non-trivial. Are there any techniques that could be used to help my use case?

Thanks for any help!