NOTICE: This proof-of-concept is no longer being maintained. See BigGrep and UrsaDB for similar capabilities.
Simple N-Gram Fast Indexer & Searcher (SNGFSH)
Got lots of malware and want to be able to quickly limit your search for an arbitrary binary string to a much smaller quantity of files? Then snugglefish is for you!
Check out this post which explains it: http://www.mitre.org/capabilities/cybersecurity/overview/cybersecurity-blog/snugglefish-provides-quick-pattern-matching
This work is based upon a paper published by CMU CERT entitled "A Scalable Search Index for Binary Files" which we highly recommend reading. It contains some optimizations we have not yet implemented.