How to add #SearchBang tag.

[Leftium]

A search bang looks like a #HashTag or #AtMention that starts with a ! (bang character.) For example, !g term searches Google.com for "term."

• I think I found an example of how to add another tag in this discussion: Additional tagging with regexp- mutate model.two or post-process hook? #1086
• Since #SearchBang is not an existing tag like #Fraction, I think I should add it via nlp.plugin and/or model.one.tagSet
  • https://observablehq.com/@spencermountain/compromise-plugins
  • https://observablehq.com/@spencermountain/compromise-world

So adding the simple version of the #SearchBang tag seems pretty straight-forward...

---

However, there are some complications:

• Search bangs may actually put the ! at the end of the term, too. Like: term g!
• Or even omit the ! altogether: term g
• There is a small configurable list of bangs that can omit the ! from a larger list of valid search bangs.
• Search bangs are either at the very start or end of the document; rarely (never?) in the middle.

Is there a way to add the complex version of the #SearchBang tag? Or would it be easier to parse the search bangs and strip them from the text before sending them to NLP compromise?

---

My use-case is analyzing the input to a textarea to determine the best action/transformation:

• If a search bang exists, forward to the search URL.
• If a date exists create an iCal event
• If a number and two currencies exist, show the currency conversion
• If a number and a single currency exists, convert currency to default currency
• etc

[spencermountain]

Hey John, this sounds like a cool project. I really like this sort feature.
Yeah, think this behaviour is good as a plugin - I'm happy to help. It's probably too niche for a new tag in main - and some of the g! stuff would be too noisy for general-purpose tag, imo.

I think there is a similar function in duckduckgo? It would be great if we could contribute to some sort of standard for this. A lot of people use compromise for this cmd+k style stuff. I can help add a searchbang plugin to the main list.

and yeah - it also seems whitelist-based. The plugin should accept possible bang-inputs, and normalize and only tag those, right?
the no-exclamation ones can be done with a normal lexicon approach, like nlp('g foobar', {g:'SearchBang'}) etc.

lemme know if we're on the same page, I can take a crack at getting this started.
cheers

[Leftium]

I think there is a similar function in duckduckgo?

Yes, DuckDuckGo made the OG search bangs, but no longer actively maintains them. There are many clones:

• Here is my version (WIP): https://zbang.vercel.app
  • https://github.com/Leftium/zbang
• I linked to Kagi bangs, the most actively maintained list of bangs. (However, there are still likely thousands of dead/broken bangs.)
  • You must be a member to make general searches, but I think anyone can do a bang search: https://kagi.com/search?q=g!+TEST
  • (Also a slight "Kagi" take on certain bangs.)
• Another notable one: https://unduck.link
  • https://youtu.be/_DnNzRaBWUU (Explains some other problems with DDG bangs)
• I could go on...

The plugin should accept possible bang-inputs, and normalize and only tag those, right?

A plugin that accepts a list of bang-inputs is definitely usable. Probably best to start with the simplest solution.

Just some things to consider:

• The list could get quite long. (At first I was imagining a very long regex. But I guess it can be done with a dictionary or even array lookup...)
• It might be nice to catch new bangs with heuristics. Or not even use a list at all. For example:
  • zbang.vercel.app just forwards requests to Kagi.
  • So if Kagi added added support for !newbang, a heuristic strategy would allow zbang.vercel.app to also support it immediately, without any updates.
  • However: the heuristic approach could also add false positives.

If we were only trying to support the simple version of bangs (only !g term and not g! term, g term), the syntax is quite unambiguous so I would say go for a rules/pattern-based approach.

• In practice, I rarely use the suffix style of bang (g!).
• I mostly use !g search terms or search terms !g
• With a few quick bangs g term (Kagi has a limit of 20 per user account.)
• I'm not sure about others' search bang habits...

For my own project, my plan had been to:

• tag and remove unambiguously identified search bangs.
• tag but leave search bangs that could also be other POS. (Like the end of an exclamation: Hello, world!)

the no-exclamation ones can be done with a normal lexicon approach, like nlp('g foobar', {g:'SearchBang'})

• The no-exclamation ones may also get tagged as other POS. For example some search bangs for YouTube are: !youtube, 'you, !watch
• Contrived example: nlp('you Catch me if you can', {you:'SearchBang'})
• In practice, this will probably be rare. I think most people choose single letter quick bangs.
• Then again: nlp('a If You Give a Mouse a Cookie', {a:'SearchBang', you:'SearchBang'}) (!a is search bang for Amazon)

So I think the heuristic that search bangs (and only search bangs) are at the very end or very beginning of the query (the "document") should be utilized.

• Also there is currently a technical limit of 1 search bang per query, but there have been requests to support multiple bangs. (Something I plan to add to zbang.vercel.app)

---

I also have aspirations to expand/extend search bangs (with the help of nlp-compromise). I'll explain more later...

[Leftium]

I started by imagining search bangs that adapt to their input. For example, the !translate bang could behave like this:

1. if no query, open the document upload tab (https://translate.kagi.com/document)
2. if query is a URL, translate the web page (https://translate.kagi.com/translate/en/QUERY)
3. otherwise, translate the query (https://translate.kagi.com/?from=auto&to=en&text=QUERY)

Then NLP compromise's tags gave me the confidence to expand beyond search bangs:

• Like MacOS Spotlight or launchers like https://keypirinha.com. (I think these are like the cmd+k "command palettes" you mentioned)
• To actions, transformations, and calculations. Like:
  • Create an iCal event from natural language description/pasted text
  • Create a vCard from natural language/pasted text
  • Currency conversion
  • Bang search
  • etc...

So bang search became a plugin of a more general system. Right now I'm trying to figure out:

• How to sort/order the most relevant plugins first
• Determine if the input is valid for a given plugin
• An interface for the plugins' inputs/outputs.

Right now, I'm thinking:

• each plugin has a list of required and optional "slots."
• each slot accepts compromise matches like '#money #currency?'
• a plugin is hidden/ranked low if required slots are missing.
• a plugin is ranked higher the more slots are filled, and the lower the percentage of "unused" document.
• if there is a #SearchBang tag, that's a signal to rank the Bang Search plugin high

---

I'm thinking about integrating other libraries to help with ranking/transformation:

• https://www.npmjs.com/package/tinyld
• https://www.npmjs.com/package/convert-units
• https://github.com/Ray-D-Song/guesslang-js
• https://github.com/ts95/lang-detector

[spencermountain]

hey John-Kim, I started some work on a command-prompt plugin. There are some tiny weird things that I'd like to change for the next release.
For one - your example with a bare ! as a single word. You'll see the tokenizer bolts it onto the preceding word, so it's hard to pull-out:

console.log(nlp('! we walk').docs[0])
/*[
  {
    text: 'we',
    pre: '! ', // ends up here
    post: ' ',
    tags: Set(1) { 'SearchBang' }
  },
  {
    text: 'walk',
    pre: '',
    post: '',
    tags: Set(3) { 'Verb', 'PresentTense', 'Infinitive' },
    normal: 'walk',
  }
]*/

really, this should obey any words that are entered into the lexcion, even if the word is '!'.

Second, I started on a slash command one too? For things like "/me writes more code" to be optionally recognized as a command.

The 'slots' concept you discussed is a strong and well-supported feature of compromise. Let me know if you struggle anywhere to implement it.

Lastly, like you mentioned, it should have the facility to fuzzy-match on a list of recognized command labels. I haven't gotten to a clear understanding of how this should work, but the fuzzy match functionality is already in compromise main. It doesn't rank things, so maybe this plugin should do its own edit-distance scoring.

that's all i got for now. Let me know how things go on your side, and if I can help with anything
cheers

[Leftium]

Wow! Thanks. Your implementation is quite different from the plugin I planned to try writing