Feature request

[JSL101]

I don't know if this is the appropriate place to ask for new features. If not, my apologies. But I'm wondering if it's possible to build a version which includes at least rudimentary formatting commands: New Paragraph, Italics, and so on. Would that be easy to do or very difficult?

[doxgt]

https://github.com/doxgt/PlayGround/blob/main/GPT_cURL.ahk

I just added "PostProcessing" to above, after using/tweaking it here and there over 3 months. This is strictly for those who tinker with AutoHotkey (Windows only regrettably). I would be highly pleased if @braden-w would kindly consider implementing something similar, as a switchable feature. Perhaps even as a "paid feature".

Anyhow, since I am a Windows serf, and AHK is deeply integrated into my daily workflow, I use the GUI-less, Push-To-Talk, cURL-based AHK app above exclusively. However, I'd like my secretary to be able to use Whispering, as it would help her, and she is not tied to Windows necessarily.

Anyway, with prompting and a few sequential, elementary RegEx statements, it is easy to deliver a "Dragon-like" experience. As a Dragon Medical user, I require a certain level of formatting/punctuating consistency myself more than anything else.

Post-processing does sacrifice "auto-punctuation". However, the upside is to be able to utter "new paragraph", "open/close quote", "left/right paren(thesis)", etc ...

P.S., I do not use AHK because I use Windows. I use Windows because AHK is only available on Windows. If it wasn't for AHK, I just can't think of a compelling reason why anyone should use Windows by choice. At all.

[JSL101]

Thanks for this, doxgt. Unfortunately, the auto punctuation is one of my favorite parts of Whisper., and I think I'd prefer to keep that and forgo formatting commands, than vice versa. It would be nice to have both, of course, though I don't know how you would implement it. Maybe by hitting the control key before speaking a formatting command or something like that.

[doxgt]

Precisely you can control behaviours such as turning auto-punctuation on/off through hotkeys. The example I posted above was meant to show you or others who are interested what could be done. Post-processing (combined with prompting for maximal bang) is not difficult, just tedious.

1. Can you keep both auto-punctuation and manual punctuation? Sure you can. Just observe some patterns and use simple regular expressions or conditional statements to eliminate conflicts or duplicates. By the way, I only eliminate the commas and full-stops that the OpenAI Whisper engine puts out. The auto-punctuated questions mark or exclamation marks are preserved. It's all down to personal preference. I prefer a "Dragon-like" experience so I can pause and compose, without worrying that Whisper will spill commas in the most inopportune places. Others may prize the "colloquial" style from the out-of-box Whisper experience. If you are willing to "tinker a bit", you get to control how you experience it.

2. We can always motivate @braden-w to do it for us. Sponsoring is not a bad way to encourage, especially as you can see - integrating such post-processing takes some time.

Cheers!

[technicznyblog]

What if there was an option to process the text through an LLM? So, when the plugin returns the text, it wouldn’t be inserted directly but instead sent as a custom query to an LLM, which would format the text. This way, we could do a wide variety of things with the text.

[braden-w]

Thank you for the feature request @JSL101 and thank you for the help and answers @doxgt ! I intend to add some post-processing features—just been really getting headaches upgrading the underlying framework right now from Tauri v1 to v2, and school has started which means my free time is not as much as before. It's on the list for future releases!

[rkhrkh]

Auto-punctuation is one of the features I appreciate most about Whispering. If any updates or changes are implemented in the future, it is essential to ensure that the option to keep auto-punctuation enabled remains available.

[doxgt]

Auto punctuation is in fact not really "a feature" of whispering, it is rather the current default of the whisper engine! Therefore, I don't think there is any risk of losing that.

For folks who perhaps are not used to the "dragon style" of dictation, a nice middle ground would be to do nothing about the automatically placed punctuations such as periods and commas. But instead, give users the option of "translating" "paragraph marks", "quotation marks", and "parentheses" into their corresponding formatting or punctuation symbols.

Again this should be user definable. "Parenthesis" is a bona-fide mouthful. I use the dragon lingo of "left paren" and "right paren".

I think this user-selectable feature might be implemented as supplemental configurable fields to allow users to determine whether they wish to "auto replace" certain punctuations from spelled out words to their corresponding symbols.

Of course, above works best with prompting. I'd say prompting is definitely a desirable feature. E.g., "left paren" or "left parent" would not work as auto-sub for "left parenthesis" if former are not part of prompts.