Speech to Text plugin - is a texteditoroperation possible?

[BurningTreeC]

Hi all,

@flancast90 and me were working on a speech-to-text TiddlyWiki5 plugin over the last two weeks (https://github.com/flancast90/Speech-To-Text-in-TW5)

We're now discussing how to create a text-editor toolbar button that allows inserting the text recognized by the SpeechRecognition engine at a tiddlers' caret-position

For that, we think about a new texteditoroperation that comes with the plugin

The text-editor button should not trigger the tm-edit-text-operation message
The button should start recording and from the JS - side we would like to dispatch the message when recording ends

The question now is, if we should build a widget that handles the JS side and if it's possible to insert that widget in the buttons' text-field - or what's the best way to inject the wanted functionality in such a custom text-editor button

We would really appreciate if you had some hints for us

best wishes
Simon

[Jermolene]

Hi @BurningTreeC the problem here is that text editor messages must do all of their work in a single synchronous function.

If they trigger an asynchronous event handler (as I imagine you would be to retrieve the spoken text), then the context originally passed to the action will no longer be valid. Consider a case such as the edit text widget being deleted in between the button click and the completion of the transcription.

I wonder if this can be approach generically: a simple JS dæmon that waits for a trigger word and then pipes the transcribed text into whatever text input has the focus. It could run as a startup module.

[BurningTreeC]

@Jermolene - what I have difficulty in wrapping my head around is how to pipe the transcribed text into the framed text-editor (I cannot get it using document.activeElement [?])

[saqimtiaz]

This might be helpful (taken from Streams):

	var activeElement = document.activeElement;
	var selection;
	if(activeElement && activeElement.tagName === "IFRAME") {
		var idoc = activeElement.contentDocument || activeElement.contentWindow.document;
		activeElement = idoc.activeElement;
		selection = idoc.getSelection();
	} else {
		selection = window.getSelection();
	}


	if(window.CodeMirror && document.activeElement.closest(".CodeMirror")) {
		var cm = document.activeElement.closest(".CodeMirror").CodeMirror;
		var cursor = cm.getCursor("start");
		var startRange = cm.getRange({"line":0,"ch":0},{"line":cursor.line,"ch":cursor.ch});
		var selectionStart = startRange.length;
		var selection = cm.getSelection();
		var selectionEnd = selectionStart + selection.length;
		
		variables["selectionStart"] = startRange.length.toString();
		variables["selectionEnd"] = (selectionStart + selection.length).toString();
		variables["selection"] = cm.getSelection().toString();
	} else if(activeElement && selection && ((activeElement.tagName === "INPUT" && activeElement.type === "TEXT") || activeElement.tagName === "TEXTAREA")) {
		variables["selectionStart"] = activeElement.selectionStart.toString();
		variables["selectionEnd"] = activeElement.selectionEnd.toString();
		variables["selection"] = selection.toString();
	}

[BurningTreeC]

Thank you @saqimtiaz , that's really helpful indeed!

[saqimtiaz]

The overall issue of wanting to get access to the widget that created an input field is something I ran into when thinking about positioning popups at the caret position as well. It is something I am planning to revisit post 5.2.0

[saqimtiaz]

Thank you @saqimtiaz , that's really helpful indeed!

I recommend you also check that the IFRAME element has the correct class:
class="tc-edit-texteditor tc-edit-texteditor-body"

This wasn't necessary because of the scope in which my code runs, but might be for you in case the user has a different iframe focused in the document.

[BurningTreeC]

Thank you @Jermolene and @saqimtiaz for you help!
But I'm in a dead end now. When I set the inputs' value and save the tiddler, the changes don't get saved

[BurningTreeC]

What I'm really about is how to make those new values of the inputs, textareas and the framed texteditor permanent

[saqimtiaz]

Have you tried using execCommand? Do the changes still not persist?

[BurningTreeC]

No I haven't tried yet, I'll try. But for inputs like the tiddler title execCommand doesn't work and I would need a way to modify the actually edited tiddler

[BurningTreeC]

Update: yes @saqimtiaz , using execCommand for the framed textarea works. It makes changes permanent. Thank you for the hint in the right direction.

The missing part is something similar for the normal text inputs

[BurningTreeC]

Ok @saqimtiaz , Thank you for your patience!
I've found out that I can execCommand for the normal inputs, too! 😸

[flancast90]

@BurningTreeC, just did some of my own tests. Document.activeElement has some issues because it is an iframe. Instead, we can call something like this monstrosity of code to get the value of the same-domain iframe.

document.getElementsByClassName('tc-edit-texteditor tc-edit-texteditor-body')[1].contentWindow.document.body.getElementsByTagName('textarea')[0].innerHTML;

We should then be able to access/change/write new content to the iframe textarea. It may not be the "TW Way", but I think it'll work.

[BurningTreeC]

Hi @flancast90 , I'm already doing something similar in my tests. The problem is, that we need to update the underlying editTiddler which is the tiddler that is being edited by the simple or framed engine (simple.js or framed.js // factory.js)

[BurningTreeC]

As of now I'm able to pipe the spoken words into inputs and textareas and the framed texteditor, too.
But if I save such a tiddler, the changes are not saved, which is because the engine (simple.js or framed.js) is not aware of the changes and doesn't update the underlying editTiddler.

Is there a way to update the editTiddler from within our startup script?

[BurningTreeC]

The problem is solved, thank you @Jermolene and @saqimtiaz