Getting JS to talk back to Python

[thorwhalen]

The following are ideas and attempts to use existing tools to have JS "talk back" to Python in a notebook.
That is, communicate some information about events happing on the JS component side back to python variables.

Note: As mentioned in Python-JS two-way communication, we're trying to make this work for browser-based jupyter notebooks (what is launched by jupyter notebook in terminal), but also VSCode.

[thorwhalen]

IPython only

from IPython.display import display, HTML, Javascript

js_code = """
function sendToPython(){
    var data = document.querySelector("#myInput").value;
    var kernel = IPython.notebook.kernel;
    kernel.execute("data_from_js = '" + data + "'");
}

document.querySelector("#myButton").addEventListener("click", function(){
    sendToPython();
});
"""

data_from_js = None

HTML_code = """
<input type="text" id="myInput" placeholder="Enter some text">
<button id="myButton">Submit</button>
<script type="text/Javascript">{}</script>
""".format(js_code)

Different cell (not necessary):

display(HTML(HTML_code))

Enter "hello" in input text field, then click "Submit" button. Then, in a different cell:

print(data_from_js)

Works with browser-based jupyter

But DOES NOT WORK on VScode

[thorwhalen]

Still just in the browser (where the code works), I got the strangest effect.
I've seen this before so want to make sure this is recorded and addressed properly in our tools.

Error reproduction

I copy pasted exactly the same code in a different cell (using browser notebook) and it didn't work: I see the form, but my entering a value in it doesn't effect the notebook's data_from_js variable.

It's exactly the same code!

I restarted the notebook kernel, and even restarted my jupyter notebook application.

The effect remains:
The first cell works, but the second, PURE COPY of it, does not.

Solution

It's actually the fact that it's a PURE COPY that is the problem.

When I replace all ids by some other, unique, ones, it works.
I changed myInput to myInput_2 and myButton to myButton_2, and it worked:

from IPython.display import display, HTML, Javascript

js_code = """
function sendToPython(){
    var data = document.querySelector("#myInput_2").value;
    var kernel = IPython.notebook.kernel;
    kernel.execute("data_from_js = '" + data + "'");
}

document.querySelector("#myButton_2").addEventListener("click", function(){
    sendToPython();
});
"""

HTML_code = """
<input type="text" id="myInput_2" placeholder="Enter some text">
<button id="myButton_2">Submit</button>
<script type="text/Javascript">{}</script>
""".format(js_code)

display(HTML(HTML_code))

[thorwhalen]

So, is it normal than even by closing the notebook page and restarting the jupyter notebook process, I still had the myInput and myButton "ghosts" hanging around?

No, it's not typical for elements like myInput and myButton to persist after closing the notebook page and restarting the Jupyter notebook process. Once you close the notebook or restart the Jupyter server, all in-memory states, including the DOM (Document Object Model) of the notebook, should be cleared.

However, there are a few scenarios that might explain the behavior you observed:

1. Browser Caching: Web browsers cache web pages and scripts to speed up load times. Sometimes, this can lead to older versions of pages or scripts being loaded, especially if there were rapid changes. A hard refresh (usually Ctrl + F5 or Cmd + Shift + R on Mac) can help clear the cache for the current page.

2. Jupyter Cell Outputs: Jupyter notebooks save the outputs of cells within the notebook file itself. If you had previously run the cell and saved the notebook, the output (including the HTML elements with IDs myInput and myButton) would be saved. When you reopen the notebook, these saved outputs are displayed again. However, this shouldn't interfere with the JavaScript execution unless you're running the cells again.

3. Multiple Instances: It's possible, though unlikely, that you had multiple instances of the same notebook open in different tabs or windows. This could lead to confusion, especially if changes are made in one instance and not reflected in another.

4. Kernel State vs. Notebook State: Restarting the kernel clears the Python execution environment, but it doesn't clear the notebook's DOM or the outputs of previously executed cells. To truly start from a clean slate, you'd need to both restart the kernel and clear all cell outputs.

In your case, the most likely explanation is the second one: the cell outputs were saved in the notebook, and when you reopened the notebook, the elements with IDs myInput and myButton were displayed again from the saved outputs. However, this should not have interfered with the new execution of the JavaScript unless there was a conflict with the IDs, as we discussed earlier.

[thorwhalen]

So, as we develop our python-js communication interface, I'm tempted to replace the ids of the JS component's code we're making available in the notebook with some suffixed version that will use a unique suffix every time, so as to not have this effect.
Does that seem like a sensible approach?
I'm concerned that this might create a lot of "junk" in the cache or namespace.
Are my concerns valid?
Might there be a smarter approach?

Your approach of appending a unique suffix to the IDs of the JS components to ensure uniqueness is a common and sensible one, especially when dynamically generating content. It helps avoid the kind of ID conflicts you experienced. However, let's address your concerns and consider potential alternatives:

Concerns:

1. "Junk" in Cache: The browser cache is designed to store assets temporarily to speed up web page loading. Dynamically generated IDs won't significantly impact the cache because the cache primarily deals with static assets like images, stylesheets, and scripts. The dynamic content of a Jupyter notebook isn't typically cached in the same way.

2. Namespace Pollution: If you're generating a lot of unique IDs, there could be concerns about polluting the DOM's namespace. However, in the context of a Jupyter notebook, this is less of a concern. Each time you refresh the notebook or clear cell outputs, the DOM elements associated with those outputs are removed.

Smarter Approaches:

1. Use Classes Instead of IDs: If the primary purpose of the ID is for styling or scripting, consider using class names instead. Classes are designed to be reusable across multiple elements, whereas IDs are meant to be unique. You can have multiple elements with the same class, which can be useful for applying the same styles or behaviors.

2. Data Attributes: Instead of using IDs to store data, you can use data attributes. This allows you to associate data with an element without affecting its ID or other attributes.

3. JavaScript Frameworks: If you're building more complex interactive components, consider using a JavaScript framework like React, Vue, or Angular. These frameworks have built-in mechanisms for component isolation and state management, which can help avoid issues like ID conflicts.

4. Generate IDs Only When Necessary: If you decide to go with the unique ID approach, generate and append the unique suffix only when there's a potential for conflict. For example, before inserting a new component, check if an element with the desired ID already exists, and if so, generate a unique ID for the new component.

5. Clean Up: If you're concerned about leaving behind unused elements, consider adding a mechanism to clean up or remove elements that are no longer needed. For example, when a component is no longer in use, remove its associated DOM elements and event listeners.

In summary, while appending a unique suffix to IDs is a valid approach, there are other strategies you can consider based on the specific requirements and complexity of your project. The key is to ensure that the chosen approach aligns with the goals of your python-js communication interface and provides a smooth user experience.

[thorwhalen]

ipywidgets

This solution works both in browser and VSCode!

Unfortunately, it's not what we want. Here, we're making the JS+HTML via ipywidgets.
What we want is to make any JS component integrated in a notebook.
Perhaps we can do so via ipywidgets, using custom widgets

import ipywidgets as widgets
from IPython.display import display

# Python function to be called when the button is clicked
def on_button_click(button):
    global data_from_js
    data_from_js = text_input.value
    print(f"Received data: {data_from_js}")

# Create a text input widget
text_input = widgets.Text(
    value='',
    placeholder='Enter some text',
    description='Text:',
    disabled=False
)

# Create a button widget
button = widgets.Button(description="Submit")
button.on_click(on_button_click)

# Display the widgets
display(text_input, button)

[thorwhalen]

I asked bing to give me a simple example of how to make a custom widget:

Sure, I can help you with that. Here are the steps to build and run a custom widget example using Jupyter notebooks:

1. Make sure you have ipywidgets and widgetsnbextension installed in your Python environment. You can use pip or conda to install them. For example, you can run the following command in your terminal: pip install ipywidgets widgetsnbextension
2. Clone the ipywidgets GitHub repository to your local machine. You can use the following command in your terminal: git clone https://github.com/jupyter-widgets/ipywidgets.git
3. Navigate to the examples folder inside the cloned repository. You can use the following command in your terminal: cd ipywidgets/examples
4. Open the Widget Custom.ipynb notebook using Jupyter notebook. You can use the following command in your terminal: jupyter notebook Widget\ Custom.ipynb
5. Follow the instructions in the notebook to build and run the custom widget example. You will need to run some cells that contain Python code, HTML code, and JavaScript code. The notebook will explain what each cell does and how to execute it.
6. Enjoy your custom widget! You should see something like this in your notebook:

: https://ipywidgets.readthedocs.io/en/7.x/_images/WidgetCustom.png

[thorwhalen]

Troubleshooting the Ipython only solution:

from IPython.display import display, HTML, Javascript

js_code = """
function sendToPython(){
    var data = document.querySelector("#myInput").value;
    var kernel = IPython.notebook.kernel;
    kernel.execute("data_from_js = '" + data + "'");
}

document.querySelector("#myButton").addEventListener("click", function(){
    sendToPython();
});
"""

HTML_code = """
<input type="text" id="myInput" placeholder="Enter some text">
<button id="myButton">Submit</button>
<script type="text/Javascript">{}</script>
""".format(js_code)

Different cell (not necessary):

display(HTML(HTML_code))

Enter "hello" in input text field, then click "Submit" button. Then, in a different cell:

print(data_from_js)

[thorwhalen]

@andeaseme says:

Not as simple as you'd hope. They suggest following "industry standards". More info in the issue comments.

I'm trying to execute through websocket but not successful yet. I can get some info about the kernel with this:

from IPython.core.getipython import get_ipython
import json

kernel = get_ipython().kernel
cf=kernel.config['IPKernelApp']['connection_file']
cf_json = json.load(open(cf))
cf_json

I'll dig through this guy's code and see what he's got:

[andeaseme]

this guy's code wasn't fruitful. The code that is supposed to find the jupyter server and return the token didn't produce a value so I couldn't build the websocket url. Vscode jupyter is not using a server that is discoverable by from jupyter_server import serverapp.

vscode jupyter has the option to connect to a jupyter server and that would remove the mystery of where vscode hosts jupyter by spinning up a server and having the link given when starting with CLI.

[Stukova]

@thorwhalen @andeaseme @rokotyan What do you think about the idea of creating our own WebSocket?
I've asked AI the question "How to create websocket communication between javascript jupyter?".

gpt-4-0613 answer

WebSocket communication between a JavaScript application and a Jupyter Notebook can be set up using JavaScript WebSocket API and Tornado WebSockets that Jupyter Notebook uses. Unfortunately, Jupyter doesn't directly expose WebSockets to the kernel. But it is possible to create a new WebSocket server inside Jupyter and use that.

Step 1: Set up WebSocket in Jupyter

You may install 'tornado' if not already installed by using pip:

pip install tornado

You can then create a WebSocketHandler in Jupyter like:

from tornado.ioloop import IOLoop
from tornado import web, websocket

class WebSocketHandler(websocket.WebSocketHandler):
    def check_origin(self, origin):
        return True

    def open(self):
        print("Websocket Opened")

    def on_message(self, message):
        self.write_message(u"You said: " + message)

    def on_close(self):
        print("Websocket closed")

app = web.Application([
    (r'/ws', WebSocketHandler),
    ])

if __name__ == '__main__':
    app.listen(8000)
    IOLoop.instance().start()

Please note, check_origin(self, origin) function is used to disable cross-origin restriction for demonstration purposes only, it is not recommended for production.

Step 2: Connect using JavaScript

Then you can connect to this WebSocket via JavaScript in your Jupyter Notebook or other JavaScript code:

let socket = new WebSocket("ws://localhost:8000/ws");

socket.onopen = function(e) {
  alert("[open] Connection established");
  alert("Sending to server");
  socket.send("Test message from Browser");
};

socket.onmessage = function(event) {
  alert(`[message] Data received from server: ${event.data}`);
};

socket.onclose = function(event) {
  if (event.wasClean) {
    alert(`[close] Connection closed cleanly, code=${event.code} reason=${event.reason}`);
  } else {
    // e.g. server process killed or network down
    // event.code is usually 1006 in this case
    alert('[close] Connection died');
  }
};

socket.onerror = function(error) {
  alert(`[error] ${error.message}`);
};

This is a simple example where messsages are sent and received in string format. For complex data, you may convert it into JSON or other format before sending.

mistral-7b-instruct answer

To create WebSocket communication between JavaScript and Jupyter, you can use the `websockets` library in JavaScript and the `websockets` module in Jupyter. Here are the steps to create a WebSocket connection:

1. In Jupyter, import the websockets module and create a WebSocket server:

import asyncio
import websockets

async def websocket_server(websocket, path):
    async for message in websocket:
        print(f"Received message: {message}")
        await websocket.send("Hello from Jupyter!")

start_server = websockets.serve(websocket_server, "localhost", 8765)

asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()

1. In JavaScript, use the websockets library to create a WebSocket connection:

const WebSocket = require('ws');

const ws = new WebSocket('ws://localhost:8765');

ws.onmessage = (event) => {
  console.log('Received message:', event.data);
};

ws.send('Hello from JavaScript!');

1. Run the Jupyter notebook and the JavaScript code in a web browser or Node.js environment.

You should see the message "Hello from Jupyter!" printed in the Jupyter notebook and the message "Received message: Hello from Jupyter!" printed in the JavaScript console.

This is a basic example of WebSocket communication between JavaScript and Jupyter. You can modify the code to send and receive data as needed.

Sounds like it could be workable. I think it can work in VSCode, too.

[thorwhalen]

Thanks @Stukova.

We discussed this kind of plan B briefly with @rokotyan.

The thing that is a great shame is that this is exactly what jupyter notebooks use: a combination of http service(s) (to manage load/save notebooks, etc.) and web socket(s) (for communication between notebook and python kernel). Doing our own "side implementation" seems like a waste, and most of all increasing the complexity (therefore decreasing reliability).

And think of it: It is possible to have two way communication using directly ipython (so no extra dependencies or complexities). We know it's possible because it works in browser-based jupyter notebooks. (At least, insofar as the simple examples I've given in this discussion.)

That said, if we can't find a way to connect to the existing jupyter server, then, sure, using our own websocket could be a way to do it. If it's not too hard to do, it will at least be a temporary patch so that we can claim our stuff works with VsCode as well. If we do so, though, we have to make sure we make an extra effort in making that concern cleanly separated so if/when we replace it with the more robust solution, there's not too much to redo.

[andeaseme]

There's not a lot of discussions out there on the inner working of vscode jupyter but they don't force us to use their backend. We can start our own server with the standard jupyter notebook command and select this server to use in vscode. I would start here with a backend we can control and try to use what is already built in with jupyter server

[thorwhalen]

Examples of functions to bind arbitrary js variables to python (variables or callbacks):

from typing import Mapping

def kernel_assignment_execution(target, src):
    return f"""IPython.notebook.kernel.execute("{target} = '" + {src} + "'");"""

def py_variable_bind_lines(var_bind):
    """Bind JS variables to Python variables."""
    if isinstance(var_bind, str):
        var_bind = [var_bind]
    if not isinstance(var_bind, Mapping):
        var_bind = list(var_bind)
        var_bind = dict(zip(var_bind, var_bind))
    
    yield 'var kernel = IPython.notebook.kernel;'
    for js_var_name, py_var_name in var_bind.items():
        yield kernel_assignment_execution(py_var_name, js_var_name)


def add_var_bind_to_js_func(js_function_code, var_bind, sep='\n'):
    from jy.js_parse import append_to_func_body

    bind_lines = sep.join(py_variable_bind_lines(var_bind))
    return append_to_func_body(js_function_code, bind_lines)

Demo:

# DEMO:

from IPython.display import display, HTML, Javascript

js_function_code = """
function sendToPython(){
    var data = document.querySelector("#myInput_5").value;
}
"""

js_function_code = add_var_bind_to_js_func(js_function_code, {'data': 'js_data'})

js_code = js_function_code + """
document.querySelector("#myButton_5").addEventListener("click", function(){
    sendToPython();
});
"""

HTML_code = """
<input type="text" id="myInput_5" placeholder="Enter some text">
<button id="myButton_5">Submit</button>
<script type="text/Javascript">{}</script>
""".format(js_code)

js_data = None   # not even necessary to initialize, but makes checking no throw an error, but print None instead

See what the HTML_code actually is.

>>> print(HTML_code)

<input type="text" id="myInput_5" placeholder="Enter some text">
<button id="myButton_5">Submit</button>
<script type="text/Javascript">
function sendToPython(){
    var data = document.querySelector("#myInput_5").value;
var kernel = IPython.notebook.kernel;
kernel.execute("js_data = '" + data + "'");}

document.querySelector("#myButton_5").addEventListener("click", function(){
    sendToPython();
});
</script>

Note: As always with IPython.notebook.kernel.execute, it doesn't work in VsCode.