Resolved #107 -- Added support for line & cell magics by subclassing MetaKernel #109

dwijy-piggy · 2025-05-23T18:32:27Z

Subclassed MetaKernel to implement custom line and cell magic commands
Updated output methods to follow MetaKernel’s conventions
Adjusted execution logic to align with MetaKernel's do_execute function

MaxCan-Code · 2025-05-23T18:48:18Z

Thanks for the PR. You can add the MetaKernel requirement at:

Lines 23 to 25 in ef152d3

    
           install_requires=[ 
        
               "ipykernel>=6.20.0", 
        
           ],

dwijy-piggy · 2025-05-23T20:01:02Z

done

MaxCan-Code · 2025-05-23T20:51:33Z

Try squashing the remove import commit: https://youtu.be/42392W7SgnE?t=334

dwijy-piggy · 2025-05-23T21:03:22Z

done

MaxCan-Code · 2025-05-23T22:05:11Z

There's an extra commit by @martanit. You can delete that commit in git rebase -i.

…ssing MetaKernel - Subclassed MetaKernel to implement custom line and cell magic commands - Updated output methods to follow MetaKernel’s conventions - Adjusted execution logic to align with MetaKernel's do_execute function

dwijy-piggy · 2025-05-24T04:10:48Z

done

setup.py

dwijy-piggy · 2025-05-24T17:52:58Z

done

…deleted comments

MaxCan-Code · 2025-05-26T13:29:10Z

My notebook hangs when I try to run a cell. Is there a working demo I can try?

`jupyter notebook` output

[I 2025-05-26 09:10:42.242 ServerApp] Connecting to kernel dc8407c1-2312-4ce1-87cf-ed61e0fdc3d9.
[W 2025-05-26 09:10:42.254 ServerApp] The websocket_ping_timeout (90000) cannot be longer than the websocket_ping_interval (30000).
    Setting websocket_ping_timeout=30000
[W 2025-05-26 09:10:42.537 ServerApp] Timeout waiting for kernel_info reply from dc8407c1-2312-4ce1-87cf-ed61e0fdc3d9
[E 2025-05-26 09:10:42.537 ServerApp] Uncaught exception GET /api/kernels/dc8407c1-2312-4ce1-87cf-ed61e0fdc3d9/channels?session_id=4b49a55f-a99c-4b30-b627-b1c68abad581 (127.0.0.1)
    HTTPServerRequest(protocol='http', host='localhost:8888', method='GET', uri='/api/kernels/dc8407c1-2312-4ce1-87cf-ed61e0fdc3d9/channels?session_id=4b49a55f-a99c-4b30-b627-b1c68abad581', version='HTTP/1.1', remote_ip='127.0.0.1')
    Traceback (most recent call last):
      File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/tornado/web.py", line 1848, in _execute
        result = await result
                 ^^^^^^^^^^^^
      File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/jupyter_server/services/kernels/websocket.py", line 66, in get
        await super().get(kernel_id=kernel_id)
      File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/tornado/websocket.py", line 277, in get
        await self.ws_connection.accept_connection(self)
      File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/tornado/websocket.py", line 890, in accept_connection
        await self._accept_connection(handler)
      File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/tornado/websocket.py", line 930, in _accept_connection
        self.selected_subprotocol = handler.select_subprotocol(subprotocols)
                                    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
      File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/jupyter_server/services/kernels/websocket.py", line 88, in select_subprotocol
        preferred_protocol = self.connection.kernel_ws_protocol
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    AttributeError: 'NoneType' object has no attribute 'kernel_ws_protocol'
[E 2025-05-26 09:10:42.574 ServerApp] {
      "Host": "localhost:8888",
      "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) QtWebEngine/6.8.2 Chrome/122.0.6261.171 Safari/537.36"
    }
[E 2025-05-26 09:17:43.076 ServerApp] 500 GET /api/kernels/c17a455b-c217-4a9d-9c34-dbdcb22425c8/channels?session_id=be0bbd1f-a5d1-446e-9ae6-e8

dwijy-piggy · 2025-05-26T20:42:34Z

Entering the following commands into Terminal on macOS successfully set up the kernel for me:

git clone -b implement-magics https://github.com/dwijy-piggy/dyalog-jupyter-kernel
jupyter kernelspec install dyalog-jupyter-kernel/dyalog_kernel/dyalog_apl --user

If you are using just a kernel.py and kernel.json file to test this locally, the error might occur if you aren't starting the kernel using IPKernelApp.launch_instance(kernel_class=DyalogKernel) within the kernel.py file. However, cloning the implement-magics branch of my forked repo and installing the kernel from there should get it working.

MaxCan-Code · 2025-05-27T20:38:13Z

Still the same problem. I'm on Linux (NixOS).

However it works when I tried it on Windows.

dwijy-piggy · 2025-05-27T20:52:45Z

Could you send the output from this command: jupyter console --kernel=dyalog_apl --debug?

MaxCan-Code · 2025-05-27T21:31:44Z

The main branch displays the same SyntaxWarning: invalid escape sequence message, then proceeds to work normally.

`jupyter console --kernel=dyalog_apl --debug`

$ jupyter console --kernel=dyalog_apl --debug
[ZMQTerminalIPythonApp] Searching ['/home/user/p/ad/dyalog-jupyter-notebooks/.venv/etc/jupyter', '/home/user/.jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
[ZMQTerminalIPythonApp] Looking for jupyter_config in /etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /usr/local/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /home/user/.jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /home/user/p/ad/dyalog-jupyter-notebooks/.venv/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /usr/local/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /home/user/.jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /home/user/p/ad/dyalog-jupyter-notebooks/.venv/etc/jupyter
[ZMQTerminalIPythonApp] Connection File not found: /home/user/.local/share/jupyter/runtime/kernel-21830.json
[ZMQTerminalIPythonApp] Instantiating kernel 'Dyalog APL' with kernel provisioner: local-provisioner
[ZMQTerminalIPythonApp] Starting kernel: ['/home/user/p/ad/dyalog-jupyter-notebooks/.venv/bin/python3', '-m', 'dyalog_kernel', '-f', '/home/user/.local/share/jupyter/runtime/kernel-21830.json']
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:46129
[ZMQTerminalIPythonApp] connecting iopub channel to tcp://127.0.0.1:35829
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:35829
[ZMQTerminalIPythonApp] connecting shell channel to tcp://127.0.0.1:37835
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:37835
[ZMQTerminalIPythonApp] connecting stdin channel to tcp://127.0.0.1:35037
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:35037
[ZMQTerminalIPythonApp] connecting heartbeat channel to tcp://127.0.0.1:44739
[ZMQTerminalIPythonApp] connecting control channel to tcp://127.0.0.1:46129
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:46129
/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/dyalog_kernel/kernel.py:205: SyntaxWarning: invalid escape sequence '\d'
  if re.match('^Dyalog-\d+\.\d+\.app$', d):
/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/dyalog_kernel/kernel.py:209: SyntaxWarning: invalid escape sequence '\d'
  if re.match('^\d+\.\d+$', v):
/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/dyalog_kernel/kernel.py:298: SyntaxWarning: invalid escape sequence '\s'
  match = re.search('^%suspend\s+(\w+)$',lines[0].lower(), re.IGNORECASE)
Traceback (most recent call last):
  File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/jupyter_core/utils/__init__.py", line 154, in wrapped
    asyncio.get_running_loop()
    ~~~~~~~~~~~~~~~~~~~~~~~~^^
RuntimeError: no running event loop

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/jupyter_console/ptshell.py", line 434, in init_kernel_info
    reply = self.client.get_shell_msg(timeout=1)
  File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/jupyter_core/utils/__init__.py", line 158, in wrapped
    return loop.run_until_complete(inner)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
  File "/home/user/.local/share/uv/python/cpython-3.13.2-linux-x86_64-gnu/lib/python3.13/asyncio/base_events.py", line 725, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/jupyter_client/client.py", line 143, in _async_get_shell_msg
    return await ensure_async(self.shell_channel.get_msg(*args, **kwargs))
                              ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/jupyter_client/channels.py", line 232, in get_msg
    raise Empty
_queue.Empty

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/bin/jupyter-console", line 10, in <module>
    sys.exit(main())
             ~~~~^^
  File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/jupyter_core/application.py", line 284, in launch_instance
    super().launch_instance(argv=argv, **kwargs)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/traitlets/config/application.py", line 1074, in launch_instance
    app.initialize(argv)
    ~~~~~~~~~~~~~~^^^^^^
  File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/traitlets/config/application.py", line 118, in inner
    return method(app, *args, **kwargs)
  File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/jupyter_console/app.py", line 136, in initialize
    self.init_shell()
    ~~~~~~~~~~~~~~~^^
  File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/jupyter_console/app.py", line 106, in init_shell
    self.shell = ZMQTerminalInteractiveShell.instance(parent=self,
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
                    manager=self.kernel_manager,
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                    client=self.kernel_client,
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
                    confirm_exit=self.confirm_exit,
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/traitlets/config/configurable.py", line 583, in instance
    inst = cls(*args, **kwargs)
  File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/jupyter_console/ptshell.py", line 352, in __init__
    self.init_kernel_info()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/user/p/ad/dyalog-jupyter-notebooks/.venv/lib/python3.13/site-packages/jupyter_console/ptshell.py", line 437, in init_kernel_info
    raise RuntimeError("Kernel didn't respond to kernel_info_request") from e
RuntimeError: Kernel didn't respond to kernel_info_request

MaxCan-Code · 2025-05-28T12:39:11Z

Still the same problem. I'm on Linux (NixOS).

However it works when I tried it on Windows.

@xpqz can you see if it works on your end? Thanks

MaxCan-Code · 2025-06-02T14:29:31Z

Still the same problem. I'm on Linux (NixOS).
However it works when I tried it on Windows.

@martanit @sloorush can you test this as well?

martanit · 2025-06-04T06:35:45Z

Works for me on Windows 👍!

xpqz · 2025-06-11T08:56:22Z

It would be good to see an outline of the consequences of switching from ipykernel.kernelbase to MetaKernel as the baseclass. Whilst convenient, are there any implications or future limitations we should be aware of?

dwijy-piggy changed the title ~~Resolved #107 -- Added support for line & cell magics by subclassing MetaKernel~~ removed unnecessary import May 23, 2025

dwijy-piggy changed the title ~~removed unnecessary import~~ Resolved #107 -- Added support for line & cell magics by subclassing MetaKernel May 23, 2025

dwijy-piggy force-pushed the implement-magics branch 2 times, most recently from 80342d9 to f3cebc6 Compare May 23, 2025 19:38

dwijy-piggy mentioned this pull request May 23, 2025

Resolve 107 -- Add support for line & cell magics by subclassing MetaKernel #108

Closed

dwijy-piggy force-pushed the implement-magics branch from f3cebc6 to a402e3d Compare May 23, 2025 21:02

dwijy-piggy added 2 commits May 23, 2025 23:10

added requirement for MetaKernel

719b48d

dwijy-piggy force-pushed the implement-magics branch from a402e3d to 719b48d Compare May 24, 2025 04:10

MaxCan-Code reviewed May 24, 2025

View reviewed changes

setup.py Outdated Show resolved Hide resolved

removed redundant ipykernel requirement and restored unintentionally …

b26bc21

…deleted comments

dwijy-piggy force-pushed the implement-magics branch from bdbe457 to b26bc21 Compare May 24, 2025 19:31

Resolved #107 -- Added support for line & cell magics by subclassing MetaKernel #109

Are you sure you want to change the base?

Resolved #107 -- Added support for line & cell magics by subclassing MetaKernel #109

Uh oh!

Conversation

dwijy-piggy commented May 23, 2025

Uh oh!

MaxCan-Code commented May 23, 2025

Uh oh!

dwijy-piggy commented May 23, 2025

Uh oh!

MaxCan-Code commented May 23, 2025

Uh oh!

dwijy-piggy commented May 23, 2025

Uh oh!

MaxCan-Code commented May 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dwijy-piggy commented May 24, 2025

Uh oh!

Uh oh!

dwijy-piggy commented May 24, 2025

Uh oh!

MaxCan-Code commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dwijy-piggy commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MaxCan-Code commented May 27, 2025

Uh oh!

dwijy-piggy commented May 27, 2025

Uh oh!

MaxCan-Code commented May 27, 2025

Uh oh!

MaxCan-Code commented May 28, 2025

Uh oh!

MaxCan-Code commented Jun 2, 2025

Uh oh!

martanit commented Jun 4, 2025

Uh oh!

xpqz commented Jun 11, 2025

Uh oh!

Uh oh!

MaxCan-Code commented May 23, 2025 •

edited

Loading

MaxCan-Code commented May 26, 2025 •

edited

Loading

dwijy-piggy commented May 26, 2025 •

edited

Loading