3D plot atoms are black without shading or lighting

[Irratzo]

Description

In a Jupyter notebook, ase_view.make_render(atoms), which uses pythreejs (version pythreejs==2.4.2) underneath, paints atoms as black spheres, with no lighting or shading. Plotting the same node with ASE standard viewer instead, in the same Python environment, paints atoms with lighting and shading (ase.visualize.view(atoms, viewer='x3d')). So, this concerns the pythreejs 3D plots, not the SVG 2D plots. Those work without issues. At first glance, it looked as if the ambient light might be missing. See attached screenshot. But that was not it, see below.

Minimal working example (MWE)

Install Python environment as described below.

Then run ase-notebook by. example. In the pythreejs plots, the atoms should appear black, in the SVG plots they appear in the right color.

System & version info

ase-notebook v0.3.1, fork PhilippRue/ase-notebook, latest commit.

Created virtualenv environment for ase-notebook from its setup.py with Python version 3.10.9. Note: Did not install all extras at once (pip install -e ".[threejs,svgconcat,svg2pdf,testing,code_style,flake8_plugins,docs]") casue that led to unresolved environment.

cd ase-notebook
python -m venv venv
source venv/bin/activate

pip install --upgrade pip
pip install -e .
pip install pytest coverage pytest-cov # [testing]
pip install -e ".[threejs,svgoncat,svg2pdf]"
# # only for testing effect of commit 4c856b11 mentioned above
# pip install "importlib_resources<2"

Resulted in environment pip freeze > requirements.txt.

Analysis 1

All tests passed.

$ pytest -v

ase_notebook/color.py::ase_notebook.color.Color PASSED [2%]
[...]
tests/test_backend/test_threejs.py::test_create_arrow_texture PASSED [100%]

The colors are there.

ase_view.get_atom_colors(atoms)
['#DF6633',
 '#DF6633',
 '#DF6633',
 '#DF6633',
 '#FFFF2F',
 '#FFFF2F',
 '#FFFF2F',
 '#FFFF2F',
 '#FFFF2F',
 '#FFFF2F',
 '#FFFF2F',
 '#FFFF2F']

This also means that the underlying data.load_data_file function works irrespective of the removal of dependency importlib_resources in that commit. Tested it with version before and after that change, same behavior.

Currently suspect that it the cause is a newer version with some of the dependencies involved in the 3D plotting, maybe pythreejs, ipywidgets, etc. Will try to downgrade some of them and see if the issue disappears.

[Irratzo]

Analysis 2

Comparing Python envs dependency versions between used ase-notebook commit's L=requirements-lock-dev.txt and used Python env R=requirements.txt from this issue, post 1.

pythreejs.

• L: pythreejs==2.1.1
• R: pythreejs==2.4.2

Creating a second venv, called venv2, same commands as in post 1, but with the setup.py line "threejs": ["pythreejs>=2.1,<3", changed to "threejs": ["pythreejs~=2.1.1".

A diff on !pip list | grep -e "ipy" -e "jupyter" -e "threejs" -e "ase" between both environments venv (above) and venv2 (here) reveals that only the pythreejs version is different.

!pip list | grep -e "ipy" -e "jupyter" -e "notebook" -e "threejs" -e "svg" -e "ase"
ase                      3.22.1
ase-notebook             0.3.2   /Users/wasmer/src/github.com/philipprue/ase-notebook
ipydatawidgets           4.3.3
ipykernel                6.23.1
ipython                  8.13.2
ipython-genutils         0.2.0
ipywidgets               7.7.5
jupyter_client           8.2.0
jupyter_core             5.3.0
jupyter-events           0.6.3
jupyter_server           2.6.0
jupyter_server_terminals 0.4.4
jupyterlab-pygments      0.2.2
jupyterlab-widgets       1.1.4
notebook                 6.5.4
notebook_shim            0.2.3
pythreejs                2.4.2
scipy                    1.10.1
svglib                   0.9.4
svgwrite                 1.4.3

Repeating the MWE in a Jupyter notebook with this env, instead, failed in showing the 3D plot altogether, instead this warning was printed.

~/src/github.com/philipprue/ase-notebook/venv2/lib/python3.10/site-packages/jupyter_client/session.py:719: UserWarning: Message serialization failed with:
Out of range float values are not JSON compliant
Supporting this message is deprecated in jupyter-client 7, please make sure your message is JSON-compliant
  content = self.pack(content)

In the shell running the notebook server, this exception ocurred.

tornado.websocket.WebSocketClosedError
[E 15:55:50.132 NotebookApp] Uncaught exception in ZMQStream callback
[...]
      File "~/src/github.com/philipprue/ase-notebook/venv2/lib/python3.10/site-packages/tornado/iostream.py", line 182, in advance
        assert 0 < size <= self._size
    AssertionError
[E 15:55:50.133 NotebookApp] Uncaught exception in zmqstream callback
[...]

[Irratzo]

Analysis 3

pytest -v also issues some warnings, in both environments (venv, venv2). Maybe one of those is at fault.

============================================================================================================ warnings summary ============================================================================================================
venv/lib/python3.10/site-packages/_pytest/config/__init__.py:1302
  /Users/wasmer/src/github.com/philipprue/ase-notebook/venv/lib/python3.10/site-packages/_pytest/config/__init__.py:1302: PytestConfigWarning: Unknown config option: timeout

    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

tests/test_viewer.py::test_get_atom_colors
  /Users/wasmer/src/github.com/philipprue/ase-notebook/ase_notebook/viewer.py:178: MatplotlibDeprecationWarning: The get_cmap function was deprecated in Matplotlib 3.7 and will be removed two minor releases later. Use ``matplotlib.colormaps[name]`` or ``matplotlib.colormaps.get_cmap(obj)`` instead.
    cmap = get_cmap(cmap)

tests/test_backend/test_threejs.py::test_make_render
  /Users/wasmer/src/github.com/philipprue/ase-notebook/venv/lib/python3.10/site-packages/jupyter_client/connect.py:20: DeprecationWarning: Jupyter is migrating its paths to use standard platformdirs
  given by the platformdirs library.  To remove this warning and
  see the appropriate new directories, set the environment variable
  `JUPYTER_PLATFORM_DIRS=1` and then run `jupyter --paths`.
  The use of platformdirs will be the default in `jupyter_core` v6
    from jupyter_core.paths import jupyter_data_dir, jupyter_runtime_dir, secure_write

tests/test_backend/test_threejs.py::test_make_render
  /Users/wasmer/src/github.com/philipprue/ase-notebook/venv/lib/python3.10/site-packages/traittypes/traittypes.py:20: DeprecationWarning:
              Sentinel is not a public part of the traitlets API.
              It was published by mistake, and may be removed in the future.

    Empty = Sentinel('Empty', 'traittypes',

tests/test_backend/test_threejs.py::test_make_render
  /Users/wasmer/src/github.com/philipprue/ase-notebook/venv/lib/python3.10/site-packages/ipywidgets/widgets/widget.py:477: DeprecationWarning: Passing unrecognized arguments to super(HTML).__init__(color='grey').
  object.__init__() takes exactly one argument (the instance to initialize)
  This is deprecated in traitlets 4.2.This error will be raised in a future release of traitlets.
    super(Widget, self).__init__(**kwargs)

TODO investigate.

[Irratzo]

Solution attempt 1

Status: FAILED.

Tried to create yet a third Python environment, this one called venv3, this time directly from ase-notebook/requirements-lock-dev.txt. Env creation failed with error. Probably because it was created on a Linux OS, while I am on MacOS.

python -m venv venv3
source venv3/bin/activate
pip install --upgrade pip
pip install -r requirements-lock-dev.txt

Errour output.

  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.

[...]
      Building wheels for collected packages: numpy
[...]
        error: subprocess-exited-with-error
[...]
            error: Command "clang -Wno-unused-result [...] failed with exit status 1
[...]
      ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects

[Irratzo]

Solution attempt 2

Status: TODO.

Downgrade all visualization-relevant packages until issue disappears.

Packages included: all from !pip list | grep -e "ipy" -e "jupyter" -e "notebook" -e "threejs" -e "svg" -e "ase".

[Irratzo]

Analysis 4

Document side issues of generating pythreejs plots in classical notebook / JupyterLab with different venvs.

Noticed that in both environments venv and venv2, there are issues even with simple plotting of pythreejs plots, unreladed to ase-notebook. So, in order to focus on the issue at hand here, have to define envs where these issues don't appear. So, document them here and try to get rid of them first.

Sometimes, plot does not appeear, rerun cell, plot appears. Sometimes need to restart kernel instead, and skip previous viz cells.

Problems.

• venv venv, interface notebook:

  • ase-notebook make_render: plot does appear most often, black atoms, just as in issue described here.
  • pythreejs example: plot appears, shaded ball, just as it should.

• venv venv, interface jupyter-lab:

  • ase-notebook make_render: throws ValueError

    ValueError [...]
    gui = ase_view.make_render( [...]
    File ~/src/github.com/philipprue/ase-notebook/ase_notebook/viewer.py:558, in AseView.make_render [...]
    --> 558 container = RenderContainer(renderer, element_renderer=renderer, **key_elements)
    

  • pythreejs example: throws JS error

    [Open Browser Console for more detailed log - Double click to close this message]
    Failed to load model class 'RendererModel' from module 'jupyter-threejs'
    loadClass@http://localhost:8891/lab/extensions/@jupyter-widgets/jupyterlab-manager/static/134.40eaa5b8e976096d50b2.js?v=40eaa5b8e976096d50b2:1:74856
    loadModelClass@http://localhost:8891/lab/extensions/@jupyter-widgets/jupyterlab-manager/static/150.b0e841b75317744a7595.js?v=b0e841b75317744a7595:1:10729
    _make_model@http://localhost:8891/lab/extensions/@jupyter-widgets/jupyterlab-manager/static/150.b0e841b75317744a7595.js?v=b0e841b75317744a7595:1:7517
    

• venv venv2, interface notebook:

  • ase-notebook make_render: plot does not appear, no error message
  • pythreejs example: plot appears, but nothing is visible, all is black (red sphere, black background)
    • This actually looks closest to what is happening in the original issue. Problem is, I cannot change the background color from black to check this (ie, whether the ball is shown but is black, or wether the ball is not rendered at all, only the black background.) TODO

• venv venv2, interface jupyter-lab:

  • ase-notebook make_render: throws ValueError on RenderContainer, like above
  • pythreejs example: throws JS error, same as above

[Irratzo]

Analysis 5

Debug ase-notebook w.r.t. threejs ambient light setting.

This was done in environment venv, interface jupyter notebook, as described above.

The "black atoms" plots looks like the ambient light is missing. In ase-notebook 3D plots, this is set in AseView --> make_render --> generate_3js_render --> scene.add([camera, ambient_light, direct_light]), here.

So, debug make_render and watch what is happening at this code location during runtime.

Strange: Just plotted the final visualization objects created in generate_3js_render, the scene object and the renderer object, which is returned. The scene object renders the atoms correctly, while the renderer object paints the atoms black. See attached screenshot.

Selectively removing components from the scene object and the renderer object or adding others does not lift the issue. Plotting scene objects individually renders them correctly (single atoms, goups of atoms, bonds, miller planes). So, it really seems as if the pythreejs Renderer class is responsible for this issue.

[Irratzo]

Solution attempt 3

Status: FAILED.

Recreate environment from requirements-lock-dev.txt indirectly.

Since recreating original ase-notebook dev environment directly with pip install -r requirements-lock-dev.txt failed on my machine, trying to recreate it indirectly, esp. wrt the dependencies involved in pythreejs 3D plots.

I did that this way. 1) Duplicate setup.py section install_requires and extras_require, comment out original. 2) In copy, replace all version specifications of dependencies listed there with exact version version constraints from requirements-lock-dev.txt. 2) To install_requires, add dependencies from pip list | grep -e "ipy" -e "jupyter" -e "threejs" -e "ase", and set their version constraints also to the exact version constraints in requirements-lock-dev.txt. If a dependency from that list is not listed in requirements-lock-dev.txt, either put not version constraint or omit it. 4) Create env called venv3 (replaces failed attempt at venv3 above). 4) Add more version constraint if anything is missing.

This process gave me this setup.py requirements section.

    # # install_require and extras_require matched to requirements-lock-dev.txt versions
    # # of threejs 3D plot dependencies (-e "ipy" -e "jupyter" -e "threejs" -e "ase")
    install_requires=[
        # core
        "ase==3.18.1",
        "attrs==19.3.0",
        # "numpy==1.16.5",
        # used for color-map
        # TODO use color-map package, with no matplotlib dependency?
        "matplotlib==3.1.1",
        # svg
        "svgwrite==1.3.1",
        # #
        # # additional dependencies relevant for threejs 3D plots, with
        # # pinned versions from requirements-lock-dev. defined as:
        # # pip list | grep -e "ipy" -e "jupyter" -e "threejs" -e "ase"
        # #
        "ipydatawidgets==4.0.1",     # via pythreejs
        "ipykernel==5.1.3",          # via ipywidgets, notebook
        "ipython==7.9.0",            # via ipykernel, ipywidgets
        "ipython-genutils==0.2.0",   # via nbformat, notebook, traitlets
        "ipywidgets==7.5.1",         #
        "jupyter-client==5.3.4",     # via ipykernel, notebook
        "jupyter-core==4.6.1",       # via jupyter-client, nbconvert, nbformat, notebook
        "jupyterlab",                #
        "pythreejs==2.1.1",          #
        # try resolving jupyter notebook not working
        "jinja2==2.10.3",
        "markupsafe==1.1.1",
    ],
    extras_require={
        "threejs": ["pythreejs==2.1.1", "ipywidgets==7.5.1"],
        "svgconcat": ["svgutils==0.3.1"],
        # "svg2pdf": ["svglib==0.9.3", "reportlab==3.5.32"],
        "svg2pdf": ["svglib==0.9.3"],
        "testing": [
            "coverage==4.5.4",
            "pytest==3.10.1",
            "pytest-cov==2.8.1",
            # "pytest-regressions",
        ],
        "code_style": [
            "black==19.3b0",
            "pre-commit==1.17.0",
            "flake8==3.7.9",
            "doc8==0.8.0",
            "pygments==2.0.2",
        ],
        "flake8_plugins": [
            "flake8-comprehensions==3.0.1",
            "flake8-docstrings==1.5.0",
            "flake8_builtins==1.4.1",
            "import-order==0.0.11",
        ],
        "docs": [
            "sphinx==2.2.1",
            "sphinx_rtd_theme==0.4.3",
            "ipypublish==0.10.10",
            "ipython==7.9.0",
        ],
    },

I expected that these would recreate as exactly as possible the orginal dev environment, so the ase-notebook 3D plots are most likely to work there. However,

• venv venv, interface notebook:

  • ase-notebook make_render: Plot window either does not appear at all, no error message in notebook or console, or plot window appears, but just showing the background color.
  • pythreejs example: Plot does not appear, or only with background color.

• venv venv, interface jupyter-lab:

  • ase-notebook make_render: Plot does not appear, instead just text Renderer(camera=CombinedCamera(height=400.0, position=(0.0, 0.0, 60.0), width=600.0), controls=[OrbitControls(… is printed. No error message in notebook or console.
  • pythreejs example: Plot does not appear, instead just text Preview(child=Mesh(geometry=SphereGeometry(), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='r… is printed. No error message in notebook or console.

I declare this attempt at more accurate reproduction of original ase-notebook dev env on my computer with venv3 as failed.

To resolve the original issue, continue to use env called venv, interface jupyter notebook above instead, where I could reproduce the issue.

[Irratzo]

Solution attempt 4

Status: FAILED.

Copied env venv as venv-copy. There, downgraded pythreejs to ase-notebook dev env's version from pythreejs==2.4.2 to pythreejs==2.1.1. Then ran examples in interface jupyter notebook again. For both ase-notebook and pythreejs examples, the plot did not show or empty plot showed with warning

/Users/wasmer/src/github.com/philipprue/ase-notebook/venv-copy/lib/python3.10/site-packages/jupyter_client/session.py:719: UserWarning: Message serialization failed with:
Out of range float values are not JSON compliant
Supporting this message is deprecated in jupyter-client 7, please make sure your message is JSON-compliant
  content = self.pack(content)

Then downgraded other dependencies one by one, down to those of ase-notebook dev env's version: jupyter_client, ipywidgets, ipydatawidgets`. Effect: ase-notebook and pythreejs examples plot only showed background, no 3D objects, as above, no error messages.

Giving up for now.

Maybe try again in env venv above, Solution attempt: Debug ase-notebook w.r.t. threejs ambient light setting., to investigate state of the atoms in the renderer object, try to find out if their color is black or something.

Problem is that I have no environment right now where ase-notebook 3D plots are actually working, so that I could compare the "sould" with the "is" case.

Giving up for now, return to this later.

[Irratzo]

Solution attempt 5

STATUS: SUCCESS. But only as a temporary fix.

Looked into pythreejs issues. It currently seems most likely that pythreejs issue #389, Javascript error when rendering with version 2.4.1, is the cause of this present ase-notebook issue, or is at least connected to it.

Here, user "moorepants" reports the same JS error that I have described with interface jupyter-lab in Analysis 4 above.

Here, user "moorepants" reports that they could resolve the issue with a temporary fix of pinning certain pythreejs dependencies as python -m pip install ipywidgets==7.7.2 jupyterlab_widgets==1.1.1 pythreejs==2.3.0 widgetsnbextension==3.6.1. User "moorepants" also suspects that the underlying issue is a change in an external JavaScript library.

The only thing that can change, I suppose, is that when you navigate to that page a javascript lib is downloaded that isn't pinned to a version that worked when it was built. I can even go back to a version of that page that was built like a year ago and it no longer works

Further below in that issue's comment thread, there are some more package version pinning suggestions.

---

It tried out the suggested version pinning above in a new branch issue-53-tmp-fix, here. In this new environment venv2, I tested the ase-notebook example again, in interface jupyter notebook. Indeed, the black atoms issue disappears, the example plots correctly again!

However, in the shell running the Jupyter server, the tornado exception already described in Analysis 2, occurred.

[E 12:54:59.031 NotebookApp] Exception in callback functools.partial(<function ZMQStream._update_handler.<locals>.<lambda> at 0x12068ed40>)
    Traceback (most recent call last):
      File "/Users/wasmer/src/github.com/philipprue/ase-notebook/venv2/lib/python3.10/site-packages/tornado/ioloop.py", line 738, in _run_callback
        ret = callback()
[...]
      File "/Users/wasmer/src/github.com/philipprue/ase-notebook/venv2/lib/python3.10/site-packages/tornado/iostream.py", line 182, in advance
        assert 0 < size <= self._size
    AssertionError

Nevertheless, it seems that using this branch issue-53-tmp-fix can at least be used as an alternative now. However, only as a temporary fix.

This ase-notebook issue can probably only be closed as soons as the referenced pythreejs issue has been closed, as well. When that happens, run the according pythreejs update, or adopt the solution there.