Shader Issues and Observations in Py5

[ruslye]

I’ve been experimenting with shaders in Py5 using py5.shader() and faced several challenges and unexpected results. Below is a summary of the main issues and observations:

1. Fragment Shader Works, But Vertex Shader Causes Issues

• Applying py5.shader() with both fragment and vertex glsl shader. I customise this shader to just simply color the faces and vertices of 3d cube in red.
  The result is a black wireframe cube with no visible faces.
• However, loading only the fragment shader, the cube renders in red with black wireframe. This suggests there may be problems with handling custom vertex shaders in Py5.

2. Shader Errors Are Hard to Debug

• Py5 doesn’t surface errors when shaders fail (e.g., compilation or linking issues).
  Example: When an invalid GLSL statement was added intentionally, the error was only visible by digging into the console logs.
• Issues like gl_FragColor is not accessible in this profile suggest possible compatibility problems with GLSL or OpenGL, but they aren’t reported clearly.

3. Differences Between py5.shader() and py5.apply_filter()

• Results vary when applying the same shader via py5.shader() vs. py5.apply_filter(), and it’s unclear how they differ. Clarification on their usage would be helpful.

Some suggestions that could be helpful:

1. Error Reporting: Provide clearer error messages for shader compilation or runtime issues.
2. Documentation: Share examples and guidance, especially for using vertex shaders and handling transformations (projection, view, model matrices).
3. Shader Debugging Tools: Add an optional function to help users find shader-related issues during runtime.

System Details:

• Py5 Version: 0.10.4a2
• Python Version: 3.12
• Operating System: Win11
• Graphics Driver and OpenGL Version: nVidia driver ver: 560.94, open gl 4.6

Thank you for maintaining Py5! I hope this feedback helps enhance shader support in future releases. Let me know if any additional information is needed.
Best regards,
Russell

[hx2A]

Hi, @ruslye ! Thank you for your analysis and feeback.

1. Fragment Shader Works, But Vertex Shader Causes Issues

I've been using both Fragment and Vertex shaders with py5 and have had great success with both. However, I almost always use Linux or MacOS. I've never written shaders on Windows. It could be this is some kind of platform specific bug, but I'd be surprised if something like this did work on Linux and MacOS and not Windows. Can you provide a code example for us to look at?

Also, are you using Windows 11 + Linux Subsystem or is this just Windows?

2. Shader Errors Are Hard to Debug

Right, they are. That gets on my nerves also. I do see compilation errors though. But perhaps errors are reported differently on Windows?

py5 does "prune" error messages to try to hide its own internals and Java stuff because showing a Java exception to someone learning Python is misleading and confusing. Can you try deactivating error pruning and seeing if the errors change?

https://py5coding.org/reference/py5functions_prune_tracebacks.html

To some extent there isn't much I can do about OpenGL errors because those kinds of errors aren't reported well to begin with. However, py5 shouldn't be filtering out any part of those error messages. The error messages should be the same as what you'd get for an analogous Sketch running in Processing.

3. Differences Between py5.shader() and py5.apply_filter()

Yes, they are different methods with different functionality. Both are documented, but I see the documentation pages are a bit sparse, especially given how useful and powerful these methods are. We can certainly add to these descriptions. Based on what you've learned about py5 so far, how would you explain the differences to someone else who is also learning py5?

https://py5coding.org/reference/sketch_apply_filter.html
https://py5coding.org/reference/sketch_shader.html

Thank you for maintaining Py5! I hope this feedback helps enhance shader support in future releases. Let me know if any additional information is needed.

You are welcome! I do appreciate your thoughts here. It is through design conversations like this that we are able to provide a well maintained library.

[ruslye]

Hi, totally skipped my mind to include the codes, but here they are:
Sketch:

import py5
from py5 import run_sketch

x = 250
y = 250
z = 0
rot = 0

def setup():
    py5.size(500, 500, py5.P3D)

    global red_shader
    red_shader = py5.load_shader("frag_shader_red.frag", "vert_shader_red.vert")
    # red_shader = py5.load_shader("frag_shader_red.frag")

def draw():
    global x, y, z, rot

    # py5.apply_filter(red_shader)  # result - white cube, black edges
    py5.shader(red_shader)  # result - black wireframe cube

    py5.background('#004477')

    py5.translate(x, y, z)
    py5.rotate_y(rot / 100)
    py5.rotate_x(rot / 100)

    py5.box(100)

    rot += 1  # rotation factor

run_sketch()

vertex shader:

#version 330 core
#extension GL_ARB_separate_shader_objects : enable

// Input vertex attributes
layout(location = 0) in vec3 position; // Position attribute

// Output to the fragment shader
out vec4 vertexColor;

uniform mat4 projection;
uniform mat4 view;
uniform mat4 model;

void main()
{ 
    gl_Position = projection * view * model * vec4(position, 1.0);

    vertexColor = vec4(1.0, 0.0, 0.0, 1.0); // Set vertices to red
}

fragment shader:

#version 330 core

// Input from the vertex shader
in vec4 vertexColor;

// Output color
out vec4 FragColor;

void main()
{
   FragColor = vec4(1.0, 0.0, 0.0, 1.0);
   // FragColor = vertexColor;
}

With vertex and fragment shader:

With fragment shader only:

I am using Pycharm 2024.3.2 CE on just Windows, no WSL.
I'm not sure if I have missed anything in the shader codes though I did a few different iterations. So far this is the closest that I could get.
Thanks for the prune traceback, I'll look into it. Hope to shed some light into this.

[hx2A]

Hi @ruslye , I am pressed for time but I wanted to reply and help you here.

There might be a problem with the vertex shader. Processing shaders are a bit different from shaders you find elsewhere like on ShaderToy because there is a Processing framework you need your shader to fit into. To make sure I get it right, I always start shader coding with one of Processing's shaders and modify it for what I want to do. You can find them here:

https://github.com/processing/processing4/tree/main/core/src/processing/opengl/shaders

First you need to know if your sketch does or does not use lighting and if it does and does not use textures. Here you are using neither, you want ColorVert and ColorFrag. Try copying both of those and using those in your Sketch. If it works and gives identical results to what you'd get with no shader code, then you know that Vertex shaders work fine on your machine.

Once you have that working, try editing the code for what you want to do. If it stops working, you know your latest code change broke something.

[hx2A]

@villares , can you cite any good documentation for using shaders in Processing? Everything I've learned is from trial and error, and from studying Processing's source code.

[villares]

@villares , can you cite any good documentation for using shaders in Processing? Everything I've learned is from trial and error, and from studying Processing's source code.

Sorry, I don't know much about shaders. My friends always mention https://thebookofshaders.com/ but I'm afraid it doesn't delve into using shaders with Processing.

A search at the old forums showed up this archived PShader tutorial, I'm not sure if this would help, but here it is: https://web.archive.org/web/20210117181532/https://processing.org/tutorials/pshader/ (it mentions this code repo https://github.com/codeanticode/pshader-tutorials)

[hx2A]

There is a lack of current Shader documentation. I remember reading about an effort to address that somewhere in Discord but I just searched there and didn't find anything worth sharing here. I don't know if anyone will invest much time in it because Processing does want to move away from the current JOGL opengl dependencies and use something else. When that happens, the details of how to use shaders will change.

[ruslye]

Thanks hx2A, villares, the references you gave are helpful. The shaders are indeed written differently. I did manage to inject a color to the wireframe using Processing's point and line shaders documentation though I'll need to look deeper to understand.
Appreciate all your effort looking into this.

[villares]

@ruslye I hope you don't mind if we move this into discussions ?