Add 3D material and lighting APIs

[ziyaointl]

This issue describes my proposal to add 3D material and lighting APIs to p5py. Please leave a comment if you have any suggestions!

General Implementation Note

The vertex shaders and fragment shaders discussed in this issue will be added to shaders3d.py. By altering the default_prog in Renderer3D via method set_shaders, we'll be able to change the current shaders that are in effect.

The lights mentioned in this issue will be described by custom structs and passed to fragment shaders in an array of such structs. Then, the blinn-phong shader will be able to loop through them and add the contribution of each light to the final result.

Given that GLSL does not allow varying array size, we may need to set a limit to the max number of lights a scene could have (e.g. 64?).

Current API in p5.js and Processing

The p5.js material API is not particularly flexible in that it forces the user to choose between individual materials like ambientMaterial and specularMaterial when in reality all of the materials can be united under the Blinn-Phong model.
The Processing Language offers better flexibility in offering individual ambient, emissive, shininess, and specular. Therefore, I suggest we go with the Processing API.

Proposed API Additions to p5py

Implement normal_material()

The is a function that exists in p5.js but not in Processing. It assigns a color to a pixel solely based on the normal vector of the fragment being rendered. Useful for debugging, so this will be one of the first shaders that I'll implement.

Implement basic_material(r, g, b)

This is the default material when fill is called. Returns a uniform color.

Implement ambient(r, g, b)

Sets the ambient light color reflected by the material. This is sometimes called the ambient coefficient.

Implement emissive(r,g,b)/diffuse(r, g, b)

Sets the diffuse light color reflected by the material.

Design note: I think emissive is rather a misnomer because the material itself does not act as a light source but rather reflect light from light sources. Given that emissive modifies the color of diffuse reflections in the Blinn-Phong model, I propose to add an alias called diffuse that calls the same function under the hood.

Implement shininess(p)

Sets the amount of gloss of the material. It is the exponential above the cosine term in Blinn-Phong

Implement specular(r, g, b)

Sets the specular light color reflected by the material.

Implement blinn_phong_material(r, g, b)

This is the material being applied whenever ambient, diffuse, shininess, or specular is called.

Blinn-Phong shading can be decomposed into three parts:
ambient, diffuse, and specular.

The ambient component is essentially a constant term that is alway present. We calculate it by summing all the ambient lights in a scene and multiplying it with the normalized ambient coefficent set by ambient.

The diffuse component takes the normal vector of a surface into account and varies how much light is reflected depending on the angle that the surface makes with the incoming light.

The specular component not only accounts for the direction of the light (like the diffuse component) but also the direction of the viewer. If the viewer is not on the path of the reflected light, the specular component falls off quickly, producing the glossy reflections we see on some materials.

The color shown on the user's screen is the sum of all three components. Here's a nice visualization of the components being progressively added.

Implement lights()

This is a wrapper for setting up default lights

def lights():
    ambient_light(128, 128, 128)
    directional_light(128, 128, 128, 0, 0, -1)
    light_falloff(1, 0, 0)

Implement ambient_light(r, g, b)

Adds an ambient light to the list of lights. Participates in ambient lighting calculations.

Implement directional_light(r, g, b, x, y, z)

Adds a directional light to the list of lights. Participates in diffuse & specular lighting calculations.

Implement point_light(r, g, b, x, y, z)

Adds a point light to the list of lights. Participates in diffuse & specular lighting calculations.

Implement light_falloff(constant, linear, quadratic)

Sets the falloff rates for point lights and ambient lights that specifies a location.

d = distance from light position to vertex position
falloff = 1 / (CONSTANT + d * LINEAR + (d*d) * QUADRATIC)

Note that like the Processing Language, directional lights are not affected because directional lights don't have a location associated with them, only a direction. This is in turn becuase we only get parallel light rays that are like directional lights in nature when the light source is very far away (e.g. the sun).

Omit light_specular(r, g, b)

The specular component should be defined by the color of the light (which is defined when creating the light) and the material it hits. Therefore, I found it unnecessary to include another function to override the specular color of a light specifically. This being said, if there is a demand, this function can still be implemented without too much work.
Update: During the first meeting with my mentors we decided to still implement this feature for better compatibility with the Processing Language.

Roadmap

• normal_material()
• basic_material(r, g, b)
• ambient(r, g, b)
• emissive(r,g,b)/diffuse(r, g, b)
• shininess(p)
• specular(r, g, b)
• blinn_phong_material(r, g, b)
• lights()
• ambient_light(r, g, b)
• directional_light(r, g, b, x, y, z)
• point_light(r, g, b, x, y, z)
• light_falloff(constant, linear, quadratic)
• light_specular(r, g, b)
• Document new APIs (to be done progressively while implementing them)
• Port tutorial from the Processing Language to demonstrate new features

[tushar5526]

The lights mentioned in this issue will be described by custom structs and passed to fragment shaders in an array of such structs. Then, the blinn-phong shader will be able to loop through them and add the contribution of each light to the final result.

Recently, I discovered that vispy do not support structs in fragment shaders. So if you are planning to have a fragment shader like this.

struct Light{
 vec3 ambient;
 vec3 diffuse;
 .... (other things)
};
uniform Light light;

void main()
{
  //main function
}

Then, the following is not gonna work and you may have to use gl backend in vispy (I tried doing that, things are working fine but with a lot of headache ) .

self.program['light.ambient'] = [1, 0, 1]

[ziyaointl]

@tushar5526 Another potential solution is to use separate variables for each field in Light, and if we need more than one light, have an array for each one of the fields.
For example:

uniform vec3 ambient[64]
uniform vec3 directional_direction[64]
uniform vec3 directional_color[64]
uniform uint ambient_lights // Number of ambient lights
uniform uint directional_lights // Number of directional lights

Definitely not as clean as structs but I think this should give us the same functionality.

[ziyaointl]

Sneak peak for the Blinn-Phong shader :D

I'll open a PR as soon as the two pending ones are merged (due to code dependencies)