Separate the Model and View Matrices

[JetStarBlues]

Most appropriate sub-area of p5.js?

• WebGL

Feature enhancement details:

Currently in the source code, the model and view matrices are combined into one (uMVMatrix). Typically, these two matrices are kept distinct from each other. The model matrix used only for translating / rotating / scaling the model. The view matrix used only for translating / rotating the camera.

One consequence of this, is that the model matrix is not available for use in shader programs. Currently, the view matrix and modelView matrix are passed to shaders as follows:

p5.Shader.prototype._setMatrixUniforms = function() {
  ...
  this.setUniform('uViewMatrix', this._renderer._curCamera.cameraMatrix.mat4);
  this.setUniform('uModelViewMatrix', this._renderer.uMVMatrix.mat4);
  ...
};

If the two are kept separate in the code, then providing the model matrix to shader programs becomes trivial.

p5.Shader.prototype._setMatrixUniforms = function() {
  ...
  const modelMatrix = this._renderer.modelMatrix;  // does not currently exist
  const viewMatrix = this._renderer._curCamera.cameraMatrix;
  const modelViewMatrix = (modelMatrix.copy()).mult(viewMatrix);

  this.setUniform('uModelMatrix', modelMatrix.mat4);
  this.setUniform('uViewMatrix', viewMatrix.mat4);
  this.setUniform('uModelViewMatrix', modelViewMatrix.mat4);
  ...
};

The separation of concerns will also make maintaining and/or expanding functionality in the source code simpler.

How would this new feature help increase access to p5.js?

Exposing the model matrix as a uniform available for users to use in their shader programs can make p5 a better platform for people looking to learn about 3D graphics and shaders. It will make it possible to use p5 to follow along with the wider ecosystem of tutorials, books, and other educational material on computer graphics.

---

This issue is an extension of the conversation here.

[stalgiag]

I agree with this. I found it painful to work on AR for p5.xr without separated model and view matrices.

As a side note, I think that this would set us up to allow for more of an object-based approach to manipulating 3D objects. I have longed for the day that we could just do myBox.position(x, y, z);. I find this more intuitive. In my experience new coders working in 2D with p5 begin by using the drawing functions that indicate position in the function (circle(x,y,r)). Some of the WebGL functions work this way but many don't and instead rely on translate(). The compound effects of translation functions can be confusing at first.

Anyways, I am a bit off-topic here but separated model and view matrices would open up the door to thinking about differently scoped model matrices as well.

[BarneyWhiteman]

Unfortunately I don't think I have the technical knowledge to implement this, nor the familiarity with the P5 code base, but I'd just like to throw my support behind this feature as well. It would be really useful to get the world-space position of vertices inside the vertex shader for certain effects.

[deveshidwivedi]

Hello! I am trying to work on this. These are the changes made so far. I'd be grateful for any tips or suggestions to make things work. Thanks!

I request @davepagurek to please take a look.. Thank you!

[davepagurek]

Thanks for taking this on, it's looking good! I noticed a few little things to update:

• In src/webgl/p5.Camera.js it looks like you've updated it to use this.modelMatrix and this.viewMatrix in one spot. I think we don't need to update the names of the properties on the current camera (this.cameraMatrix and this.projMatrix still make sense to keep); it's just anything it sets on the renderer that might need to change. The nice thing about separating the model/view matrices is that now, when you switch cameras, you don't need to change the renderer's model matrix at all, you just need to replace its view matrix with the new camera's view matrix!
• When setting a framebuffer, after updating the renderer's view matrix, I believe you need to reset the renderer's model matrix to an identity matrix in order to reset it. I think calling target.resetMatrix() will do that.
• In resetMatrix, I'm not 100% sure but I think calling .set() with no args isn't supported yet as a way to reset it to an identity matrix. Maybe you could add a .reset() method to p5.Matrix that sets it back to the identiy matrix values found in the p5.Matrix constructor?
• When setting uNormalMatrix, I think you want to do the inverse transpose on modelViewMatrix to take the camera transform into account too.

Feel free to open a PR for easier reviewing/conversation!

[deveshidwivedi]

Sure, thank you!

[nakednous]

I've implemented a temporary hack for this using bindMatrices (source code available here). Check out the visualization of perspective transformation to NDC for an example. I'm also looking for advice on applying the shader there to scene lines in addition to just polygons?