Stroke cap+join support for WebGL

[davepagurek]

Resolves #5790

Changes

• Adds support for strokeCap and strokeJoin in p5.RendererGL
• Updates the line drawing pipeline (generation of vertices and attributes, shaders) to handle caps + joins
• Updates the default join to ROUND for WebGL mode (I'll explanation below, let me know if this makes sense!)

Screenshots of the change:

Live demo here: https://editor.p5js.org/davepagurek/sketches/8SSUdaVhi

Explanation

Design goals

• Support the stroke caps/joins from 2D mode
• Have caps/joins look correct from all angles
• Have geometry not change based on cap/join mode, so that it can be cached
• Use as little new data as possible
• Not be significantly slower

Design

To accomplish all this, I've chosen the following design:

• Similar to the existing design, all line vertex positions are calculated in the fragment shader to account for the view direction correctly
• One quad is added for every cap and join side
• The following vertex attributes are used:
  • Straight line segments
    • aTangentIn = aTangentOut = the direction of the segment
    • aSide is either 1 or -1, indicating what side of the centerline it's on
  • Joins
    • aTangentIn is the tangent of the incoming segment
    • aTangentOut is the tangent of the outgoing segment
    • aSide will be plus or minus 1, 2, or 3. The sign represents the side of the centerline; 1 indicates the vertex on the incoming side of the join; 2 is the elbow of the join; 3 is the outgoing side
  • Caps
    • aTangentIn is the tangent going into the cap
    • aTangentOut is 0
    • aSide will be plus or minus 1 or 2, with 1 representing the side coming out of the segment, and 2 being the side projected out from the end of the segment
• The fragment shader then discards pixels based on the type of cap or join
  • ROUND: vCenter, the position of the centerline, is passed in from the vertex shader, and pixels greater than the radius away are discarded
  • BEVEL: pixels are discarded if they are father along the normal from the center than vMaxDist, a value provided by the vertex shader indicating the height of the bevel

Tradeoffs

Different default join mode

One thing to note is that MITER joins tend to look worse in WebGL mode than 2D mode. Whenever the join angle approaches 180 degrees, the stroke edges are so parallel that it takes a long distance for them to intersect into a join. This happens often with joins on the edge of 3D shapes, leading to little spikes being present:

I believe these will be unexpected to users and probably are not what they want at first. Because of this, I've opted to make ROUND joints be the default in WebGL. This does introduce an inconsistency between modes, so let me know if you think this is reasonable or not!

Antialiasing

Using discard in the fragment shader does not seem to produce as good antialiasing compared to lines produced with actual geometry. I believe this tradeoff is still worthwhile, as we are trading slightly better antialiasing for fully rounded joins instead of polylines, and faster rendering due to significantly fewer vertices (we'd need a full circle of vertices in every join otherwise.) For what it's worth, a user on the p5 discord mentioned that they thought the shader version looked better than a prototype polyline version, providing this screenshot for comparison:

Also, I'm using discard and not changing the alpha in the fragment shader is because the transparent and semitransparent pixels would still be written to the depth buffer, occluding any subsequently drawn shapes that are supposed to be behind it, which would make a bit of a halo around the stroke.

PR Checklist

• npm run lint passes
• [Inline documentation] is included / updated
• [Unit tests] are included / updated

[stalgiag]

Amazing! A quick scan of this had me really excited. My capacity is very limited for the remainder of this month so I am unable to do a detailed review. @AdilRabbani would you want to take a look as an initial reviewer? I am remembering that you worked on some aspects of line/stroke rendering in the past.

[aferriss]

This is great! For the anti-aliasing, would it be possible to draw the alpha using a smoothstep based on screen resolution instead? That should give a much cleaner line edge than discard.

I believe in some circumstances discard can actually decrease performance as well, so it may be good to avoid it if we can.

[davepagurek]

would it be possible to draw the alpha using a smoothstep based on screen resolution instead

It would be, although the issue with this is that the semi-transparent pixels would still register in the depth buffer, so if after drawing one shape, I translate further from the camera and then draw another, the semi transparent pixels won't blend with this new object. I hear there are order-independent transparency algorithms, do you have any experience with those? I haven't read enough into them to know yet how feasible they would be to integrate into p5.

I believe in some circumstances discard can actually decrease performance as well

Do you have any pointers for when this might be, at least so I can test the discard solution better? I'm leaning on discard right now to avoid having to create more vertices in order to get rounded corners (while also juggling the transparency issue mentioned above) and found discard to be faster than extra vertices at least, but I'd definitely be interested in testing this more thoroughly to feel more confident with any tradeoffs we decide to make.

[aferriss]

That's a good point about the depth testing, I don't have any experience with the order independent algorithms sadly.

If you do a search for "discard performance glsl" you'll see a few things. I believe it can degrade performance if your depth test would otherwise ignore pixels being processed. It's definitely niche!

https://gamedev.stackexchange.com/questions/40301/do-i-lose-gain-performance-for-discarding-pixels-even-if-i-dont-use-depth-testi

[aferriss]

Actually after playing around with your demo a bit, the anti-aliasing didn't bother me as much as I thought it would, so just ignore my previous comments!

However, I did notice some strange shifting of the line widths, especially visible on the sphere when you zoom in closely. Any idea what might be causing that to happen?

[davepagurek]

https://gamedev.stackexchange.com/questions/40301/do-i-lose-gain-performance-for-discarding-pixels-even-if-i-dont-use-depth-testi

Thanks for this link! It sounds like WebGL won't be able to early-discard fragments behind lines. I guess if we wanted to optimize this, we could split line rendering into two passes: first all the straight segments using a fragment shader with no discards, and then just with the joins and caps and a discard shader so that the bits that will never discard can use that optimization. I also want to read some more and see if it defers draws for as long as it can so that it can do that optimization on fragments from other draw calls vs if it only optimizes within the current draw (so in this case, discarding other fragments on the same line, which wouldn't be as much of a gain probably.)

However, I did notice some strange shifting of the line widths, especially visible on the sphere when you zoom in closely. Any idea what might be causing that to happen?

I noticed this happening on v1.4.2 as well, so I wonder if it's because of this line that puts line vertices slightly above faces? I wonder if maybe, if you keep rotating a face, there's an angle at which that scale value isn't enough to keep the line on top of the face.

[davepagurek]

It sounds like WebGL won't be able to early-discard fragments behind lines. I guess if we wanted to optimize this, we could split line rendering into two passes: first all the straight segments using a fragment shader with no discards, and then just with the joins and caps and a discard shader so that the bits that will never discard can use that optimization. I also want to read some more and see if it defers draws for as long as it can so that it can do that optimization on fragments from other draw calls vs if it only optimizes within the current draw (so in this case, discarding other fragments on the same line, which wouldn't be as much of a gain probably.)

Update on this! I did some reading, and have slightly more fully formed answers to the questions I raised.

• It seems like, yes, if nothing flushes the pipeline between calls, then the GL backend might wait to execute drawing commands^[1], so it has the ability to optimize fragments from multiple draw commands. This means that discard in the line shader has the potential to prevent anything that would be behind the line from being discarded (whether or not it actually does that optimization to begin with is hardware dependent.)
• Supporting alpha blending might have similar issues^[2], since that also would be dependent on the fragment below rendering.
• I wanted to see if there were other 3D renderers with 3D strokes with cap+join support that do it with shaders as opposed to tesselating via GPU (not really an option in WebGL without geometry shaders) or CPU, and found that mapbox-gl also does this^[3]. They rely on both setting alpha and discarding fragments^[4], but they also have much tighter control of when lines are drawn in their rendering pipeline. (I avoided using fragment shader alpha before since it writes to the depth buffer and might incorrectly occlude subsequently drawn shapes.)

Based on this:

• I think our best bet is to still use discard for now, as it's still faster than CPU-based tesselation or duplicating way more vertices on the CPU (to account for a lack of geometry shaders) to tesselate on the GPU
• We can optimize this more in the future by splitting cap+join triangles into a second shader pass so that we can draw line segment triangles with a simple shader with no discards
• ...but I think that probably isn't worth the code complexity yet? I'm open to other opinions here though!
• Performance will likely be affected most on mobile, so we should at least look out for that
  • Maybe we should consider giving users a way to opt into using a simpler line shader that omits caps+join support for the sake of performance

References:

1. https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/WebGL_best_practices#flush_when_expecting_results
2. https://stackoverflow.com/questions/8509051/is-discard-bad-for-program-performance-in-opengl
3. https://blog.mapbox.com/drawing-antialiased-lines-with-opengl-8766f34192dc
4. https://github.com/mapbox/mapbox-gl-js/blob/main/src/shaders/line.fragment.glsl

[davepagurek]

Updated now to support per-vertex stroke colors, added in #5915:

There's a little bit of z fighting that can be noticed around joints when using really thick strokes with different colours on either side of the join, but I think that case is uncommon enough that we can address it separately.

[Qianqianye]

Thanks @davepagurek for working on it. I think this PR is ready to be merged if it looks good to you @stalgiag @aferriss.

[stalgiag]

I noticed this happening on v1.4.2 as well, so I wonder if it's because of this line that puts line vertices slightly above faces? I wonder if maybe, if you keep rotating a face, there's an angle at which that scale value isn't enough to keep the line on top of the face.

I think this is the correct diagnosis as you can't see this issue in noFill() geometry. The relationship between stroke and face could probably use a better solution now that large stroke weights are beginning to look better with the changes from this PR.

I am personally good with this being merged. I wasn't able to do a detailed reading but I tested everything out and skimmed the changes. Everything looks great and I love the frag shader discard solution for the joins. I agree that a simplified shader could be an important addition down the road as I am seeing more people using p5 WebGL for mobile/VR work with a lot of geometry on screen simultaneously to the point where the caps/joins won't really be visible.

Thanks so much for this @davepagurek !

[davepagurek]

Thanks @stalgiag! I'm going to merge this in for now and make an issue to track the stroke jittering issue, and another to discuss mobile performance and potential paths forward for that.