Optimise USD layer writer diffs

[tomc-cinesite]

Summary

Optimise the USD layer writer where possible to allow production use of brute-force diffs to define variants.

User story

As a studio, we would like to make the authoring of variants more flexible, based on scene deltas. The current implementation prohibits this due to the time taken to build the diffs. It would be great if we could optimise these two typical scenarios:

1. Exporting lookdev layer for assets with very heavy objects e.g. heavy grooms. In our case, there is no difference between objects , but the diff is still taking very long time. One possible optimisation could perhaps be to check if hashes of objects are the same and skip the diff if it is the case.
2. Exporting lookdev layer for large assemblies. The assembly lookdev layers tend to have shaders at the top level of the scene hierarchy, but processing the diff between two huge scenes still takes lots of time. The hashing optimisation will hopefully make a difference for this scenario too, but the hashing would need to account for attributes and transforms as well.

[johnhaddon]

Will you be exporting a single frame, or an animation? I think the single frame case is relatively simple - we can just write a node that compares input hashes and outputs nothing if they match. But the animation case seems potentially trickier - I think we need to still write out objects when they are identical on a particular frame, if and only if they are different on a later frame. Not sure how to achieve that without doing a potentially expensive pre-pass over everything.

[alex-savenko-at-cinesite]

Single frames only. All our shaders are supposed to be handling animation internally i.e. by reading time/frame inside shaders

[alex-savenko-at-cinesite]

Another possible optimisation: do not generate time-samples if exporting single-frame. It is unlikely to speed up export much, but it will generate smaller, more efficient USD layers

[johnhaddon]

do not generate time-samples if exporting single-frame

We definitely want to do that, and we even want to do it when exporting an animation but finding a property to be constant. That will need to be implemented in IECoreUSD separately though, at which point USDLayerWriter would benefit from it automatically. How high in priority would you say that part is?

[johnhaddon]

Single frames only.

@tomc-cinesite, would this apply to your light rig exports too? Or would those be animated?

[tomc-cinesite]

@tomc-cinesite, would this apply to your light rig exports too? Or would those be animated?

Light rigs would be animated.

[alex-savenko-at-cinesite]

will need to be implemented in IECoreUSD separately though, at which point USDLayerWriter would benefit from it > automatically. How high in priority would you say that part is?

Very low priority. We optimise samples in our postprocess anyway