Changing terrain provider moves billboard to incorrect location

[OmarShehata]

This came up on the forum. Here's a Sandcastle example. To reproduce:

1. Zoom in close enough to the billboards and tilt the camera until you see them curve

2. Select "delete billboards"
3. Select "change terrain provider"
4. Select "add billboards"
5. Select "change terrain provider"
6. Observe that they're now in a different position:

This may be just another manifestation of #4776 but I couldn't dig deep enough to figure out what the underlying issue was.

[raycrummey]

One detail that may be relevant. If there weren't different pixel offsets for each billboard in this sandcastle, the billboards would render on top of other billboards in the set.

So they end up rendering on top of each other when this issue happens.

[bn-dignitas]

So many billboard tickets out there, but I think this is the most relevant to what I'm seeing and I wanted to add my observations. So here it goes...

Stationary billboards do not clamp correctly when entities are removed just before terrain is reloaded and entities are added while terrain is still loading.

Note that in the example code(see link below), all entities are removed, but even if one or more entities are removed and new ones are added you can see the issue.

Observations

• calling removeAll seems to work better vs individually removing entities
  • we don't have the option to call removeAll in our use case
• This does not happen when disabling the reuse of "unused" billboards in EntityCluster
  • I modified code so that new billboards are always created and deleted on removal
  • Though this makes the problem go away, I do not think this is the exact underlying cause since it just prevents updateHeight callbacks on reused billboard objects

It looks like the QuadtreePrimitive customData list keeps growing and some stale height callbacks are being made with old coordinates. This seems to be in agreement with @lilleyse comment in #8650

For my case I can delay loading of entities (by timer or wait until all tiles are loaded), but it really isn't optimal.

Note that we have been seeing this issue since at least 1.57, even possibly as far back as 1.51, but have not had time to debug.

Middle click will reload terrain and demonstrate issue
https://sandcastle.cesium.com/#c=jVVtb+M2DP4rRIABys6nuC06oE1SrGuKtUDuerhkuy8BAsWmE6GyFEhyguyQ/z4qfkmal2EfbFkU+ZB8RNIrYWElcY0W+qBxDU/oZJHzv3cyNmklu/2T0V5IjXbSancnekVWiZLJ+4vQqTq2HSUWUY+WIsHnFWpfKbHSD3cJauSJIBQXwA6BuEP/qpeFf0y8NJqxpXEyfLWh/wA/JxrAIumMAgYL1tvoktvxZon8y+tgMHyeDt5+fK0D92gt5fLNmpVM0ZJ0orNC7xwSujIiHZcqjzp91p78o2Pt2vvB+S4AAPIrPO41u9DpwIyE7y4ch4cEKSqxAW9Aoa9jgKxQagMBstRaS52adWBhLHM0hWcfwSO4juN4l/flsOtYjxKlS6pu4JQBuHBENhW7ZRw/jFUfsv8YxjETIY4dQ0Y7D84L64fCE+jNb91juZ6T/PNVXJ5kxgILTMl+3AUJPbim5dOnKjXYsShzMUeymrQ47+w2rpOJRCrpN3wus0mrWyrLDJiEX+Aa+v0+xA0InIUoM54OzdxMzQot3Rtf6nmDti2XijGssuUiTdkeN70Hyb0ZeSv1nLWj+qAu6Pua2CdKnr6EvuGZNfkA51TGjtWkRA1tn5n8NeZxfNXeo82kUjMjLHlrXFdJ3ZdLtBcvUM4X/jtmaFEn2ETw8lHOn4aPX75Nx2/TP7+//fV10CBsq69tWfbb0yLcd2YZDpW9dK5ASA060MaDSZLCwlr6BannxO6jUrBy5YaYAqMRZpuwlADHLDdWVfOFJ9RCrbCv8sZkJVRBi6M5g5VVc0Z19iySBWMk2c+Y08st3f6xeU2DJpdpxUEdRHifNu+laRLB7UEXj2j2JcJ5hXxHIaXQcFozWcWTiByt4JnavJgcWdw+7N0LHAWN0ADVbaeY0SBP2VGnt9tHqZ9OgnNj49yUKXTloy4T+I+x2j2b3diwnyk6L7X4P80SxkYEN7f87u7ulmZkYDeCtLCVdbwtidjSuxW1es5vFD7Udf27zJfGeiisYjQFPOZLRWPMdWZF8o6eJ861q2R7nUPTXipX1On9M39J+j0K5+gkTPiR/AcnrYdeh/RPTAMxVPhv5aAJaourh2Ep5Jz3OrQ9b+mNUTNhj5D/BQ

[bn-dignitas]

Note that the delay in loading entities does not always help and also I noticed that the customData length continues to grow each terrain reload so that is a bit worrisome.