Optimize global watershed delineation for large watersheds

[aufdenkampe]

The first step of watershed delineation is extremely fast, due to our implementation of the Modified Nested Set Index.

The second step -- dissolving the basin geometries for the delineated watershed in a custom boundary -- is increasingly slow with increasing watershed size and can lead to memory issues for very large watersheds.

@ptomasula and I anticipated this, and developed an additional set of performance optimizations to address it. As time and funding for our 2024 effort was winding down, we all collectively decided on a wait and see approach to determine if implementing the optimizations were necessary. We now know that they are! See:

• Investigate Postgres Out of Memory Errors in Production #3667

Our two sets of optimizations are:

1. Pre-dissolve basins into Hydrologic Units (HUs) of ~200 basins.

As described in:

• Dissolving Logic for Upstream Watershed Boundary Given LINKNO/streamID  global-hydrography#7
• Develop Pre-dissolved Basins Layer for Faster Delineations global-hydrography#9

This pre-dissolve step has performance benefits (2-4x speedup) even if the pre-dissolve is done on-the-fly, and the implementation has a much, much smaller memory footprint! See WikiWatershed/global-hydrography#9 (comment)

2. Simpifly geometries for Hydrologic Units

As described in:

• Create geometry simplification pipeline for basins global-hydrography#11

When the pre-dissolved HUs are saved as a separate layer, and especially if they are simplified, then the performance benefits are likely to be an additional 3-4x. See WikiWatershed/global-hydrography#7 (comment) and WikiWatershed/global-hydrography#11 (comment).

Last, the new global HUs could serve as a mechanism to cache watershed analysis results, similar to how we cache results for HUC12 polygons within CONUS. These 200-basin HUs from TDX-Hydro are between the size of a HUC10 and a HUC08.