Optimizing HROM Setup Performance #12270
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SADPR
assigned SADPR, roigcarlo, rubenzorrilla, Rbravo555, Marco1410, NicolasSR and RiccardoRossi
Rbravo555
reviewed
Apr 16, 2024
| @@ -167,7 +175,7 @@ def ModifyAfterSolverInitialize(self): | |||
| aux[j,i] = petrov_galerkin_nodal_modes[node_id][j][i].GetDouble() | |||
| node.SetValue(KratosROM.ROM_LEFT_BASIS, aux) | |||
|
|
|||
| elif self.rom_parameters["rom_format"].GetString() == "numpy": | |||
There was a problem hiding this comment.
let us name the options:
"numpy_svd", "numpy_rsvd", later on we could add: "dislib_tsqr_svd", "dask_rsvd", and so on
| @@ -0,0 +1,1463 @@ | |||
| Begin ModelPartData | |||
There was a problem hiding this comment.
Why arent we refering to the original file, therefore avoiding copying it in the RomApp folder? I mean:
"input_filename" : "../../../../FluidDynamicsApplication/tests/CouetteFlowTest/couette_flow_test"
|
Not passing the tests. Is this related to the use of the model part from the CFDApp? |
|
as a TODO for future PRs, we should create an SVD class, so that we can move there the svds implemented in this PR in the hrom_training_untility.py |
Rbravo555
approved these changes
Apr 29, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces substantial improvements to the setup process for HROMs in Kratos. It replaces the previous dictionary-based storage for ROM elements and weights with a more efficient list-based approach. This change addresses performance bottlenecks identified during the handling of large datasets, where the former method was projected to take impractically long execution times (up to weeks for extensive cases like the motor setup; now, it takes a few seconds).
Changes
Replaced Data Structures
Old Implementation: Used a dictionary to store weights and indices for ROM elements and conditions. This method was not only memory-intensive but also significantly slower for large data sets due to the overhead associated with dictionary operations in Python and C++.
New Implementation: Shifts to using lists and numpy arrays that streamline the data handling and improve the execution speed. The new method directly loads and processes
.npyfiles for element and condition IDs, and weights into numpy arrays, facilitating faster access and operations.Additional Enhancements
Minor Bug Fixes: Addressed a few default minor bugs that were affecting the efficiency and correctness of the HROM setups.
SVD Type Setting: Introduced a new setting (
svd_type) that allows users to choose between using the randomized SVD (rSVD) and numpy's SVD.HROM Creation Test: Added a test for the HROM creation process, an essential part of our testing suite that was previously missing.