Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HDF5] Update - 1 - Refactor HDF5File #11473

Merged
merged 64 commits into from
Aug 22, 2023
Merged

Conversation

sunethwarna
Copy link
Member

@sunethwarna sunethwarna commented Aug 13, 2023

📝 Description
This PR refactors HDF5File. Followings changes/updates were applied.

  1. HDF5File ctor now requires DataCommunicator. The python ctor still has a ctor without DataCommunicator, in this case a warning is thrown. This is planned to be removed once the python side of the HDF5Application is updated.
  2. All the raw mpi calls are now done via DataCommunicator.
  3. HDF5FileSerial and HDF5FileParallel are removed. Everything is done now in HDF5File
  4. Extended the attribute and dataset interfaces for data types array_1d<double, 4>, array_1d<double, 6>, array_1d<double, 9>
  5. The HDF5FileParallel version throws an error if the dataset is existing when writing, but HDF5SerialFile retireves the existing and changes it. The behaviour is now uniform in mpi and openmp.
  6. The tests and occurances of HDFFile updated.

Requires #11469

This closes #10728

🆕 Changelog

  • As above.

@sunethwarna sunethwarna marked this pull request as ready for review August 14, 2023 20:50
@sunethwarna sunethwarna requested review from a team as code owners August 14, 2023 20:50
Copy link
Member

@philbucher philbucher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment on lines +97 to +101
if(USE_MPI)
target_link_libraries(KratosHDF5Core PUBLIC KratosCore KratosMPICore PRIVATE ${HDF5_C_LIBRARIES} ${KRATOS_HDF5_APPLICATION_EXTRA_LIBS})
else(USE_MPI)
target_link_libraries(KratosHDF5Core PUBLIC KratosCore PRIVATE ${HDF5_C_LIBRARIES} ${KRATOS_HDF5_APPLICATION_EXTRA_LIBS})
endif(USE_MPI)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend splitting this into a separate MPI-Extension, same as the other apps

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be ok if I do it in a seperate PR? I think that will be cleaner, because there are some other files which also needs to go into this mpi-extension.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok for me

applications/HDF5Application/tests/test_utils.cpp Outdated Show resolved Hide resolved
applications/HDF5Application/tests/test_utils.cpp Outdated Show resolved Hide resolved
private:
///@name Member Variables
///@{

DataCommunicator const * mpDataCommunicator;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not const &?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm the one to blame here, but I'll stand by my opinion. Storing a const member always messes up move and copy semantics. In this case, copy construction/assignment and move assignment doesn't make sense, but I don't see why move construction should be forbidden.

@@ -22,67 +49,9 @@ bool IsPath(const std::string& rPath)
return regex_match(rPath, std::regex("(/[\\w\\(\\)]+)+"));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps this can be done better with std::filesystem

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure how to do this with std::filesystem since these paths are not actual file paths, but hdf5 paths within one hdf5 file.

Copy link
Contributor

@matekelemen matekelemen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll review this in chunks and start with hdf5_file.h

applications/HDF5Application/custom_io/hdf5_file.h Outdated Show resolved Hide resolved
applications/HDF5Application/custom_io/hdf5_file.h Outdated Show resolved Hide resolved
applications/HDF5Application/custom_io/hdf5_file.h Outdated Show resolved Hide resolved
applications/HDF5Application/custom_io/hdf5_file.h Outdated Show resolved Hide resolved
applications/HDF5Application/custom_io/hdf5_file.h Outdated Show resolved Hide resolved
applications/HDF5Application/custom_io/hdf5_file.h Outdated Show resolved Hide resolved
applications/HDF5Application/custom_io/hdf5_file.h Outdated Show resolved Hide resolved
Copy link
Contributor

@matekelemen matekelemen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hdf5_file.cpp

applications/HDF5Application/custom_io/hdf5_file.cpp Outdated Show resolved Hide resolved
applications/HDF5Application/custom_io/hdf5_file.cpp Outdated Show resolved Hide resolved
applications/HDF5Application/custom_io/hdf5_file.cpp Outdated Show resolved Hide resolved
applications/HDF5Application/custom_io/hdf5_file.cpp Outdated Show resolved Hide resolved
applications/HDF5Application/custom_io/hdf5_file.cpp Outdated Show resolved Hide resolved
applications/HDF5Application/custom_io/hdf5_file.cpp Outdated Show resolved Hide resolved
applications/HDF5Application/custom_io/hdf5_file.cpp Outdated Show resolved Hide resolved
applications/HDF5Application/custom_io/hdf5_file.cpp Outdated Show resolved Hide resolved
applications/HDF5Application/custom_io/hdf5_file.cpp Outdated Show resolved Hide resolved
@sunethwarna
Copy link
Member Author

@matekelemen @philbucher I applied your suggestions. Could you have a look? Thanks :D

@sunethwarna sunethwarna changed the title [HDF5] Refactor HDF5File [HDF5] Update - 1 - Refactor HDF5File Aug 19, 2023
@philbucher
Copy link
Member

LGTM

@sunethwarna
Copy link
Member Author

@matekelemen I changed to std::array, and replied to others. Could you take a look?

@sunethwarna sunethwarna merged commit 54cd860 into master Aug 22, 2023
@sunethwarna sunethwarna deleted the hdf5/update/hdf5_file branch August 22, 2023 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Applications C++ Python Refactor When code is moved or rewrote keeping the same behavior
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[HDF5] Replace Raw MPI Calls
3 participants