Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve startup time of applications #682

Merged
merged 7 commits into from
Nov 17, 2023
Merged

Conversation

GernotMaier
Copy link
Contributor

@GernotMaier GernotMaier commented Nov 16, 2023

Applications are very slow in starting up, see issue #681 .

This is mostly related to imports and this pull request is start to optimize this:

  • moved hd5-related functions out of utils/general.py into a separate module (io_hdf5)
  • for clarity, moved io_handler and io_hdf5 into a new directory simtools.io_operations (most changes of this PR are related to the change of the import statement; note that I have in an earlier version this was simtools.io, but I realised that io is in conflict with the python io package)

This improves the speed of calling e.g., python simtools/applications/validate_schema_files.py --help by a factor of 4!

Copy link

codecov bot commented Nov 16, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (7fa1f84) 99.30% compared to head (550c286) 99.30%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #682   +/-   ##
=======================================
  Coverage   99.30%   99.30%           
=======================================
  Files          39       40    +1     
  Lines        3573     3575    +2     
=======================================
+ Hits         3548     3550    +2     
  Misses         25       25           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@VictorBarbosaMartins VictorBarbosaMartins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these updates, it is improving also in the direction of #665.
I left only one comment/suggestion.

_logger = logging.getLogger(__name__)


def fill_hdf5_table(hist, x_bin_edges, y_bin_edges, x_label, y_label, meta_data):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is good to have all the hdf5 functions together, but this function is not actually an io as it does not deal with input and output. Instead, if fills the table. Maybe we should call the module hdf5_handler instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - this is a good suggestion. I've renamed io_hdf5 to hd5_handler.

Waiting for the tests and will merge when successful.

@GernotMaier GernotMaier merged commit c779140 into main Nov 17, 2023
9 checks passed
@GernotMaier GernotMaier deleted the separate_ctapipe_imports branch November 17, 2023 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Applications are painfully slow to even start.
2 participants